Autofocus, autofocus within area, and auto configuration of beamforming microphone lobes with suppression

Abstract

An array microphone system and method are provided that can automatically focus and / or configure beamforming lobes in response to detected sound activity. The automatic focus and / or configuration of the beamforming lobes can be suppressed based on a remote far-end audio signal. The quality of coverage of an audio source in an environment can be improved by ensuring that the beamforming lobes optimally pick up the audio source even if the audio source has moved and changed position.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 821,800 filed on March 21, 2019, U.S. Provisional Patent Application No. 62 / 855,187 filed on May 31, 2019, and U.S. Provisional Patent Application No. 62 / 971,648 filed on February 7, 2020. The contents of each application are fully incorporated herein by reference in their entirety. Technical Field

[0003] The present application generally relates to an array microphone with automatic focus and configuration of beamforming microphone lobes. In particular, the present application relates to an array microphone that adjusts the focus and configuration of the beamforming microphone lobes based on detection of sound activity after the lobes have been initially configured, and allows the adjustment of the focus and configuration of the beamforming microphone lobes to be suppressed based on a remote far-end audio signal. Background Art

[0004] Meeting environments, such as conference rooms, boardrooms, video conferencing applications, etc., may involve the use of microphones to capture sounds from various audio sources active in such environments. For example, such audio sources may include people who are speaking. The captured sound may be transmitted to local listeners in the environment through amplified speakers (for sound reinforcement), and / or to other people far away from the environment (such as via television broadcasts and / or webcasts). The type of microphone and its configuration in a specific environment may depend on the location of the audio source, physical space requirements, aesthetics, room layout, and / or other considerations. For example, in some environments, the microphone may be configured on a table or podium near the audio source. For example, in other environments, the microphone may be mounted overhead to capture sound from the entire room. Therefore, microphones of various sizes, form factors, mounting options, and wiring options may be used to meet the needs of a specific environment.

[0005] Traditional microphones typically have fixed polar patterns and few manually selectable settings. To capture sound in a conference environment, many traditional microphones may be used simultaneously to capture audio sources within the environment. However, traditional microphones also tend to capture unwanted audio, such as room noise, echo, and other undesirable audio elements. Using many microphones exacerbates the capture of these unwanted noises.

[0006] An array microphone with multiple microphone elements may provide benefits such as a steerable coverage or pickup pattern (with one or more lobes), which allows the microphone to focus on desired audio sources and reject undesirable sounds such as room noise. The ability to steer the audio pickup pattern provides the following benefits: the accuracy of the microphone configuration may be reduced, and in this way, the array microphone is more forgiving. In addition, the array microphone provides the ability to pick up multiple audio sources with one array microphone or unit, again due to the ability to steer the pickup pattern.

[0007] However, in certain environments and situations, the position of the lobes of the pickup pattern of the array microphone may not be optimal. For example, the audio source initially detected by the lobes may move and change position. In this case, the lobes may not optimally pick up the audio source in its new position.

[0008] Therefore, there is an opportunity for array microphones to address these issues. More specifically, there is an opportunity for array microphones to automatically focus and / or configure beamforming microphone lobes based on detection of sound activity after the lobes have been initially configured, while also being able to suppress the focus and / or configuration of the beamforming microphone lobes based on distant far-end audio signals, which can result in higher quality sound capture and better coverage of the environment. Summary of the invention

[0009] The present invention aims to solve the above-mentioned problems by providing an array microphone system and method, which is designed to, among other things: (1) automatically focus the beamforming lobes of the array microphone in response to the detection of sound activity after the lobes have been initially configured; (2) automatically configure the beamforming lobes of the array microphone in response to the detection of sound activity; (3) automatically focus the lobes within the lobe area in response to the detection of sound activity after the beamforming lobes of the array microphone have been initially configured; and (4) suppress or limit the automatic focus or automatic configuration of the beamforming lobes of the array microphone based on the activity of a remote far-end audio signal.

[0010] In one embodiment, when new sound activity is detected at new coordinates substantially near the initial coordinates, the beamforming lobe that was positioned at the initial coordinates may be focused by moving the lobe to the new coordinates.

[0011] In another embodiment, the beamforming lobe may be configured or moved to new coordinates when new sound activity is detected at the new coordinates.

[0012] In yet another embodiment, when new sound activity is detected at the new coordinates, the beamforming lobe that has been positioned at the initial position may be focused by moving the lobe, but confined within the lobe area.

[0013] In another embodiment, when the activity of the far-end audio signal exceeds a predetermined threshold, the movement or configuration of the beamforming lobe may be suppressed or limited.

[0014] These and other embodiments and various arrangements and aspects will become apparent and more fully understood from the following detailed description and accompanying drawings, which set forth illustrative embodiments that are indicative of the various ways in which the principles of the invention may be employed. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a schematic diagram of an array microphone with auto-focusing beamforming lobes in response to detection of voice activity in accordance with some embodiments.

[0016] Figure 2 is a flow chart illustrating operation for auto-focusing a beamforming lobe, according to some embodiments.

[0017] Figure 3 A flow chart illustrating operations for auto-focusing of beamforming lobes utilizing a cost functional, according to some embodiments.

[0018] Figure 4 Schematic diagram of a beamforming lobe having an array microphone that automatically configures in response to detection of voice activity according to some embodiments.

[0019] Figure 5 Flowchart illustrating operations for automatically configuring beamforming lobes according to some embodiments.

[0020] Figure 6 A flow chart illustrating operations for finding a lobe near detected sound activity, according to some embodiments.

[0021] Figure 7 is an exemplary depiction of a microphone having a beamforming lobe within a lobe region in accordance with some embodiments.

[0022] Figure 8 A flow chart illustrating operations for auto-focusing a beamforming lobe within a lobe region in accordance with some embodiments.

[0023] Fig. 9 A flow chart illustrating operations for determining whether detected acoustic activity is within an apparent radius of a lobe, according to some embodiments.

[0024] Fig.10 is an exemplary depiction of an array microphone having beamforming lobes within a lobe region and illustrating the apparent radius of the lobes in accordance with some embodiments.

[0025] Fig.11A flow chart illustrating operations for determining movement of a flap within a radius of movement of the flap, according to some embodiments.

[0026] Fig.12 is an exemplary depiction of an array microphone having beamforming lobes within a lobe region and illustrating the movement radius of the lobes in accordance with some embodiments.

[0027] Fig.13 is an exemplary depiction of an array microphone having beamforming lobes within lobe regions and showing boundary pads between lobe regions in accordance with some embodiments.

[0028] Fig.14 Flowchart illustrating operations for limiting flap movement based on boundary pads between flap regions, according to some embodiments.

[0029] Fig.15 is an exemplary depiction of an array microphone having beamforming lobes within regions and demonstrating movement of the lobes based on boundary pads between regions in accordance with some embodiments.

[0030] Fig.16 Schematic diagram of an array microphone with auto-focusing beamforming lobes in response to detection of voice activity and suppressing auto-focus based on a far-end audio signal, according to some embodiments.

[0031] Fig.17 Schematic diagram of an array microphone having beamforming lobes that automatically configure an array microphone in response to detection of voice activity and suppressing automatic configuration based on a far-end audio signal, according to some embodiments.

[0032] Fig.18 Flowchart illustrating operations for automatically adjusting beamforming lobes of an array microphone based on remote far-end audio signal suppression in accordance with some embodiments.

[0033] Fig.19 Schematic diagram of an array microphone with automatic configuration of beamforming lobes of the array microphone in response to detection of voice activity and active detection of voice activity according to some embodiments.

[0034] Fig. 20 A flow diagram illustrating operations for automatically configuring beamforming lobes, the operations including activity detection of voice activity, according to some embodiments. DETAILED DESCRIPTION

[0035] The following description describes, illustrates and illustrates one or more specific embodiments of the present invention according to the principles of the present invention. This description is not provided to limit the present invention to the embodiments described herein, but to disclose and teach the principles of the present invention in a manner that enables those skilled in the art to understand these principles and, with the understanding, to apply them to practice not only the embodiments described herein but also other embodiments that are thought of according to these principles. The scope of the present invention is intended to cover all such embodiments that may fall within the scope of the appended claims, both literally and under the doctrine of equivalents.

[0036] It should be noted that in the specification and drawings, similar or substantially similar elements may be marked with the same reference numerals. However, sometimes these elements may be marked with different numbers, such as in situations where this marking helps to describe more clearly. In addition, the drawings set forth herein are not necessarily drawn to scale, and in some cases, the scale may be enlarged to more clearly depict certain features. Such marking and drawing conventions do not necessarily imply a potential substantive purpose. As described above, this specification is intended to be interpreted as a whole and in accordance with the principles of the present invention as taught herein and understood by those skilled in the art.

[0037] The array microphone systems and methods described herein can achieve automatic focusing and configuration of beamforming lobes in response to detection of sound activity, and allow the focus and configuration of beamforming lobes to be suppressed based on remote far-end audio signals. In an embodiment, the array microphone may include multiple microphone elements, an audio activity locator, a lobe autofocuser, a database, and a beamformer. The audio activity locator can detect the coordinates and confidence score of new sound activity, and the lobe autofocuser can determine whether there is a previously configured lobe near the new sound activity. If such a lobe exists and the confidence score of the new sound activity is greater than the confidence score of the lobe, the lobe autofocuser can transmit the new coordinates to the beamformer so that the lobe moves to the new coordinates. In these embodiments, the position of the lobe can be improved, and the latest position of the audio source inside and near the lobe can be automatically focused, while also preventing the lobe from overlapping, pointing in an undesirable direction (e.g., toward unwanted noise), and / or moving too suddenly.

[0038] In other embodiments, the array microphone may include a plurality of microphone elements, an audio activity locator, a lobe autoconfigurator, a database, and a beamformer. The audio activity locator may detect the coordinates of a new sound activity, and the lobe autoconfigurator may determine whether a lobe exists near the new sound activity. If such a lobe does not exist, the lobe autoconfigurator may transmit the new coordinates to the beamformer so that an inactive lobe is configured at the new coordinates, or so that an existing lobe is moved to the new coordinates. In these embodiments, the set of active lobes of the array microphone may point to the latest sound activity in the coverage area of ​​the array microphone.

[0039] In other embodiments, the audio activity locator may detect the coordinates and confidence score of new sound activity, and if the confidence score of the new sound activity is greater than a threshold, the lobe autofocuser may identify the lobe region to which the new sound activity belongs. In the identified lobe region, if the coordinates are within the appearance radius of the current coordinates of the lobe (i.e., the three-dimensional region of space around the current coordinates of the lobe in which the new sound activity can be considered), then the previously configured lobe may be moved. The movement of the lobe in the lobe region may be limited to the movement radius of the current coordinates of the lobe, i.e., the maximum distance the lobe is allowed to move in three-dimensional space, and / or limited to the outside of the boundary pad between lobe regions, i.e., how close the lobe can move to the boundary between lobe regions. In these embodiments, the position of the lobe can be improved, and autofocus can be focused on the latest position of the audio source inside the lobe region associated with the lobe, while also preventing the lobe from overlapping, pointing in an undesirable direction (e.g., toward unwanted noise), and / or moving too suddenly.

[0040] In other embodiments, the activity detector may receive a remote audio signal, such as from a far end. The sound of the remote audio signal may be played in a local environment, such as on a speaker in a conference room. If the activity of the remote audio signal exceeds a predetermined threshold, the occurrence of automatic adjustment (i.e., focus and / or configuration) of the beamforming lobe may be suppressed. For example, the activity of the remote audio signal may be measured by the energy level of the remote audio signal. In this example, the energy level of the remote audio signal may exceed a predetermined threshold when there is a certain level of voice or speech contained in the remote audio signal. In this case, it may be desirable to prevent automatic adjustment of the beamforming lobe so that the lobe is not oriented to pick up sound from the remote audio signal, such as played in the local environment. However, if the energy level of the remote audio signal does not exceed the predetermined threshold, automatic adjustment of the beamforming lobe may be performed. Automatic adjustment of the beamforming lobe may include automatic focus and / or configuration of the lobe, such as described herein. In these embodiments, the position of the lobe may be improved and automatically focused and / or configured when the activity of the remote audio signal does not exceed a predetermined threshold, and automatically focused and / or configured may be suppressed or limited when the activity of the remote audio signal exceeds a predetermined threshold.

[0041] By using the systems and methods herein, the quality of coverage of an audio source in an environment can be improved by, for example, ensuring that the beamforming lobe optimally picks up the audio source even if the audio source has moved from an initial location and changed position. For example, the quality of coverage of an audio source in an environment can also be improved by reducing the likelihood that the beamforming lobe is deployed (e.g., focused or configured) to pick up undesired sounds (such as speech, voice, or other noise from the far end).

[0042] Figure 1 and 4Schematic diagram of an array microphone 100, 400 that can detect sounds from audio sources of various frequencies. The array microphone 100, 400 may be used in a conference room or boardroom, for example, where the audio source may be one or more human speakers. Other sounds that may be undesirable may be present in the environment, such as noise from ventilation equipment, other people, audio / visual equipment, electronic equipment, etc. In a typical case, the audio source may be sitting in a chair at a table, although other configurations and arrangements of the audio source are contemplated and possible.

[0043] Array microphones 100, 400 may be arranged on or in a table, podium, tabletop, wall, ceiling, etc., so that sounds from an audio source, such as voices spoken by a human speaker, can be detected and captured. Array microphones 100, 400 may include, for example, any number of microphone elements 102a, 102b, ..., 102zz, 402a, 402b, ..., 402zz, and may be capable of forming a plurality of pickup patterns with lobes so that sounds from an audio source can be detected and captured. Any appropriate number of microphone elements 102, 402 is possible and contemplated.

[0044] Each of the microphone elements 102, 402 in the array microphone 100, 400 can detect sound and convert the sound into an analog audio signal. Components in the array microphone 100, 400, such as analog-to-digital converters, processors, and / or other components, can process the analog audio signals and ultimately generate one or more digital audio output signals. In some embodiments, the digital audio output signals may conform to the Dante standard for transmitting audio over Ethernet, or may conform to another standard and / or transmission protocol. In an embodiment, each of the microphone elements 102, 402 in the array microphone 100, 400 can detect sound and convert the sound into a digital audio signal.

[0045] The beamformer 170, 470 in the array microphone 100, 400 may form one or more pickup patterns based on the audio signals of the microphone elements 102, 402. The beamformer 170, 470 may generate digital output signals 190a, 190b, 190c, ..., 190z, 490a, 490b, 490c, ..., 490z corresponding to each pickup pattern. The pickup pattern may be composed of one or more lobes (e.g., a main lobe, a side lobe, and a rear lobe). In other embodiments, the microphone elements 102, 402 in the array microphone 100, 400 may output analog audio signals so that other components and devices (e.g., a processor, a mixer, a recorder, an amplifier, etc.) outside the array microphone 100, 400 may process the analog audio signals.

[0046] Automatically focusing the beamforming lobe in response to detection of voice activity Figure 1The array microphone 100 may include microphone elements 102; an audio activity locator 150 in wired or wireless communication with the microphone elements 102; a flap autofocuser 160 in wired or wireless communication with the audio activity locator 150; a beamformer 170 in wired or wireless communication with the microphone elements 102 and the flap autofocuser 160; and a database 180 in wired or wireless communication with the flap autofocuser 160. These components will be described in more detail below.

[0047] Automatic configuration of beamforming lobes in response to detection of voice activity Figure 4 The array microphone 400 may include microphone elements 402; an audio activity locator 450 in wired or wireless communication with the microphone elements 402; a flap autoconfigurator 460 in wired or wireless communication with the audio activity locator 450; a beamformer 470 in wired or wireless communication with the microphone elements 402 and the flap autoconfigurator 460; and a database 480 in wired or wireless communication with the flap autoconfigurator 460. These components will be described in more detail below.

[0048] In embodiments, the array microphone 100, 400 may include other components that work with the audio activity locator 150, 450 and / or the beamformer 170, 470, such as an echo canceller or an automatic mixer. For example, as described herein, when a lobe is moved to a new coordinate in response to detecting new sound activity, information from the lobe movement may be used by the echo canceller to minimize echo during the movement and / or by the automatic mixer to improve its decision-making ability. As another example, the movement of the lobe may be influenced by the decision of the automatic mixer, such as allowing the lobe movement that the automatic mixer has identified as having relevant voice activity. The beamformer 170, 470 may be any suitable beamformer, such as a delay-sum beamformer or a minimum variation distortion-free response (MVDR) beamformer.

[0049] The various components included in the array microphones 100, 400 may be implemented using software that may be executed by one or more servers or computers, such as a computing device having a processor and memory, a graphics processing unit (GPU), and / or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASICs), programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.).

[0050] In some embodiments, microphone elements 102, 402 may be arranged into concentric rings and / or harmonic nesting. In some embodiments, microphone elements 102, 402 may be arranged to be generally symmetrical. In other embodiments, microphone elements 102, 402 may be arranged asymmetrically or in another arrangement. In other embodiments, for example, microphone elements 102, 402 may be arranged on a substrate, configured in a frame, or suspended individually. An embodiment of an array microphone is described in commonly assigned U.S. Patent No. 9,565,493, which is hereby incorporated herein by reference in its entirety. In an embodiment, microphone elements 102, 402 may be unidirectional microphones that are primarily sensitive in one direction. In other embodiments, microphone elements 102, 402 may have other directivity or polarity patterns, such as cardioid, subcardioid, or omnidirectional, as desired. Microphone elements 102, 402 may be any suitable type of sensor that can detect sound from an audio source and convert the sound into an electrical audio signal. In one embodiment, the microphone elements 102, 402 may be micro-electromechanical systems (MEMS) microphones. In other embodiments, the microphone elements 102, 402 may be condenser microphones, balanced armature microphones, electret microphones, dynamic microphones, and / or other types of microphones. In an embodiment, the microphone elements 102, 402 may be arranged in one or two dimensions. The array microphones 100, 400 may be configured or mounted on a table, wall, ceiling, etc. and may be, for example, next to, below, or above a video monitor.

[0051] Figure 2 An embodiment of a process 200 for auto-focusing a previously configured beamforming lobe of an array microphone 100 is shown in FIG. The process 200 may be performed by the lobe autofocuser 160 so that the array microphone 100 may output one or more audio signals 180 from the array microphone 100, wherein the audio signal 180 may include sounds picked up by the beamforming lobe that focuses on new sound activity of an audio source. One or more processors and / or other processing components (e.g., analog-to-digital converters, encryption chips, etc.) internal or external to the array microphone 100 may perform any, some, or all steps of the process 200. One or more other types of components (e.g., memory, input and / or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be used in conjunction with processors and / or other processing components to perform any, some, or all steps of the process 200.

[0052] At step 202, coordinates and confidence scores corresponding to new sound activity may be received from the audio activity locator 150 at the flap autofocuser 160. The audio activity locator 150 may continuously scan the environment of the array microphone 100 to find new sound activity. The new sound activity discovered by the audio activity locator 150 may include a suitable audio source, such as a non-stationary human speaker. The coordinates of the new sound activity may be specific three-dimensional coordinates relative to the position of the array microphone 100, such as in Cartesian coordinates (i.e., x, y, z) or in spherical coordinates (i.e., radial distance / magnitude r, elevation angle θ (theta), azimuth angle θ (θ)). (phi)). For example, the confidence score of the new sound activity may represent the certainty of the coordinates and / or the quality of the sound activity. In embodiments, other suitable metrics related to the new sound activity may be received and utilized at step 202. It should be noted that Cartesian coordinates may be easily converted to spherical coordinates, and vice versa, as desired.

[0053] At step 204, the lobe autofocuser 160 may determine whether the coordinates of the new sound activity are near (i.e., in the vicinity of) an existing lobe. Whether the new sound activity is near an existing lobe may be based on the difference in azimuth and / or elevation of (1) the coordinates of the new sound activity and (2) the coordinates of the existing lobe relative to a predetermined threshold. The distance of the new sound activity from the microphone 100 may also affect the determination of whether the coordinates of the new sound activity are near an existing lobe. In some embodiments, the lobe autofocuser 160 may retrieve the coordinates of the existing lobe from the database 180 for use in step 204. Figure 6 An embodiment of determining whether the coordinates of new sound activity are near an existing lobe is described in more detail.

[0054] If the lobe autofocuser 160 determines at step 204 that the coordinates of the new sound activity are not near an existing lobe, the process 200 may end at step 210 and the position of the lobe of the array microphone 100 is not updated. In this case, the coordinates of the new sound activity may be considered outside the coverage area of ​​the array microphone 100, and thus the new sound activity may be ignored. However, if at step 204, the lobe autofocuser 160 determines that the coordinates of the new sound activity are near an existing lobe, the process 200 continues to step 206. In this case, the coordinates of the new sound activity may be considered an improved (i.e., more focused) position of the existing lobe.

[0055] At step 206, the flap autofocuser 160 may compare the confidence score of the new sound activity with the confidence score of the existing flap. In some embodiments, the flap autofocuser 160 may retrieve the confidence score of the existing flap from the database 180. If the flap autofocuser 160 determines at step 206 that the confidence score of the new sound activity is less than (i.e., inferior to) the confidence score of the existing flap, the process 200 may end at step 210 and the position in the flap of the array microphone 100 is not updated. However, if the flap autofocuser 160 determines at step 206 that the confidence score of the new sound activity is greater than or equal to (i.e., better than or more favorable to) the confidence score of the existing flap, the process 200 may continue to step 208. At step 208, the flap autofocuser 160 may transmit the coordinates of the new sound activity to the beamformer 170 so that the beamformer 170 may update the position of the existing flap to the new coordinates. In addition, the flap autofocuser 160 may store the new coordinates of the flap in the database 180.

[0056] In some embodiments, at step 208, petal autofocuser 160 may limit the movement of existing petals to prevent and / or minimize sudden changes in the position of the petals. For example, if a particular petal has recently moved within some recent period of time, then petal autofocuser 160 may not move the petal to the new coordinates. As another example, petal autofocuser 160 may not move a particular petal to the new coordinates if the new coordinates are too close to the current coordinates of the petal, too close to another petal, overlap with another petal, and / or are considered too far away from the petal to the existing position.

[0057] Process 200 may be continuously performed by array microphone 100 as audio activity locator 150 discovers new sound activity and provides the coordinates and confidence scores of the new sound activity to lobe autofocuser 160. For example, process 200 may be performed as an audio source (e.g., a human speaker) moves around a conference room so that one or more lobes can focus on the audio source to best pick up its sound.

[0058] Figure 3An embodiment of a process 300 for automatically focusing previously configured beamforming lobes of an array microphone 100 using a cost functional is shown in FIG. The process 300 may be performed by the lobe autofocuser 160 so that the array microphone 100 may output one or more audio signals 180, wherein the audio signal 180 may include sounds picked up by the beamforming lobes that focus on new sound activity of an audio source. One or more processors and / or other processing components (e.g., analog-to-digital converters, encryption chips, etc.) internal or external to the microphone array 100 may perform any, some, or all steps of the process 300. One or more other types of components (e.g., memory, input and / or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be used in conjunction with processors and / or other processing components to perform any, some, or all steps of the process 300.

[0059] Steps 302, 304, and 306 of process 300 of flap autofocuser 160 may be the same as described above. Figure 2 Steps 202, 204, and 206 of process 200 of are substantially the same. Specifically, coordinates and confidence scores corresponding to new sound activity may be received at the lobe autofocuser 160 from the audio activity locator 150. The lobe autofocuser 160 may determine whether the coordinates of the new sound activity are near (i.e., in the vicinity of) an existing lobe. If the coordinates of the new sound activity are not near an existing lobe (or if the confidence score of the new sound activity is less than the confidence score of the existing lobe), the process 300 may proceed to step 324, and the position of the lobe of the array microphone 100 is not updated. However, if at step 306, the lobe autofocuser 160 determines that the confidence score of the new sound activity is greater than (i.e., better than or more favorable to) the confidence score of the existing lobe, the process 300 may continue to step 308. In this case, the coordinates of the new sound activity may be considered as candidate positions to which the existing lobe is moved, and the cost functional of the existing lobe may be evaluated and maximized, as described below.

[0060] The cost functional of a lobe may take into account the spatial aspects of the lobe and the audio quality of the new sound activity. As used herein, cost functional and cost function have the same meaning. Specifically, in some embodiments, the cost functional of a lobe i may be defined as the coordinate (LC) of the new sound activity i ), the signal-to-noise ratio (SNR i ), the gain value of the flap (Gain i ), voice activity detection information (VAD) related to new sound activity i ) and the distance (LO i )). In other embodiments, the cost functional for a petal may be a function of other information. For example, the cost functional for petal i may be expressed as J with Cartesian coordinatesi (x,y,z) or J with spherical coordinates i (azimuth, elevation, magnitude). Taking the cost functional with Cartesian coordinates as an example, the cost functional J i (x,y,z)=f(LC i ,distance(LO i ),Gain i ,SNR i ,VAD i ). Therefore, the flap can be obtained by evaluating and maximizing the cost functional J i The lobe is moved over the spatial grid of coordinates such that movement of the lobe is in the direction of the gradient (i.e., steepest ascent) of the cost functional. In some cases, the maximum of the cost functional may be the same as the coordinates (i.e., candidate location) of the new sound activity received by the lobe autofocuser 160 at step 302. In other cases, the maximum of the cost functional may move the lobe to a location different from the coordinates of the new sound activity while taking into account the other parameters described above.

[0061] At step 308, a cost functional of the flap may be evaluated by the flap autofocuser 160 at the coordinates of the new sound activity. In some embodiments, the evaluated cost functional may be stored by the flap autofocuser 160 in the database 180. At step 310, the flap autofocuser 160 may move the flap by each of the amounts Δx, Δy, Δz in the x, y, and z directions, respectively, from the coordinates of the new sound activity. After each movement, the cost functional may be evaluated by the flap autofocuser 160 at each of these positions. For example, the flap may be moved to position (x+Δx, y, z) and the cost functional may be evaluated at that position; then moved to position (x, y+Δy, z) and the cost functional may be evaluated at that position; and then moved to position (x, y, z+Δz) and the cost functional may be evaluated at that position. At step 310, the flap may be moved by the amounts Δx, Δy, Δz in any order. In some embodiments, each of the evaluated cost functionals at these locations may be stored in database 180 by flap autofocuser 160. As described below, evaluation of the cost functionals is performed by flap autofocuser 160 at step 310 in order to compute estimates of the partial derivatives and gradients of the cost functionals. It should be noted that while the above description relates to Cartesian coordinates, similar operations may be performed for spherical coordinates (e.g., Δ azimuth, Δ elevation, Δ magnitude).

[0062] At step 312, the gradient of the cost functional may be calculated by the flap autofocuser 160 based on the estimated set of partial derivatives. It can be calculated as follows:

[0063]

[0064] At step 314, the flap autofocuser 160 may focus the flap along the gradient calculated at step 312. Specifically, the petal can be moved to a new position: (x i +μgx i ,y i +μgy i ,z i +μgz i ). At step 314, the flap autofocuser 160 may also evaluate a cost functional for the flap at this new position. In some embodiments, this cost functional may be stored by the flap autofocuser 160 in the database 180.

[0065] At step 316, the flap autofocuser 160 may compare the cost functional of the flap at the new position (evaluated at step 314) with the cost functional of the flap at the coordinates of the new sound activity (evaluated at step 308). If, at step 316, the cost functional of the flap at the new position is less than the cost functional of the flap at the coordinates of the new sound activity, then the step size μ at step 314 may be considered too large, and the process 300 may continue to step 322. At step 322, the step size may be adjusted, and the process may return to step 314.

[0066] However, if at step 316 the cost functional of the lobe at the new position is not less than the cost functional of the lobe at the coordinates of the new sound activity, then process 300 may continue to step 318. At step 318, lobe autofocuser 160 may determine whether the difference between (1) the cost functional of the lobe at the new position (evaluated at step 314) and (2) the cost functional of the lobe at the coordinates of the new sound activity (evaluated at step 308) is close, that is, whether the absolute value of the difference is within a small amount ε. If the condition is not met at step 318, then it may be considered that the local maximum of the cost functional has not been reached. Process 300 may proceed to step 324, and the position of the lobe of array microphone 100 is not updated.

[0067] However, if the condition is met at step 318, then it can be considered that the local maximum of the cost functional has been reached and the lobe has been auto-focused, and process 300 continues to step 320. At step 320, lobe auto-focuser 160 can transmit the coordinates of the new sound activity to beamformer 170 so that beamformer 170 can update the position of the lobe to the new coordinates. In addition, lobe auto-focuser 160 can store the new coordinates of the lobe in database 180.

[0068] In some embodiments, at step 320, the petal autofocuser 160 may apply an annealing / dithering move of the petal. The annealing / dithering move may be applied to nudge the petal out of a local maximum of the cost functional in an attempt to find a better local maximum (and therefore a better position for the petal). The annealing / dithering position may be given by (xi +rx i ,y i +ry i ,z i +rz i ) definition, where (rx i ,ry i ,rz i ) is a small random value.

[0069] Process 300 may be continuously performed by array microphone 100 as audio activity locator 150 discovers new sound activity and provides the coordinates and confidence scores of the new sound activity to lobe autofocuser 160. For example, process 300 may be performed as an audio source (e.g., a human speaker) moves around a conference room so that one or more lobes can focus on the audio source to best pick up its sound.

[0070] In an embodiment, for example, the cost functional may be re-evaluated and updated in steps 308 to 318 and 322, and the coordinates of the lobe may be adjusted, for example, without receiving a set of coordinates of new sound activity at step 302. For example, the algorithm may detect which lobe of the array microphone 100 has the greatest sound activity without providing a set of coordinates of new sound activity. Based on the sound activity information from this algorithm, the cost functional may be re-evaluated and updated.

[0071] Figure 5 An embodiment of a process 500 for automatically configuring or deploying beamforming lobes for an array microphone 400 is shown in FIG. The process 500 may be performed by a lobe automatic configurator 460 so that the array microphone 400 may be configured from Figure 4 4 outputs one or more audio signals 480, wherein the audio signals 480 may include the sound of new sound activity from the audio source picked up by the configured beamforming lobe. One or more processors and / or other processing components (e.g., analog-to-digital converters, encryption chips, etc.) inside or outside the microphone array 400 may perform any, some, or all steps of the process 500. One or more other types of components (e.g., memory, input and / or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be used in conjunction with processors and / or other processing components to perform any, some, or all steps of the process 500.

[0072] At step 502, coordinates corresponding to new sound activity may be received at the flap autoconfigurator 460 from the audio activity locator 450. The audio activity locator 450 may continuously scan the environment of the array microphone 400 to find new sound activity. The new sound activity discovered by the audio activity locator 450 may include a suitable audio source, such as a non-stationary human speaker. The coordinates of the new sound activity may be specific three-dimensional coordinates relative to the position of the array microphone 400, such as in Cartesian coordinates (i.e., x, y, z) or in spherical coordinates (i.e., radial distance / magnitude r, elevation angle θ (theta), azimuth angle θ (θ)). (phi)).

[0073] In an embodiment, configuration of the beamforming lobes may occur based on whether the amount of activity of the new sound activity exceeds a predetermined threshold. Fig.19 1 is a schematic diagram of an array microphone 1900 that can detect sounds from audio sources of various frequencies and automatically configure beamforming lobes in response to detection of sound activity while taking into account the amount of activity of the new sound activity. In an embodiment, the array microphone 1900 may include some or all of the same components as the array microphone 400 described above, such as the microphone 402, the audio activity locator 450, the lobe automatic configurator 460, the beamformer 470, and / or the database 480. The array microphone 1900 may also include an activity detector 1904 in communication with the lobe automatic configurator 460 and the beamformer 470.

[0074] Activity detector 1904 may detect the amount of activity in the new sound activity. In some embodiments, the amount of activity may be measured as the energy level of the new sound activity. In other embodiments, methods in the time domain and / or frequency domain may be used to measure the amount of activity, such as by applying machine learning (e.g., using cepstral coefficients), measuring signal non-stationarity in one or more frequency bands, and / or searching for features of the desired sound or voice.

[0075] In an embodiment, the activity detector 1904 may be a voice activity detector (VAD) that may determine whether speech and / or noise are present in the remote audio signal. For example, speech and / or noise may be detected by analyzing the spectral variation of the remote audio signal, using linear predictive coding, applying machine learning or deep learning techniques, and / or using ETSI standards for VAD calculations such as those included in the GSM specification, or long-term pitch prediction.

[0076] Based on the amount of activity detected, automatic flap configuration may or may not be performed. Automatic flap configuration may be performed when the detected activity of new sound activity meets predetermined criteria. Conversely, automatic flap configuration may not be performed when the detected activity of new sound activity does not meet predetermined criteria. For example, meeting the predetermined criteria may indicate that the new sound activity includes speech, voice, or other sounds that are preferably picked up by a flap. As another example, not meeting the predetermined criteria may indicate that the new sound activity does not include speech, voice, or other sounds that are preferably picked up by a flap. By suppressing automatic flap configuration in the latter case, the flap will not be configured to avoid picking up sound from the new sound activity.

[0077] As in Fig. 20 As seen in the process 2000 of , at step 2003 after step 502, it may be determined whether the activity amount of the new sound activity meets a predetermined criterion. For example, the activity detector 1904 may receive the new sound activity from the beamformer 470. The detected activity amount may correspond to the amount of speech, voice, noise, etc. in the new sound activity. In an embodiment, the activity amount may be measured as an energy level of the new sound activity, or as the amount of speech in the new sound activity. In an embodiment, the detected activity amount may specifically indicate the amount of speech or voice in the new sound activity. In other embodiments, the detected activity amount may be a speech-to-noise ratio, or indicate the amount of noise in the new sound activity.

[0078] If the amount of activity does not meet the predetermined criteria at step 2003, process 2000 may end at step 522 and the position of the lobes of array microphone 1900 is not updated. When the amount of voice or speech is relatively low and / or the speech-to-noise ratio is relatively low in the new sound activity, the amount of activity detected for the new sound activity may not meet the predetermined criteria. Similarly, when there is a relatively high amount of noise in the new sound activity, the amount of activity detected for the new sound activity may not meet the predetermined criteria. Therefore, not automatically configuring the lobes to detect new sound activity can help ensure that undesirable sounds are not picked up.

[0079] If the amount of activity satisfies the predetermined criteria at step 2003, process 2000 may continue to step 504 as described below. The amount of activity detected for the new sound activity may satisfy the predetermined criteria when the amount of voice or speech is relatively high and / or the speech-to-noise ratio is relatively high in the new sound activity. Similarly, the amount of activity detected for the new sound activity may satisfy the predetermined criteria when there is a relatively low amount of noise in the new sound activity. Thus, in this case, it may be desirable to automatically configure one lobe to detect the new sound activity.

[0080] Returning to process 500, at step 504, flap autoconfigurator 460 may update the timestamp to a current value such as a clock. In some embodiments, the timestamp may be stored in database 480. In embodiments, the timestamp and / or clock may be a real-time value, such as hours, minutes, seconds, etc. In other embodiments, the timestamp and / or clock may be based on an increasing integer value, which may enable tracking of the temporal sequence of events.

[0081] The flap autoconfigurator 460 may determine in step 506 whether the coordinates of the new sound activity are near (i.e., in the vicinity of) an existing active flap. Whether the new sound activity is near an existing flap may be based on the difference in azimuth and / or elevation of (1) the coordinates of the new sound activity and (2) the coordinates of the existing flap relative to a predetermined threshold. The distance of the new sound activity from the microphone 400 may also affect the determination of whether the coordinates of the new sound activity are near an existing flap. In some embodiments, the flap autoconfigurator 460 may retrieve the coordinates of the existing flap from the database 480 for use in step 506. Figure 6 An embodiment of determining whether the coordinates of new sound activity are near an existing lobe is described in more detail.

[0082] However, if at step 506, the lobe autoconfigurator 460 determines that the coordinates of the new sound activity are near an existing lobe, then the process 500 continues to step 520. At step 520, the timestamp of the existing lobe is updated from step 504 to the current timestamp. In this case, the existing lobe is considered to be able to cover (i.e., pick up) the new sound activity. The process 500 can end at step 522, and the position of the lobe of the array microphone 400 is not updated.

[0083] However, if at step 506, the flap automatic configurator 460 determines that the coordinates of the new sound activity are near the existing flap, then the process 500 continues to step 508. In this case, the coordinates of the new sound activity can be considered to be outside the current coverage area of ​​the array microphone 400, and therefore the new sound activity needs to be covered. At step 508, the flap automatic configurator 460 can determine whether the inactive flap of the array microphone 400 is available. In some embodiments, if the flap is not pointed to a specific set of coordinates or if the flap is not deployed (i.e., does not exist), then the flap can be considered inactive. In other embodiments, based on whether a metric (e.g., time, age, etc.) of the deployed flap meets a certain criterion, the deployed flap can be considered inactive. If the flap automatic configurator 460 determines at step 508 that there is an available inactive flap, then the inactive flap is selected at step 510, and the timestamp of the newly selected flap is updated to the current timestamp (from step 504) at step 514.

[0084] However, if at step 508 the flap autoconfigurator 460 determines that there are no available inactive flaps, then the process 500 may continue to step 512. At step 512, the flap autoconfigurator 460 may select the currently active flap for recycling to point to the new sound activity coordinates. In some embodiments, the flap selected for recycling may be the active flap with the lowest confidence score and / or the oldest timestamp. The confidence score of the flap may represent, for example, the certainty of the coordinates and / or the quality of the sound activity. In embodiments, other suitable metrics related to the flap may be utilized. The oldest timestamp of the active flap may indicate that no sound activity has been detected recently for the flap, and may indicate that an audio source is no longer present in the flap. The flap selected for recycling at step 512 may have its timestamp updated to the current timestamp (from step 504) at step 514.

[0085] At step 516, a new confidence score may be assigned to the flap, whether it is the selected inactive flap from step 510 or the selected recirculation flap from step 512. At step 518, the flap autoconfigurator 460 may transmit the coordinates of the new sound activity to the beamformer 470 so that the beamformer 470 may update the position of the flap to the new coordinates. In addition, the flap autoconfigurator 460 may store the new coordinates of the flap in the database 480.

[0086] Process 500 may be continuously performed by array microphone 400 as audio activity locator 450 discovers new sound activity and provides the coordinates of the new sound activity to lobe autoconfigurator 460. For example, process 500 may be performed as an audio source (e.g., a human speaker) moves around a conference room so that one or more lobes may be configured to optimally pick up the sound of the audio source.

[0087] exist Figure 6 An embodiment of a process 600 for finding a previously configured lobe near a sound activity is shown in . Process 600 may be used by lobe autofocuser 160 at step 204 of process 200, at step 304 of process 300, and / or at step 806 of process 800, and / or by autoconfigurator 460 at step 506 of process 500. Specifically, process 600 may determine whether the coordinates of a new sound activity are near an existing lobe of array microphone 100, 400. Whether the new sound activity is near an existing lobe may be based on a difference in azimuth and / or elevation of (1) the coordinates of the new sound activity and (2) the coordinates of the existing lobe relative to a predetermined threshold. The distance of the new sound activity from array microphone 100, 400 may also affect the determination of whether the coordinates of the new sound activity are near an existing lobe.

[0088] At step 602, coordinates corresponding to new sound activity may be received from the audio activity locator 150, 450 at the flap autofocuser 160 or the flap autoconfigurer 460, respectively. The coordinates of the new sound activity may be specific three-dimensional coordinates relative to the position of the array microphone 100, 400, such as in Cartesian coordinates (i.e., x, y, z) or in spherical coordinates (i.e., radial distance / magnitude r, elevation angle θ (theta), azimuth angle θ (θ)). (phi)). It should be noted that Cartesian coordinates can be easily converted to spherical coordinates, and vice versa, as desired.

[0089] At step 604, the flap autofocuser 160 or flap autoconfigurer 460 may determine whether the new sound activity is relatively far from the array microphone 100, 400 by evaluating whether the distance of the new sound activity is greater than the determined threshold. The distance of the new sound activity may be determined by the magnitude of the vector representing the coordinates of the new sound activity. If it is determined at step 604 that the new sound activity is relatively far from the array microphone 100, 400 (i.e., greater than the threshold), then at step 606, a lower azimuth threshold may be set for later use in the process 600. If it is determined at step 604 that the new sound activity is not relatively far from the array microphone 100, 400 (i.e., less than or equal to the threshold), then at step 608, a higher azimuth threshold may be set for later use in the process 600.

[0090] After setting the azimuth threshold at step 606 or step 608, process 600 may continue to step 610. At step 610, the flap autofocuser 160 or flap autoconfigurer 460 may determine whether there are any flaps to be checked for new sound activity. If there are no flaps of the array microphone 100, 400 to be checked at step 610, process 600 may end at step 616 and indicate that there are no flaps near the array microphone 100, 400.

[0091] However, if there is a lobe of the array microphone 100, 400 to be checked at step 610, then the process 600 may continue to step 612 and check one of the existing lobes. At step 612, the lobe autofocuser 160 or the lobe autoconfigurer 460 may determine whether the absolute value of the difference between (1) the azimuth of the existing lobe and (2) the azimuth of the new sound activity is greater than the azimuth threshold (the azimuth threshold is set at step 606 or step 608). If the condition is met at step 612, then it can be considered that the lobe being checked is not near the new sound activity. The process 600 may return to step 610 to determine whether there are other lobes to be checked.

[0092] However, if the condition is not met at step 612, then process 600 may proceed to step 614. At step 614, the lobe autofocuser 160 or the lobe autoconfigurer 460 may determine whether the absolute value of the difference between (1) the elevation angle of the existing lobe and (2) the elevation angle of the new sound activity is greater than a predetermined elevation angle threshold. If the condition is met at step 614, then it may be considered that the lobe under examination is not near the new sound activity. Process 600 may return to step 610 to determine whether there are other lobes to be examined. However, if the condition is not met in step 614, then process 600 may end at step 618, indicating that the lobe under examination is near the new sound activity.

[0093] Figure 7 is an exemplary depiction of an array microphone 700 that can automatically focus previously configured beamforming lobes within associated lobe regions in response to detecting new sound activity. In embodiments, the array microphone 700 may include some or all of the same components as the array microphone 100 described above, such as the audio activity locator 150, the lobe autofocuser 160, the beamformer 170, and / or the database 180. Each lobe of the array microphone 700 may move within its associated lobe region, and the lobe may not cross the boundary between lobe regions. It should be noted that although Figure 7 Eight petals with eight associated petal areas are depicted, but any number of petals and associated petal areas are possible and contemplated, such as Fig.10 , 12 , 13 and 15 depict four lobes with four associated lobe regions. It should also be noted that Figure 7 , 10 , 12, 13 and 15 are depicted as two-dimensional representations of the three-dimensional space around the array microphone.

[0094] At least two sets of coordinates may be associated with each lobe of the array microphone 700: (1) original or initial coordinates LO i (e.g., coordinates automatically or manually constructed when setting up the array microphone 700), and (2) the current coordinates to which the current lobe is pointing at a given time In some embodiments, the set of coordinates may indicate the location of the center of the flap. In some embodiments, the set of coordinates may be stored in database 180.

[0095] In addition, each lobe of the array microphone 700 may be associated with a lobe region of three-dimensional space surrounding it. In an embodiment, the lobe region may be defined as a set of points in space that are closer to the initial coordinates LO of the lobe than the coordinates of any other lobe of the array microphone. i In other words, if p is defined as a point in space, then at point p and lobe i (LO i) is the smallest compared to any other lobe, then point p may belong to a particular lobe region LR i , as in the following formula: A region defined in this way is called a Vorono region or a Vorono cell. Figure 7 As can be seen in FIG. 1 , there are eight lobes with associated lobe regions with borders drawn between each of the lobe regions. The borders between lobe regions are the set of points in space that are equidistant from two or more adjacent lobes. Some edges of the lobe regions may also be unbounded. In an embodiment, the distance D may be the distance between point p and LO. i (For example, ) In some embodiments, the lobe area can be recalculated as a particular lobe moves.

[0096] In an embodiment, the lobe regions may be calculated and / or updated based on sensing the environment (e.g., objects, walls, people, etc.) in which the array microphone 700 is located using infrared sensors, visual sensors, and / or other suitable sensors. For example, the array microphone 700 may use information from the sensors to set approximate boundaries of the lobe regions, which in turn may be used to configure the associated lobe. In other embodiments, the lobe regions may be calculated and / or updated based on user-defined lobe regions, such as through a graphical user interface of the array microphone 700.

[0097] like Figure 7 As further shown in FIG. 1 , as described below, there may be various parameters associated with each petal that may limit its movement during the autofocus process. One parameter is the apparent radius of the petal, which is the radius of the petal at the initial coordinate LO of the petal. i The three-dimensional region of space around which new sound activity may be considered. In other words, if new sound activity is detected in a lobe region, but outside the apparent radius of the lobe, then there will be no movement or autofocus of the lobe in response to detecting the new sound activity. Thus, points outside the apparent radius of a lobe may be considered to be ignored or "irrelevant" parts of the associated lobe region. For example, Figure 7 , the point denoted A is outside the apparent radius of lobe 5 and its associated lobe region 5, so any new sound activity at point A will not cause the lobe to move. Conversely, if new sound activity is detected in a particular lobe region and the new sound activity is within the apparent radius of its lobe, the lobe may be automatically moved and focused in response to the detection of the new sound activity.

[0098] Another parameter is the movement radius of the petal, which is the maximum distance in space that the petal is allowed to move. The movement radius of the petal is usually smaller than the apparent radius of the petal and can be set to prevent the petal from moving too far from the array microphone or from the initial coordinate LO of the petal. i For example, in Figure 7 In FIG. 5 , the point denoted as B is within both the apparent radius and the moving radius of the lobe 5 and its associated lobe area 5. If new sound activity is detected at point B, the lobe 5 can be moved to point B. As another example, in Figure 7 , the point denoted as C is within the apparent radius of the petal 5, but outside the movement radius of the petal 5 and its associated petal area 5. If new sound activity is detected at point C, the maximum distance that the petal 5 can move is limited to the movement radius.

[0099] Another parameter is the boundary pad of the flap, which is the maximum distance in space that the flap is allowed to move toward the adjacent flap area and toward the boundary between flap areas. Figure 7 In , the point denoted as D is outside the boundary pad of flap 8 and its associated flap region 8 (adjacent to flap region 7). The boundary pads of the flaps may be arranged to minimize the overlap of adjacent flaps. Figure 7 , 10 , 12, 13 and 15, the boundaries between the lobe areas are represented by dotted lines, and the boundary pad of each lobe area is represented by a dot-dash line parallel to the boundary.

[0100] Figure 8 An embodiment of a process 800 for automatically focusing a previously configured beamforming lobe of an array microphone 700 within an associated lobe region is shown in FIG. The process 800 may be performed by the lobe autofocuser 160 so that the array microphone 700 may output one or more audio signals 180 from the array microphone 700, wherein the audio signal 180 may include sounds picked up by the beamforming lobe that focuses on new sound activity of an audio source. One or more processors and / or other processing components (e.g., analog-to-digital converters, encryption chips, etc.) internal or external to the array microphone 700 may perform any, some, or all steps of the process 800. One or more other types of components (e.g., memory, input and / or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be used in conjunction with processors and / or other processing components to perform any, some, or all steps of the process 800.

[0101] Step 802 of process 800 for flap autofocuser 160 may be similar to the process described above. Figure 2800 may be substantially the same as step 202 of process 200 of . Specifically, at step 802, coordinates and confidence scores corresponding to new sound activity may be received from audio activity locator 150 at flap autofocus 160. In an embodiment, other suitable metrics related to the new sound activity may be received and utilized at step 802. At step 804, flap autofocus 160 may compare the confidence score of the new sound activity with a predetermined threshold to determine whether the new confidence score is satisfactory. If flap autofocus 160 determines at step 804 that the confidence score of the new sound activity is less than the predetermined threshold (i.e., the confidence score is not satisfactory), process 800 may end at step 820 and the position of the flap of array microphone 700 is not updated. However, if flap autofocus 160 determines at step 804 that the confidence score of the new sound activity is greater than or equal to the predetermined threshold (i.e., the confidence score is satisfactory), process 800 may continue to step 806.

[0102] At step 806, the lobe autofocuser 160 may identify the lobe region where the new sound activity is located, that is, the lobe region to which the new sound activity belongs. In an embodiment, at step 806, the lobe autofocuser 160 may find the lobe closest to the coordinates of the new sound activity in order to identify the lobe region. For example, the lobe region may be identified by finding the initial coordinates LO of the lobe closest to the new sound activity. i To identify the lobe region, for example, by finding the lobe index i such that the coordinates of the new sound activity coincide with the initial coordinates LO of the lobe i Minimize the distance between: The lobe and its associated lobe region that include the new sound activity may be determined as the lobe and lobe region identified at step 806 .

[0103] After the lobe region has been identified at step 806, at step 808, lobe autofocuser 160 may determine whether the coordinates of the new sound activity are outside the apparent radius of the lobe. If lobe autofocuser 160 determines at step 808 that the coordinates of the new sound activity are outside the apparent radius of the lobe, then process 800 may end at step 820 and the position of the lobe of array microphone 700 is not updated. In other words, if the new sound activity is outside the apparent radius of the lobe, then the new sound activity may be ignored and may be considered to be outside the coverage of the lobe. As an example, Figure 7 Point A in is within the petal region 5 associated with petal 5, but outside the apparent radius of petal 5. Fig. 9 and 10 Describes the details of determining whether the coordinates of new sound activity are outside the appearance radius of the lobe.

[0104] However, if at step 808, the flap autofocuser 160 determines that the coordinates of the new sound activity are not outside the apparent radius of the flap (i.e., inside it), then the process 800 may continue to step 810. In this case, as described below, the flap may be moved toward the new sound activity based on evaluating the coordinates of the new sound activity relative to other parameters (such as the moving radius and the boundary pad). At step 810, the flap autofocuser 160 may determine whether the coordinates of the new sound activity are outside the moving radius of the flap. If at step 810, the flap autofocuser 160 determines that the coordinates of the new sound activity are outside the moving radius of the flap, then the process 800 may continue to step 816, where the movement of the flap may be restricted or limited. In particular, at step 816, the new coordinates to which the flap may be temporarily moved may be set to be no greater than the moving radius. As described below, since the movement of the flap can still be evaluated relative to the boundary pad parameters, the new coordinates may be temporary. In an embodiment, the movement of the petal at step 816 may be limited based on a scaling factor α (where 0<α≤1) to prevent the petal from moving away from its initial coordinate LO i Too far. As an example, Figure 7 Point C in is outside the moving radius of petal 5, so the farthest distance that petal 5 can move is the moving radius. After step 816, process 800 can continue to step 812. Fig.11 and 12 Describe the details of limiting the movement of the flap to within its radius of motion.

[0105] If at step 810, the lobe autofocuser 160 determines that the coordinates of the new sound activity are not outside (ie, inside) the lobe's movement radius, then the process 800 may also continue to step 812. As an example, Figure 7 Point B in is within the movement radius of petal 5, so petal 5 can move to point B. At step 812, the petal autofocus 160 may determine whether the coordinates of the new sound activity are close to the boundary pad and therefore too close to the adjacent petal. If the petal autofocus 160 determines at step 812 that the coordinates of the new sound activity are close to the boundary pad, then the process 800 may continue to step 818, where the movement of the petal may be restricted or limited. Specifically, at step 818, the new coordinates to which the petal can move may be set to be just outside the boundary pad. In an embodiment, the movement of the petal at step 818 may be limited based on a calibration factor β (where 0<β≤1). As an example, Figure 7 Point D in is outside the boundary pad between adjacent flap region 8 and flap region 7. Process 800 may continue to step 814 after step 818. Figures 13 to 15 Describes details about the boundary pad.

[0106] If lobe autofocuser 160 determines at step 812 that the coordinates of the new sound activity are not close to the boundary pad, process 800 may also continue to step 814. At step 812, lobe autofocuser 160 may transmit the new coordinates of the lobe to beamformer 170 so that beamformer 170 may update the position of the existing lobe to the new coordinates. In an embodiment, the new coordinates of the lobe Can be defined as in is the motion vector, and is a constrained motion vector, as described in more detail below. In an embodiment, the lobe autofocuser 160 may store the new coordinates of the lobe in the database 180 .

[0107] Depending on the steps of process 800 described above, when the flap moves due to detection of new sound activity, the new coordinates of the flap may be: (1) the coordinates of the new sound activity if the coordinates of the new sound activity are within the apparent radius of the flap, within the moving radius of the flap, and not close to the boundary pad of the associated flap area; (2) a point in the direction of the motion vector toward the new sound activity, and the point is limited to the range of the moving radius, if the coordinates of the new sound activity are within the apparent radius of the flap, outside the moving radius of the flap, and not close to the boundary pad of the associated flap area; or (3) just outside the boundary pad if the coordinates of the new sound activity are within the apparent radius of the flap and close to the boundary pad.

[0108] Process 800 may be continuously performed by array microphone 700 as audio activity locator 150 discovers new sound activity and provides the coordinates and confidence scores of the new sound activity to lobe autofocuser 160. For example, process 800 may be performed as an audio source (e.g., a human speaker) moves around a conference room so that one or more lobes can focus on the audio source to best pick up its sound.

[0109] exist Fig. 9 , an embodiment of a process 900 for determining whether the coordinates of a new sound activity are outside the apparent radius of a lobe is shown in FIG. For example, process 900 may be used by lobe autofocuser 160 at step 808 of process 800. Specifically, process 900 may begin at step 902, where a motion vector Calculated as The motion vector can be the original coordinate LO of the lobe i The center of the connection is connected to the coordinates of the new sound activity For example, if Fig.10 As shown in FIG. 1 , the new sound activity S exists in lobe region 3, and the motion vector is shown between the original coordinates LO3 of the lobe 3 and the coordinates of the new sound activity S. The apparent radius of the lobe 3 is also depicted at Fig.10middle.

[0110] At step 902, the motion vector is calculated. Thereafter, process 900 may continue to step 904. At step 904, lobe autofocuser 160 may determine whether the magnitude of the motion vector is greater than the apparent radius of the lobe, as in the following equation: If the motion vector If the magnitude of is greater than the apparent radius of the lobe at step 904, then at step 906, the coordinates of the new sound activity may be represented as being outside the apparent radius of the lobe. Fig.10 As shown in FIG. 9 , since the new sound activity S is outside the apparent radius of lobe 3, the new sound activity S will be ignored. However, if at step 904 the motion vector If the magnitude of is less than or equal to the apparent radius of the lobe, then at step 908 the coordinates of the new sound activity may be represented as being within the apparent radius of the lobe.

[0111] exist Fig.11 An embodiment of a process 1100 for limiting the movement of a flap within its movement radius is shown in FIG. For example, process 1100 may be used by flap autofocuser 160 at step 816 of process 800. Specifically, process 1100 may begin at step 1102, where a motion vector Calculated as Similar to the above Fig. 9 902 of process 900 shown in FIG. Fig.12 As shown in FIG. 1 , the new sound activity S exists in lobe region 3, and the motion vector is shown between the original coordinates LO3 of the lobe 3 and the coordinates of the new sound activity S. The apparent radius of the lobe 3 is also depicted at Fig.12 middle.

[0112] At step 1102, the motion vector is calculated. Thereafter, process 1100 may continue to step 1104. At step 1104, flap autofocuser 160 may determine motion vector Is the value of less than or equal to the moving radius of the petal, as in the following formula: If at step 1104 the motion vector If the magnitude of is less than or equal to the moving radius, then at step 1106, the new coordinates of the lobe may be temporarily moved to the coordinates of the new sound activity. Fig.12 As shown in , since the new sound activity S is within the moving radius of lobe 3, the lobe will temporarily move to the coordinates of the new sound activity S.

[0113] However, if at step 1104 the motion vector If the magnitude of the motion vector is greater than the movement radius, then at step 1108, the magnitude of the motion vector can be scaled to the maximum value of the movement radius by a scaling factor α while maintaining the same direction, as in the following equation: where the scaling factor α can be defined as: where the scaling factor α can be defined as:

[0114]

[0115] Figures 13 to 15 Regarding the boundary pad of the lobe region, which is a part of the space adjacent to the boundary or edge of another lobe region close to the lobe region. Specifically, a vector i connecting the original coordinates of two lobes (i.e., LO j ) can be used to indirectly describe the boundary pad near the boundary between two lobes i and j. Therefore, this vector can be described as: The midpoint of this vector can be a point on the boundary between two lobe regions. Specifically, moving along the direction of the vector from the original coordinates LO i of lobe i is the shortest path towards the adjacent lobe j. In addition, moving along the direction of the vector from the original coordinates LO i of lobe i but keeping the amount of movement as half of the magnitude of the vector will be the exact boundary between the two lobe regions. Based on the above, moving along the direction of the vector

[0116] from the original coordinates LO i of lobe i but restricting the amount of movement based on the value A (where 0 < A < 1) (i.e., ) will be within (100 * A)% of the boundary between the lobe regions. For example, if A is 0.8 (i.e., 80%), then the new coordinates of the moving lobe will be within 80% of the boundary between the lobe regions. Therefore, the value A can be used to create a boundary pad between two adjacent lobe regions. Generally, a larger boundary pad can prevent the lobe from moving into another lobe region, while a smaller boundary pad can allow the lobe to move closer to another lobe region. ) will be within (100 * A)% of the boundary between the lobe regions. For example, if A is 0.8 (i.e., 80%), then the new coordinates of the moving lobe will be within 80% of the boundary between the lobe regions. Therefore, the value A can be used to create a boundary pad between two adjacent lobe regions. Generally, a larger boundary pad can prevent the lobe from moving into another lobe region, while a smaller boundary pad can allow the lobe to move closer to another lobe region.

[0117] In addition, it should be noted that if due to the detection of new voice activity, lobe i moves in the direction towards lobe j (e.g., along the direction of the motion vector as described above), then there is a movement component in the direction of lobe j (i.e., along the direction of the vector ). To find the movement component in the direction of the vector , the motion vector can be projected onto the unit vector (which has the same direction as the vector with unit magnitude)same direction) to calculate the projection vector As an example, Fig.13 Shows the vector connecting petals 3 and 2 This is also the shortest path from the center of petal 3 towards petal region 2 . Fig.13 The projection vector shown in is the motion vector In the unit vector Projection on.

[0118] Fig.14 An embodiment of a process 1400 for creating a boundary pad of a lobe region using vector projection is shown in FIG. For example, process 1400 may be used by lobe autofocuser 160 at step 818 of process 800. Process 1400 may result in limiting motion vectors The magnitude is such that the movement of the flap in the direction of any other flap region does not exceed a certain percentage of the size that characterizes the boundary pad.

[0119] Prior to executing process 1400, vectors may be calculated for all pairs of active lobes. and unit vector As described previously, the vector The original coordinates of petals i and j can be connected. The parameter A can be determined for all active petals i (where 0 i <1), the parameter characterizing the size of the boundary pad of each lobe region. As previously described, prior to performing process 1400 (i.e., prior to step 818 of process 800), lobe regions of new sound activity may be identified (i.e., at step 806) and motion vectors may be calculated (i.e., using process 1100 / step 810).

[0120] At step 1402 of process 1400, a projection vector may be calculated for all lobes not associated with a lobe region identified for new sound activity. Projection vector The size of Fig.13 The projection vector This magnitude can be calculated as a scalar, such as by the motion vector With unit vector The dot product is calculated so that the projection PM ij =M x Du ij,z +M y Du ij,y +M z Du ij,z .

[0121] When PM ij <0, the motion vector​ With vector This means that the motion of petal i will be in the opposite direction to the boundary of petal j. In this case, the boundary pad between petals i and j is not a problem because the motion of petal i will be away from the boundary with petal j. However, in PM ij >0, the motion vector With vector This means that the movement of petal i will be in the same direction as the boundary of petal j. In this case, the movement of petal i can be restricted outside the boundary pad so that Among them A i (where 0 i <1) is a parameter characterizing the boundary pad of the lobe area associated with lobe i.

[0122] The scaling factor β can be used to ensure The scaling factor β can be used to scale the motion vector and defined as Therefore, if new sound activity is detected outside the boundary pad of the lobe area, the scaling factor β may be equal to 1, indicating that there is no motion vector At step 1404, a calibration vector β may be calculated for all lobes that are not associated with a lobe region identified for new sound activity.

[0123] At step 1406, a minimum scaling factor β corresponding to the boundary pad of the nearest lobe region may be determined, such as in the following equation: After the minimum scaling factor β has been determined at step 1406, then at step 1408, the minimum scaling factor β may be applied to the motion vector To determine the constrained motion vector

[0124] For example, Fig.15 The new sound activity S present in the flap area 3 and the motion vector between the initial coordinates LO3 of the flap 3 and the coordinates of the new sound activity S are shown. vector And the projection vector is depicted between flap 3 and each of the other flaps not associated with flap region 3 (ie, flaps 1, 2, and 4). Specifically, the vectors may be calculated for all pairs of active flaps (ie, flaps 1, 2, 3, and 4). and calculate projections PM for all lobes not associated with lobe region 3 (identified for new sound activity S) 31 、PM 32 、PM 34 ​The magnitude of the projection vector can be used to calculate the scaling factor β, and the minimum scaling factor β can be used to scale the motion vector Therefore, the motion vector It may be restricted to outside the boundary pad of lobe region 3 because the new sound activity S is too close to the boundary between lobe 3 and lobe 2. Based on the restricted motion vector, the coordinates of lobe 3 can be moved to the coordinates S outside the boundary pad of lobe region 3. r .

[0125] Fig.15 The projection vector depicted in is negative, and the corresponding scaling factor β4 (for petal 4) is equal to 1. The scaling factor β1 (for petal 1) is also equal to 1, because The scaling factor β2 (for lobe 2) is less than 1 because the new sound activity S is inside the boundary pad between lobe 2 and lobe 3 (i.e., ). Therefore, a minimum scaling factor β2 can be used to ensure that the flap 3 moves to the coordinate S r .

[0126] Fig.16 and 17 Schematic diagram of array microphones 1600, 1700 that can detect sounds from audio sources of various frequencies. Fig.16 The array microphone 1600 can automatically focus the beamforming lobe in response to the detection of sound activity, while being able to suppress the autofocus of the beamforming lobe when the activity of the remote audio signal from the far end exceeds a predetermined threshold. In an embodiment, the array microphone 1600 may include some or all of the same components as the array microphone 100 described above, such as the microphone 102, the audio activity locator 150, the lobe autofocuser 160, the beamformer 170, and / or the database 180. The array microphone 1600 may also include a sensor 1602, such as a speaker, and an activity detector 1604 in communication with the lobe autofocuser 160. The remote audio signal from the far end may be communicated with the sensor 1602 and the activity detector 1604.

[0127] Fig.17The array microphone 1700 can automatically configure the beamforming lobe in response to the detection of sound activity, and can suppress the automatic configuration of the beamforming lobe when the activity of the remote audio signal from the far end exceeds a predetermined threshold. In an embodiment, the array microphone 1700 may include some or all of the same components as the array microphone 400 described above, such as the microphone 402, the audio activity locator 450, the lobe automatic configurator 460, the beamformer 470 and / or the database 480. The array microphone 1700 may also include a sensor 1702, such as a speaker, and an activity detector 1704 in communication with the lobe automatic configurator 460. The remote audio signal from the far end may be communicated with the sensor 1702 and the activity detector 1704.

[0128] Sensors 1602, 1702 may be used to play the sound of the remote audio signal in the local environment where the array microphones 1600, 1700 are located. Activity detectors 1604, 1704 may detect the amount of activity in the remote audio signal. In some embodiments, the amount of activity may be measured as the energy level of the remote audio signal. In other embodiments, methods in the time domain and / or frequency domain may be used to measure the amount of activity, such as by applying machine learning (e.g., using cepstral coefficients), measuring signal non-stationarity in one or more frequency bands, and / or searching for features of desired sounds or voices.

[0129] In an embodiment, the activity detector 1604, 1704 may be a voice activity detector (VAD) that may determine whether speech is present in the remote audio signal. For example, speech may be detected by analyzing the spectral variation of the remote audio signal, using linear predictive coding, applying machine learning or deep learning techniques, and / or using ETSI standards for VAD calculations such as those included in the GSM specification, or long-term pitch prediction.

[0130] Based on the amount of detected activity, automatic lobe adjustment may be performed or suppressed. As described herein, automatic lobe adjustment may include, for example, automatic focusing of the lobe, automatic focusing of the lobe within a region, and / or automatic configuration of the lobe. Automatic lobe adjustment may be performed when the detected activity of the remote audio signal does not exceed a predetermined threshold. Conversely, automatic lobe adjustment may be suppressed (i.e., not performed) when the detected activity of the remote audio signal exceeds a predetermined threshold. For example, exceeding the predetermined threshold may indicate that the remote audio signal includes speech, voice, or other sounds that are preferably not picked up by the lobe. By suppressing automatic lobe adjustment in this case, the lobe will not be focused or configured to avoid picking up sounds from the remote audio signal.

[0131] In some embodiments, the activity detectors 1604, 1704 may determine whether the amount of activity of the detected remote audio signal exceeds a predetermined threshold. When the amount of activity detected does not exceed the predetermined threshold, the activity detectors 1604, 1704 may transmit an enable signal to the flap autofocuser 160 or the flap autoconfigurator 460, respectively, to allow the flap to be adjusted. Additionally or alternatively, when the amount of activity of the detected remote audio signal exceeds the predetermined threshold, the activity detectors 1604, 1704 may transmit a pause signal to the flap autofocuser 160 or the flap autoconfigurator 460, respectively, to prevent the flap from being adjusted.

[0132] In other embodiments, the activity detectors 1604, 1704 may transmit the amount of activity detected in the remote audio signal to the flap autofocuser 160 or the flap autoconfigurator 460, respectively. The flap autofocuser 160 or the flap autoconfigurator 460 may determine whether the amount of activity detected exceeds a predetermined threshold. Based on whether the amount of activity detected exceeds the predetermined threshold, the flap autofocuser 160 or the flap autoconfigurator 460 may execute or suspend adjustment of the flap.

[0133] The various components included in the array microphones 1600, 1700 may be implemented using software that may be executed by one or more servers or computers, such as a computing device having a processor and memory, a graphics processing unit (GPU), and / or by hardware (e.g., discrete logic circuits, application specific integrated circuits (ASICs), programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc.).

[0134] Fig.18 An embodiment of a process 1800 for automatically adjusting the beamforming lobe of an array microphone based on remote far-end audio signal suppression is shown in FIG. Process 1800 may be performed by array microphones 1600, 1700 so that auto-focus or auto-configuration of the beamforming lobe may be performed or suppressed based on the amount of activity of the remote audio signal from the far end. One or more processors and / or other processing components (e.g., analog-to-digital converters, encryption chips, etc.) inside or outside the array microphones 1600, 1700 may perform any, some, or all steps of process 1800. One or more other types of components (e.g., memory, input and / or output devices, transmitters, receivers, buffers, drivers, discrete components, etc.) may also be used in conjunction with processors and / or other processing components to perform any, some, or all steps of process 1800.

[0135] At step 1802, a remote audio signal may be received at the array microphone 1600, 1700. The remote audio signal may be from a far end (e.g., a remote location) and may include sounds (e.g., voice, speech, noise, etc.) from the far end. The remote audio signal may be output on the sensor 1602, 1702 (e.g., a speaker in a local environment) at step 1804. Thus, the sound from the far end may be played in the local environment, such as during a conference call, so that the local participants can hear the remote participants.

[0136] The remote audio signal may be received by the activity detector 1604, 1704, which may detect the amount of activity of the remote audio signal at step 1806. The detected amount of activity may correspond to the amount of voice, speech, noise, etc. in the remote audio signal. In an embodiment, the amount of activity may be measured as the energy level of the remote audio signal. At step 1808, if the detected amount of activity of the remote audio signal does not exceed a predetermined threshold, the process 1800 may continue to step 1810. The detected amount of activity of the remote audio signal not exceeding the predetermined threshold may indicate that there is a relatively small amount of voice, speech, noise, etc. in the remote audio signal. In an embodiment, the detected amount of activity may specifically indicate the amount of speech or speech in the remote audio signal. At step 1810, lobe adjustment may be performed. Step 1810 may include, for example, processes 200 and 300 for automatically focusing a beamforming lobe, process 400 for automatically configuring a beamforming lobe, and / or process 800 for automatically focusing a beamforming lobe within a lobe region, as described herein. Lobe adjustment may be performed in this case because, even though the lobe may be focused or configured, there is a lesser likelihood that such lobe will pick up undesirable sounds from a remote audio signal being output in the local environment. After step 1810, process 1800 may return to step 1802.

[0137] However, if the amount of activity of the remote audio signal detected at step 1808 exceeds a predetermined threshold, then process 1800 may continue to step 1812. At step 1812, no lobe adjustment is performed, that is, lobe adjustment may be suppressed. The amount of activity of the remote audio signal detected exceeding the predetermined threshold may indicate that a relatively high amount of voice, speech, noise, etc. is present in the remote audio signal. In this case, suppressing the occurrence of lobe adjustment may help ensure that the lobe is not focused or configured to pick up sound from the remote audio signal output from the local environment. In some embodiments, process 1800 may return to step 1802 after step 1812. In other embodiments, process 1800 may wait for a specific duration at step 1812 before returning to step 1802. Waiting for a specific duration may allow reverberation in the local environment (e.g., caused by the sound of the remote audio signal being played) to dissipate.

[0138] Process 1800 may be continuously performed by array microphones 1600, 1700 when a far-end audio signal is received from the far end. For example, the far-end audio signal may include a low amount of activity (e.g., no voice or speech) that does not exceed a predetermined threshold. In this case, lobe adjustment may be performed. As another example, the far-end audio signal may include a high amount of activity (e.g., voice or speech) that exceeds a predetermined threshold. In this case, lobe adjustment may be suppressed from being performed. Therefore, whether lobe adjustment is performed or suppressed may change as the amount of activity of the far-end audio signal changes. Process 1800 may result in better pickup of sound in the local environment by reducing the likelihood of undesirably picking up sound from the far end.

[0139] Any process descriptions or blocks in the figures should be understood to represent code modules, segments or portions, which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternative implementations are included within the scope of the embodiments of the present invention, where functions may be performed out of the order from that shown or discussed, depending on the functions involved, including substantially simultaneously or in reverse order, as will be understood by those skilled in the art.

[0140] The present invention is intended to explain how to form and use various embodiments according to the present technology, rather than to limit the true, intended and fair scope and spirit thereof. The foregoing description is not intended to be exhaustive or limited to any precise form disclosed. Modifications or changes are possible in light of the above teachings. The embodiments are selected and described to provide the best illustration of the principles of the described technology and its practical application, and to enable those skilled in the art to use the technology in various embodiments and with various modifications suitable for the intended specific use. When interpreted according to the breadth to which it is fairly, legally and equitably authorized, all such modifications and variations are within the scope of the embodiments determined by the appended claims and all their equivalents, which may be amended during the prosecution of this patent application.

Claims

1. A method comprising: deploying multiple lobes from an array microphone in the environment; selecting one of the plurality of lobes to move based on location data of sound activity in the environment; determining whether a metric associated with the acoustic activity is greater than or equal to a metric associated with the selected lobe; as well as When it is determined that the metric associated with the acoustic activity is greater than or equal to the metric associated with the selected lobe, the selected lobe is repositioned based on the position data of the acoustic activity. 2 . The method of claim 1 , wherein the location data of the sound activity comprises coordinates of the sound activity in the environment.

3. A method according to claim 2, wherein the selection of the one of the multiple lobes is based on the proximity of the coordinates of the sound activity to the selected lobe. 4 . The method of claim 1 , wherein the metric associated with the voice activity comprises a confidence score representing one or more of certainty of the location data for the voice activity or quality of the voice activity.

5. The method according to claim 1 further includes storing the metric associated with the sound activity in a database as the metric associated with the selected lobe when it is determined that the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe.

6. The method of claim 5, wherein determining whether the metric associated with the acoustic activity is greater than or equal to the metric associated with the selected lobe comprises: retrieving from said database said metric associated with said selected flap; and The metric associated with the acoustic activity is compared to the retrieved metric associated with the selected lobe.

7. A method according to claim 2, wherein selecting the one of the multiple lobes to move is based on one or more of the following: (1) the relationship between the azimuth of the coordinate of the sound activity and the azimuth of the selected lobe and an azimuth threshold, or (2) the relationship between the elevation of the coordinate of the sound activity and the elevation of the selected lobe and an elevation threshold.

8. The method of claim 7, wherein selecting the one of the plurality of lobes to move is based on a distance of the coordinates of the sound activity from the array microphone. 9 . The method of claim 8 , further comprising setting the azimuth threshold based on the distance of the coordinates of the sound activity from the array microphone.

10. The method according to claim 7, wherein selecting the one of the multiple lobes to move includes selecting the selected lobe when (1) the absolute value of the difference between the azimuth angle of the coordinates of the sound activity and the azimuth angle of the selected lobe is not greater than the azimuth angle threshold; and (2) the absolute value of the difference between the elevation angle of the coordinates of the sound activity and the elevation angle of the selected lobe is greater than the elevation angle threshold.

11. The method according to claim 1 further includes storing the position data associated with the sound activity in a database as the new position of the selected lobe when it is determined that the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe.

12. The method according to claim 1: It further includes determining a duration since last movement of said selected petal; Wherein when the metric associated with the sound activity is greater than or equal to the metric associated with the selected lobe and the duration exceeds a time threshold, the selected lobe is repositioned based on the position data of the sound activity.

13. The method according to claim 1, further comprising: evaluating and maximizing a cost functional associated with the coordinates of the sound activity; and When the cost functional associated with the coordinates of the sound activity has been maximized, the selected lobe is repositioned based on adjusted position data, wherein the adjusted position data includes the position data of the sound activity adjusted based on the evaluation and maximization of the cost functional associated with the coordinates of the sound activity.

14. A method according to claim 13, wherein the cost functional is evaluated and maximized based on one or more of the coordinates of the sound activity, the signal-to-noise ratio associated with the selected lobe, the gain value associated with the selected lobe, the voice activity detection information associated with the sound activity, or the distance between the selected lobe and the position data of the sound activity. The method of claim 13 , wherein the adjusted position data is adjusted in the direction of a gradient of the cost functional.

16. The method of claim 13, wherein evaluating and maximizing the cost functional comprises: (A) moving the selected lobe based on the position data of the sound activity; (B) evaluating the cost functional of moving the selected lobe; (C) moving the selected petal by a predetermined amount in each three-dimensional direction; (D) after each movement of the selected lobe at step (C), evaluating the cost functional of the selected lobe at each of the moved positions; (E) calculating a gradient of the cost functional based on an estimate of a partial derivative, the partial derivative being calculated based on the evaluated cost functional at each of the position data of the sound activity and the movement position of step (C); (F) moving the selected lobe along the direction of the gradient by a predetermined step length; (G) evaluating the cost functional of the selected lobe at the shifted position of step (F); (H) adjusting the predetermined step size when the cost functional of step (G) is less than the cost functional of step (B), and repeating step (F); and (I) When the absolute value of the difference between the cost functional of step (G) and the cost functional of step (B) is smaller than a predetermined amount, the moved position of step (F) is represented as the adjusted position data.

17. The method of claim 16, wherein evaluating and maximizing the cost functional further comprises: When the absolute value of the difference between the cost functional of step (G) and the cost functional of step (B) is less than the predetermined amount: dithering the selected lobe at the shifted position of step (F) by a random amount; and The cost functional of the selected lobe is evaluated at the shifted position of the dither.

18. The method of claim 1, further comprising: determining constrained position data for movement of the selected flap based on the position data of the acoustic activity and parameters associated with the selected flap; and The selected flap is repositioned based on the constrained position data.

19. The method according to claim 18: wherein each of the plurality of lobes is associated with one of the plurality of lobe regions; The method further includes identifying the lobe region containing the sound activity based on the location data of the sound activity in the environment, wherein the identified lobe region is associated with the selected lobe; Wherein the parameters are further associated with the identified flap region.

20. The method according to claim 18, wherein the parameters include an apparent radius around the selected lobe, the apparent radius including the space around the selected lobe within which the sound is believed to be active; and Wherein determining whether the selected lobe is near the sound activity comprises determining whether the sound activity is within the appearance radius based on the position data of the sound activity.

21. The method according to claim 18, wherein the parameters include a movement radius, the movement radius including a maximum distance from the selected petal that the selected petal is permitted to move; and The restricted location data includes: the position data of the sound activity when the position data of the sound activity indicates that the sound activity is within the movement radius; or When the location data of the sound activity indicates that the sound activity is outside the moving radius, the moving radius.

22. The method according to claim 18, wherein the parameters include a boundary pad in a petal region, the boundary pad including a maximum distance from the selected petal that permits the selected petal to move toward a boundary of an adjacent petal region; and The restricted location data includes: the position data of the sound activity when the position data of the sound activity indicates that the sound activity is within the boundary pad; or When the position data of the sound activity represents the position outside the boundary pad when the sound activity is within the boundary pad.

23. The method of claim 1, further comprising: Receive remote audio signals from a far end; detecting an amount of activity of the remote audio signal; as well as When the amount of activity of the remote audio signal exceeds a predetermined threshold, performance of the step of selecting the one of the plurality of lobes to move and reposition the selected lobe is inhibited.

24. An array microphone system, comprising: a plurality of microphone elements, each of the plurality of microphone elements being configured to detect sound and output an audio signal; a beamformer in communication with the plurality of microphone elements, the beamformer configured to generate one or more beamformed signals based on the audio signals of the plurality of microphone elements, wherein the one or more beamformed signals correspond to one or more lobes, each lobe being positioned at a location in an environment; an audio activity locator in communication with the plurality of microphone elements, the audio activity locator being configured to determine (1) coordinates of new sound activity in the environment and (2) metrics associated with the new sound activity; and a flap autofocuser in communication with the audio activity locator and the beamformer, the flap autofocuser being constructed to: receiving the coordinates of the new sound activity and the metric associated with the new sound activity; determining whether the coordinates of the new sound activity are in the vicinity of an existing lobe, wherein the existing lobe includes one of the one or more lobes; determining whether the metric associated with the new acoustic activity is greater than or equal to the metric associated with the existing lobe when the coordinates of the new acoustic activity are determined to be in the vicinity of the existing lobe; and When it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, the coordinates of the new sound activity are transmitted to the beamformer to cause the beamformer to update the position of the existing lobe to the coordinates of the new sound activity.

25. The system according to claim 24: wherein the metric comprises a confidence score for the new voice activity; and Wherein the lobe autofocuser is constructed to determine whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe by determining whether the confidence score of the new sound activity is greater than or equal to the confidence score of the existing lobe.

26. The system of claim 25, wherein the confidence score represents one or more of certainty of the coordinates of the new voice activity or quality of the new voice activity.

27. The system of claim 25, further comprising a database in communication with the lobe autofocuser, wherein the lobe autofocuser is further constructed to store the confidence score associated with the new sound activity in the database as a new confidence score for the existing lobe when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe.

28. The system of claim 27, wherein the lobe autofocuser is constructed to determine whether the confidence score of the new sound activity is greater than or equal to the confidence score of the existing lobe by: retrieving the confidence score for the existing flap from the database; and The confidence score associated with the new vocal activity is compared to the confidence score of the retrieved existing lobe.

29. A system according to claim 24, wherein the lobe autofocuser is constructed to determine whether the coordinates of the new sound activity are near the existing lobe based on one or more of the following: (1) the relationship between the difference between the azimuth of the coordinates of the new sound activity and the azimuth of the position of the existing lobe and an azimuth threshold, or (2) the relationship between the difference between the elevation of the coordinates of the new sound activity and the elevation of the position of the existing lobe and an elevation threshold.

30. The system of claim 29, wherein the lobe autofocuser is constructed to determine whether the coordinates of the new sound activity are near an existing lobe based on the distance of the coordinates of the new sound activity from the system.

31. The system of claim 30, wherein the flap autofocuser is further constructed to set the azimuth threshold based on the distance of the coordinates of the new sound activity from the system.

32. A system according to claim 29, wherein the lobe autofocuser is constructed to: determine that the coordinates of the new sound activity are near the existing lobe when (1) the absolute value of the difference between the azimuth angle of the coordinates of the new sound activity and the azimuth angle of the position of the existing lobe is not greater than the azimuth angle threshold; and (2) the absolute value of the difference between the elevation angle of the coordinates of the new sound activity and the elevation angle of the position of the existing lobe is greater than the elevation angle threshold.

33. The system of claim 24, further comprising a database in communication with the lobe autofocuser, wherein the lobe autofocuser is further constructed to store the coordinates of the new sound activity in the database as the new position of the existing lobe when it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe.

34. The system of claim 24, wherein the flap autofocuser is further configured to: When determining that the coordinates of the new sound activity are near the existing lobe, determining whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe and based on a duration from a last movement of the existing lobe; and When it is determined that the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe and the duration from the last movement of the existing lobe exceeds a time threshold, the coordinates of the new sound activity are transmitted to the beamformer to cause the beamformer to update the position of the existing lobe to the coordinates of the new sound activity.

35. The system of claim 24, wherein the flap autofocuser is further configured to: When it is determined that the metric associated with the new acoustic activity is greater than or equal to the metric associated with the existing lobe, evaluating and maximizing a cost functional associated with the coordinates of the new acoustic activity; and when the cost functional associated with the coordinates of the new sound activity has been maximized, transmitting adjusted coordinates of the new sound activity to the beamformer to cause the beamformer to update the position of the existing lobe to the adjusted coordinates; Wherein the adjusted coordinates include the coordinates of the new voice activity, the coordinates adjusted based on the evaluation and maximization of the cost functional associated with the coordinates of the new voice activity.

36. A system according to claim 35, wherein the cost functional is evaluated and maximized based on one or more of: the coordinates of the new sound activity, the signal-to-noise ratio associated with the existing lobe, the gain value associated with the existing lobe, voice activity detection information associated with the new sound activity, or the distance between the position of the existing lobe and the coordinates of the new sound activity.

37. The system of claim 24, wherein the flap autofocuser is further configured to: Upon determining that the metric associated with the new acoustic activity is greater than or equal to the metric associated with the existing lobe: determining a lobe region within which the coordinates of the new sound activity lie; determining, based on the coordinates of the new sound activity and parameters associated with the existing lobe and the lobe region, whether the coordinates of the new sound activity are near the existing lobe; and When the coordinates of the new sound activity are determined to be near the existing lobe: limiting the updating of the position of the existing lobe to restricted coordinates within the lobe region around the existing lobe, wherein the restricted coordinates are based on the coordinates of the new sound activity and the parameters associated with the existing lobe and the lobe region; and The restricted coordinates are transmitted to the beamformer to cause the beamformer to update the position of the existing lobe to the restricted coordinates.

38. The system according to claim 37: wherein the parameters include an apparent radius in the lobe region around the existing lobe, the apparent radius including a space around the location of the existing lobe in which the new sound is believed to be active; and Wherein the lobe autofocuser is further constructed to determine whether the coordinates of the new sound activity are near an existing lobe by determining whether the coordinates of the new sound activity are within the appearance radius.

39. The system according to claim 37: wherein the parameters include a movement radius in the flap region, the movement radius including a maximum distance the existing flap is permitted to move from the position of the existing flap; and Where the restricted coordinates are: including the coordinates of the new sound activity when the coordinates of the new sound activity are within the moving radius; or When the coordinates of the new sound activity are outside the moving radius, the moving radius is included.

40. The system according to claim 37: wherein the parameters include a boundary pad in the petal region, the boundary pad including a maximum distance from the position of the existing petal that the existing petal is permitted to move toward a boundary of an adjacent petal region; and Where the restricted coordinates are: including the coordinates of the new sound activity when the coordinates of the new sound activity are outside the boundary pad; or When the coordinates of the new sound activity are within the boundary pad, a location outside the boundary pad is included.

41. The system according to claim 24: It further includes an activity detector in communication with the distal end and the flap autofocus, the activity detector being constructed to: receiving a remote audio signal from the remote end; detecting an amount of activity in the remote audio signal; and transmitting the detected amount of activity to the flap autofocus; and wherein the flap autofocus device is further constructed to: When the activity amount of the remote audio signal exceeds a predetermined threshold, the lobe autofocuser is suppressed to perform the following steps: determine whether the coordinates of the new sound activity are near the existing lobe, determine whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, and transmit the coordinates of the new sound activity to the beamformer.

42. The system according to claim 24: It further includes an activity detector in communication with the distal end and the flap autofocus, the activity detector being constructed to: receiving a remote audio signal from the remote end; detecting an amount of activity of the remote audio signal; and When the amount of activity of the remote audio signal exceeds a predetermined threshold, a signal is transmitted to the lobe autofocuser to cause the lobe autofocuser to stop performing the following steps: determining whether the coordinates of the new sound activity are near the existing lobe, determining whether the metric associated with the new sound activity is greater than or equal to the metric associated with the existing lobe, and transmitting the coordinates of the new sound activity to the beamformer.

43. An array microphone system, comprising: a plurality of microphone elements, each of the plurality of microphone elements being configured to detect sound and output an audio signal; a beamformer in communication with the plurality of microphone elements, the beamformer configured to generate one or more beamformed signals based on the audio signals of the plurality of microphone elements, wherein the one or more beamformed signals correspond to one or more lobes, each lobe being positioned at a location in an environment; an audio activity locator in communication with the plurality of microphone elements, the audio activity locator being configured to determine (1) coordinates of new sound activity in the environment and (2) metrics associated with the new sound activity; and a flap autofocuser in communication with the audio activity locator and the beamformer, the flap autofocuser being constructed to: receiving the coordinates of the new sound activity and the metric associated with the new sound activity; determining whether the coordinates of the new sound activity are in the vicinity of an existing lobe, wherein the existing lobe includes one of the one or more lobes; determining whether the metric associated with the new acoustic activity is greater than or equal to the metric associated with the existing lobe when the coordinates of the new acoustic activity are determined to be in the vicinity of the existing lobe; When it is determined that the metric associated with the new acoustic activity is greater than or equal to the metric associated with the existing lobe, evaluating and maximizing a cost functional associated with the coordinates of the new acoustic activity such that the existing lobe is moved to adjust the coordinates in the direction of the gradient of the cost functional; and When the cost functional associated with the coordinates of the new sound activity has been maximized, adjusting coordinates are transmitted to the beamformer to cause the beamformer to update the position of the existing lobe to the adjusted coordinates.

44. A system according to claim 43, wherein the cost functional is evaluated and maximized based on one or more of: the coordinates of the new sound activity, the signal-to-noise ratio associated with the existing lobe, the gain value associated with the existing lobe, the voice activity detection information associated with the new sound activity, or the distance between the position of the existing lobe and the coordinates of the new sound activity.

45. The system of claim 43, wherein the flap autofocuser is constructed to evaluate and maximize the cost functional associated with the coordinates of the new sound activity by: (A) moving the position of the existing lobe to the coordinates of the new sound activity; (B) evaluating the cost functional of the existing lobe at the coordinates of the new sound activity; (C) moving the existing lobe by a predetermined amount in each three-dimensional direction from the coordinates of the new acoustic activity; (D) after each movement of the existing lobe at step (C), evaluating the cost functional of the existing lobe at each of the moved coordinates; (E) calculating the gradient of the cost functional based on estimates of partial derivatives calculated based on the evaluated cost functional at each of the coordinates of the new sound activity and the moved coordinates of step (C); (F) moving the existing petal along the direction of the gradient by a predetermined step length; (G) evaluating the cost functional of the existing lobe at the shifted coordinates of step (F); (H) when the cost functional of step (G) is less than the cost functional of step (B), adjusting the predetermined step size and repeating step (F); and (I) When an absolute value of a difference between the cost functional of step (G) and the cost functional of step (B) is less than a predetermined amount, expressing the moved coordinates of step (F) as the adjusted coordinates.

46. ​​The system of claim 45, wherein the flap autofocuser is constructed to: When the absolute value of the difference between the cost functional of step (G) and the cost functional of step (B) is less than a predetermined amount: the existing lobe jitter random amount at the shifted coordinates of step (F); and The cost functional of the existing lobe is evaluated at the dithered moved coordinates.

47. A method comprising: deploying multiple lobes from an array microphone in the environment; selecting one of the plurality of lobes to move based on location data of sound activity in the environment; determining constrained position data for movement of the selected flap based on the position data of the acoustic activity and parameters associated with the selected flap; and repositioning the selected flap based on the constrained position data; determining whether a metric associated with the voice activity is greater than or equal to a metric threshold; as well as When it is determined that the metric associated with the voice activity is greater than or equal to the metric threshold, the steps of selecting the one of the plurality of lobes to move, determining the restricted position data, and repositioning the selected lobe are performed.

48. The method according to claim 47: wherein each of the plurality of lobes is associated with one of the plurality of lobe regions; The method further includes identifying the lobe region containing the sound activity based on the location data of the sound activity in the environment, wherein the identified lobe region is associated with the selected lobe; Wherein the parameters are further associated with the identified flap region.

49. The method of claim 47, wherein the location data for the sound activity comprises coordinates of the sound activity in the environment.

50. The method of claim 47, further comprising: determining whether the selected flap is proximate to the sound activity based on the location data of the sound activity and the parameters associated with the selected flap; and When it is determined that the selected lobe is in the vicinity of the acoustic activity, the steps of determining the restricted position data and repositioning the selected lobe are performed.

51. The method according to claim 50, wherein the parameters include an apparent radius around the selected lobe, the apparent radius including the space around the selected lobe within which the sound is believed to be active; and Wherein determining whether the selected lobe is near the sound activity comprises determining whether the sound activity is within the appearance radius based on the position data of the sound activity.

52. The method of claim 51, determining whether the sound activity is within the appearance radius comprises: calculating a motion vector between the selected lobe and the sound activity; determining whether the magnitude of the motion vector is less than or equal to the appearance radius; When the magnitude of the motion vector is less than or equal to the appearance radius, it indicates that the sound activity is within the appearance radius; as well as When the magnitude of the motion vector is greater than the appearance radius, it indicates that the sound activity is outside the appearance radius.

53. The method according to claim 47, wherein the parameters include a movement radius, the movement radius including a maximum distance from the selected petal that the selected petal is permitted to move; and The restricted location data includes: the position data of the sound activity when the position data of the sound activity indicates that the sound activity is within the movement radius; or When the location data of the sound activity indicates that the sound activity is outside the moving radius, the moving radius.

54. The method of claim 53, wherein determining the restricted location data comprises: calculating a motion vector between the selected lobe and the sound activity; determining whether the magnitude of the motion vector is less than or equal to the movement radius; When the magnitude of the motion vector is less than or equal to the moving radius, it indicates that the sound activity is within the moving radius; as well as When the magnitude of the motion vector is greater than the moving radius, it indicates that the sound activity is outside the moving radius.

55. The method according to claim 48, wherein the parameters include a boundary pad in the petal region, the boundary pad including a maximum distance from the selected petal that permits the selected petal to move toward a boundary of an adjacent petal region; and The restricted location data includes: the position data of the sound activity when the position data of the sound activity indicates that the sound activity is within the boundary pad; or When the position data of the sound activity represents the position outside the boundary pad when the sound activity is within the boundary pad.

56. The method of claim 55, wherein the boundary of the adjacent lobe region comprises a set of points that are equidistant from the selected lobe and adjacent lobes associated with the adjacent lobe region.

57. The method of claim 55, wherein determining the restricted location data comprises: calculating a motion vector between the selected lobe and the sound activity; calculating a projection vector between each of the plurality of lobes and the selected lobe; calculating a scaling factor for each of the plurality of lobes except the selected lobes; determining a minimum scaling factor of the calculated scaling factors for each of the plurality of lobes except the selected lobes; and A constrained motion vector is calculated by applying the minimum scaling factor to the motion vector, wherein whether the sound activity is outside or within the boundary pad is based on the constrained motion vector.

58. A method according to claim 49, wherein selecting the one of the multiple lobes to move is based on one or more of the following: (1) the relationship between the azimuth of the coordinate of the sound activity and the azimuth of the selected lobe and an azimuth threshold, or (2) the relationship between the elevation of the coordinate of the sound activity and the elevation of the selected lobe and an elevation threshold.

59. The method of claim 58, wherein selecting the one of the plurality of lobes to move is based on a distance of the coordinates of the sound activity from the array microphone.

60. The method of claim 59, further comprising setting the azimuth threshold based on the distance of the coordinates of the sound activity from the array microphone.

61. A method according to claim 58, wherein selecting the one of the multiple lobes to move includes selecting the selected lobe when (1) the absolute value of the difference between the azimuth angle of the coordinates of the sound activity and the azimuth angle of the selected lobe is not greater than the azimuth angle threshold; and (2) the absolute value of the difference between the elevation angle of the coordinates of the sound activity and the elevation angle of the selected lobe is greater than the elevation angle threshold.

62. The method of claim 48, wherein each of the plurality of petal regions comprises a space surrounding its associated petal that is closer to the associated petal than to any other petal of the plurality of petals.

63. The method of claim 48, further comprising calculating each of the plurality of lobe regions based on a definition received via a user interface.

64. The method of claim 48, further comprising calculating each of the plurality of lobe regions based on information from a sensor that detects the environment in which the array microphone is located.

65. The method of claim 47, further comprising: Receive remote audio signals from a far end; detecting an amount of activity of the remote audio signal; as well as When the amount of activity of the remote audio signal exceeds a predetermined threshold, performance of the steps of selecting the one of the plurality of lobes to move, determining the restricted position data, and repositioning the selected lobe is inhibited.

66. An array microphone system, comprising: a plurality of microphone elements, each of the plurality of microphone elements being configured to detect sound and output an audio signal; a beamformer in communication with the plurality of microphone elements, the beamformer configured to generate one or more beamformed signals based on the audio signals of the plurality of microphone elements, wherein the one or more beamformed signals correspond to one or more lobes, each lobe being positioned at a location in an environment; an audio activity locator in communication with the plurality of microphone elements, the audio activity locator being configured to determine (1) coordinates of new sound activity in the environment and (2) metrics associated with the new sound activity; and a flap autofocuser in communication with the audio activity locator and the beamformer, the flap autofocuser being constructed to: receiving the coordinates of the new sound activity and the metric associated with the new sound activity; determining whether the metric associated with the new voice activity is greater than or equal to a metric threshold; When it is determined that the metric associated with the new voice activity is greater than or equal to the metric threshold: determining a lobe region within which the coordinates of the new sound activity are located, wherein the lobe region includes an existing lobe, and the existing lobe includes one of the one or more lobes; determining, based on the coordinates of the new sound activity and parameters associated with the existing lobe and the lobe region, whether the coordinates of the new sound activity are near the existing lobe; When the coordinates of the new sound activity are determined to be near the existing lobe: limiting the updating of the position of the existing lobe to restricted coordinates within the lobe region around the existing lobe, wherein the restricted coordinates are based on the coordinates of the new sound activity and the parameters associated with the existing lobe and the lobe region; and The restricted coordinates are transmitted to the beamformer to cause the beamformer to update the position of the existing lobe to the restricted coordinates.

67. The system according to claim 66: wherein the parameters include an apparent radius in the lobe region around the existing lobe, the apparent radius including a space around the location of the existing lobe in which the new sound is believed to be active; and Wherein the lobe autofocuser is further constructed to determine whether the coordinates of the new sound activity are near an existing lobe by determining whether the coordinates of the new sound activity are within the appearance radius.

68. The system of claim 67, wherein the flap autofocuser is constructed to determine whether the coordinates of the new sound activity are within the appearance radius by: calculating a motion vector between the position of the existing lobe and the coordinates of the new sound activity; determining whether the magnitude of the motion vector is less than or equal to the appearance radius; When the magnitude of the motion vector is less than or equal to the appearance radius, the coordinates representing the new sound activity are within the appearance radius; and When the magnitude of the motion vector is greater than the appearance radius, the coordinates representing the new sound activity are outside the appearance radius.

69. The system according to claim 66: wherein the parameters include a movement radius in the flap region, the movement radius including a maximum distance the existing flap is permitted to move from the position of the existing flap; and Where the restricted coordinates are: including the coordinates of the new sound activity when the coordinates of the new sound activity are within the moving radius; or When the coordinates of the new sound activity are outside the moving radius, the moving radius is included.

70. The system of claim 69, wherein the flap autofocuser is constructed to restrict the update of the position of the existing flap to the restricted coordinates by: calculating a motion vector between the position of the existing lobe and the coordinates of the new sound activity; determining whether the magnitude of the motion vector is less than or equal to the movement radius; When the magnitude of the motion vector is less than or equal to the moving radius, the coordinates representing the new sound activity are within the moving radius; and When the magnitude of the motion vector is greater than the moving radius, the coordinates representing the new sound activity are outside the moving radius.

71. The system of claim 66: wherein the parameters include a boundary pad in the petal region, the boundary pad including a maximum distance from the position of the existing petal that the existing petal is permitted to move toward a boundary of an adjacent petal region; and Where the restricted coordinates are: including the coordinates of the new sound activity when the coordinates of the new sound activity are outside the boundary pad; or When the coordinates of the new sound activity are within the boundary pad, a location outside the boundary pad is included.

72. A system according to claim 71, wherein the boundary of the adjacent petal region comprises a set of points that are equidistant from the position of the existing petal and the position of an adjacent petal associated with the adjacent petal region.

73. The system of claim 71, wherein the flap autofocuser is constructed to restrict the update of the position of the existing flap to the restricted coordinates by: calculating a motion vector between the position of the existing lobe and the coordinates of the new sound activity; calculating a projection vector between the position of each of the one or more petals and the position of the existing petal; calculating a scaling factor for each of the one or more petals other than the existing petal; determining a minimum scaling factor of the calculated scaling factors for each of the one or more petals other than the existing petal; and A constrained motion vector is calculated by applying the minimum scaling factor to the motion vector, wherein whether the coordinates of the new sound activity are outside or within the boundary pad is based on the constrained motion vector.

74. The system of claim 66: wherein the metric comprises a confidence score for the new voice activity; and Wherein the flap autofocus is constructed to determine whether the metric associated with the new sound activity is greater than or equal to the metric threshold by determining whether the confidence score of the new sound activity is greater than or equal to the metric threshold.

75. The system of claim 74, wherein the confidence score represents one or more of the certainty of the coordinates of the new voice activity or the quality of the new voice activity.

76. A system according to claim 66, wherein the lobe autofocuser is constructed to determine the lobe area within which the coordinates of the new sound activity are located by determining the existing lobe near the new sound activity associated with the lobe area based on one or more of: (1) the relationship between the difference between the azimuth of the coordinates of the new sound activity and the azimuth of the position of the existing lobe and an azimuth threshold, or (2) the relationship between the difference between the elevation of the coordinates of the new sound activity and the elevation of the position of the existing lobe and an elevation threshold.

77. A system according to claim 76, wherein the lobe autofocus is constructed to determine the lobe area within which the coordinates of the new sound activity are located by determining the existing lobes near the new sound activity associated with the lobe area based on the distance of the coordinates of the new sound activity from the system.

78. The system of claim 77, wherein the flap autofocuser is further constructed to set the azimuth threshold based on the distance of the coordinates of the new sound activity from the system.

79. A system according to claim 76, wherein the lobe autofocus is constructed to determine the lobe area within which the coordinates of the new sound activity are located by determining the existing lobe near the new sound activity associated with the lobe area based on the following: (1) the absolute value of the difference between the azimuth angle of the coordinates of the new sound activity and the azimuth angle of the position of the existing lobe is not greater than the azimuth angle threshold; and (2) the absolute value of the difference between the elevation angle of the coordinates of the new sound activity and the elevation angle of the position of the existing lobe is greater than the elevation angle threshold.

80. A system according to claim 66, wherein the flap region includes a space around the position of the existing flap, the space being closer to the existing flap than the position of any other flap of the one or more flaps.

81. The system of claim 66, wherein the lobe autofocuser is further constructed to calculate the lobe area associated with each of the one or more lobes based on a definition received through a user interface.

82. The system of claim 66, wherein the lobe autofocuser is further constructed to calculate the lobe area associated with each of the one or more lobes based on information from a sensor that detects the environment in which the array microphone is located.

83. The system of claim 66: It further includes an activity detector in communication with the distal end and the flap autofocus, the activity detector being constructed to: receiving a remote audio signal from the remote end; detecting an amount of activity in the remote audio signal; and transmitting the detected amount of activity to the flap autofocus; and wherein the flap autofocus device is further constructed to: When the activity amount of the remote audio signal exceeds a predetermined threshold, the lobe autofocuser is suppressed to perform the following steps: determining whether the metric associated with the new sound activity is greater than or equal to the metric threshold, determining the lobe region within which the coordinates of the new sound activity are located, determining whether the coordinates of the new sound activity are near the existing lobe, limiting the update of the position of the existing lobe to restricted coordinates within the lobe region around the existing lobe, and transmitting the coordinates of the new sound activity to the beamformer.

84. The system of claim 66: It further includes an activity detector in communication with the distal end and the flap autofocus, the activity detector being constructed to: receiving a remote audio signal from the remote end; detecting an amount of activity of the remote audio signal; and When the amount of activity of the remote audio signal exceeds a predetermined threshold, a signal is transmitted to the lobe autofocuser to cause the lobe autofocuser to stop performing the following steps: determining whether the metric associated with the new sound activity is greater than or equal to the metric threshold, determining the lobe region within which the coordinates of the new sound activity are located, determining whether the coordinates of the new sound activity are near the existing lobe, limiting the update of the position of the existing lobe to restricted coordinates within the lobe region around the existing lobe, and transmitting the coordinates of the new sound activity to the beamformer.

85. An array microphone system, comprising: a plurality of microphone elements, each of the plurality of microphone elements being configured to detect sound and output an audio signal; a beamformer in communication with the plurality of microphone elements, the beamformer configured to generate one or more beamformed signals based on the audio signals of the plurality of microphone elements, wherein the one or more beamformed signals correspond to one or more lobes, each lobe being positioned at a location in an environment; an audio activity locator in communication with the plurality of microphone elements, the audio activity locator being configured to determine (1) coordinates of new sound activity in the environment and (2) a confidence score associated with the new sound activity; and a flap autofocuser in communication with the audio activity locator and the beamformer, the flap autofocuser being constructed to: receiving the coordinates of the new voice activity and the confidence score associated with the new voice activity; determining whether the confidence score of the new voice activity is greater than or equal to a threshold; When it is determined that the confidence score of the new voice activity is greater than the threshold: determining a lobe region within which the coordinates of the new sound activity are located, wherein the lobe region includes an existing lobe, and the existing lobe includes one of the one or more lobes; determining whether the coordinates of the new sound activity are within an appearance radius of the location of the existing lobe in the lobe region, wherein the appearance radius includes a space around the location of the existing lobe within which the new sound activity is believed to be; When determining that the coordinates of the new sound activity are within the appearance radius of the position of the existing petal, determining whether the coordinates of the new sound activity are within a movement radius of the position of the existing petal within the petal region, wherein the movement radius includes a maximum distance that the existing petal is permitted to move from the position of the existing petal; When it is determined that the coordinates of the new sound activity are within the movement radius of the position of the existing petal within the petal area: determining whether the coordinates of the new sound activity are close to a boundary pad in the lobe region, wherein the boundary pad comprises a maximum distance from the position of the existing lobe that the existing lobe is allowed to move toward an adjacent lobe region; When it is determined that the coordinates of the new sound activity are close to the boundary pad: modifying the coordinates of the new sound activity to first modified coordinates outside the boundary pad; and transmitting the first modified coordinates to the beamformer to cause the beamformer to update the position of the existing lobe to the first modified coordinates; and When it is determined that the coordinates of the new sound activity are not close to the boundary pad, transmitting the coordinates of the new sound activity to the beamformer to update the position of the existing lobe to the coordinates of the new sound activity; When it is determined that the coordinates of the new sound activity are not within the movement radius of the position of the existing petal within the petal area: modifying the coordinates of the new sound activity to second modified coordinates within the movement radius; determining whether the second modified coordinate is close to the boundary pad of the lobe region; When it is determined that the second modified coordinate is close to the boundary pad: modifying the second modified coordinate to a third modified coordinate outside the boundary pad; and transmitting the third modified coordinates to the beamformer to cause the beamformer to update the position of the existing lobe to the third modified coordinates; and When it is determined that the coordinates of the new sound activity are not proximate to the boundary pad, the second modified coordinates are transmitted to the beamformer to update the position of the existing lobe to the second modified coordinates.

Images