Audio Capture in the Presence of Noise

Abstract

An apparatus includes components for the following operations: capturing an audio scene using a plurality of microphones; determining that the captured audio scene has unacceptable detection noise; depending on determining that the captured audio scene has unacceptable detection noise, searching both different sets of one or more microphones and different physical rotation angles of the microphones to find a combination of a first set of one or more microphones and a first physical rotation angle of the microphones that captures the audio scene with acceptable detection noise; and controlling the physical rotation angle of the microphones to be the first physical rotation angle of the microphones and capturing the audio scene using the combination of the first set of one or more microphones and the first physical rotation angle of the microphones.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to audio capture in the presence of noise. Background Art

[0002] An audio scene captured using one or more microphones may be affected by noise.

[0003] In some examples, the signal (audio scene) can be improved and noise reduced by digitally processing the audio scene using a noise reduction algorithm. In some examples, frequency-dependent attenuation can be used. In some cases, the correlation of signals from spatially separated microphones can be used.

[0004] However, noise reduction algorithms are not always effective. This is especially true in the case of noise prevention or noise suppression during signal capture.

[0005] In some cases, the dynamic response of a microphone can saturate due to its response to noise. In such a case, signals other than the noise cannot be detected. Summary of the Invention

[0006] According to various but not necessarily all embodiments, there is provided an apparatus including components for:

[0007] capturing an audio scene using a plurality of microphones;

[0008] determining that the captured audio scene has unacceptable detected noise;

[0009] depending on determining that the captured audio scene has unacceptable detected noise,

[0010] searching both different groups of one or more microphones and different physical rotation angles of the microphones to find a combination of a first group of one or more microphones and a first physical rotation angle of the microphones that captures an audio scene having acceptable detected noise; and

[0011] controlling the physical rotation angle of the microphones to be the first physical rotation angle of the microphones, and capturing an audio scene using the combination of the first group of one or more microphones and the first physical rotation angle of the microphones.

[0012] In some but not necessarily all examples, the component for controlling the physical rotation angle of the microphones includes a component for electrically controlling the physical rotation angle of a mounted camera, the mounted camera including an image sensor for capturing a visual scene and at least some of the plurality of microphones.

[0013] In some but not necessarily all examples, the apparatus includes components for automatically tracking a selected visual object to keep the selected visual object within the captured visual scene.

[0014] In some but not necessarily all examples, the device includes components for automatically moving a logical sub - part of an image sensor that is used to capture an image of a visual scene across the image sensor to keep the selected visual object within the captured visual scene.

[0015] In some but not necessarily all examples, the apparatus includes components for automatically resizing a logical portion of an image sensor that is used to capture an image of a visual scene to keep the selected visual object within the captured visual scene.

[0016] In some but not necessarily all examples, the apparatus includes components for enabling a user to select a visual object.

[0017] In some but not necessarily all examples, the apparatus includes an additional camera that includes an image sensor for capturing an image of a visual scene, where the camera and the additional camera have different viewing angles, and the apparatus includes components for using the camera or the additional camera to keep the selected visual object within the captured visual scene.

[0018] In some but not necessarily all examples, searching both different sets of one or more microphones and different physical rotation angles of the microphones to find a combination of a first set of one or more microphones and a first physical rotation angle of the microphones is dependent on one or more parameters controlled by a user, the one or more parameters changing a relative priority between the captured visual scene and the captured audio scene, and the combination capturing an audio scene with an acceptable detection noise.

[0019] In some but not necessarily all examples, the components for controlling the physical rotation angle of a microphone include components for electrically controlling the physical rotation angle of a mounted camera that includes an image sensor for capturing a visual scene and the microphone, where the search for different physical rotation angles of the microphone is a constrained search, and the physical rotation angle of the mounted camera is constrained such that the selected visual object remains within the captured visual scene at the first physical rotation angle of the microphone.

[0020] In some but not necessarily all examples, the search for different sets of one or more microphones is a constrained search, where noise microphones are removed from the different sets of one or more microphones.

[0021] In some but not necessarily all examples, the different physical rotation angles of the microphones being searched include multiple components, where searching for different physical rotation angles of the microphones includes searching for different combinations of values of the components of the physical rotation angles of the microphones.

[0022] In some but not necessarily all examples, searching both for different groups of one or more microphones and for different physical rotation angles of the microphones to find, as a result, a combination of a first group of one or more microphones and a first physical rotation angle of the microphones, is a search for

[0023] a result having the largest number of undistorted microphones

[0024] or

[0025] a result that optimally performs a noise reduction algorithm, the combination capturing an audio scene with acceptable detected noise.

[0026] In some but not necessarily all examples, the apparatus includes components for detecting unacceptable detected noise using a wind noise detection algorithm.

[0027] According to various but not necessarily all embodiments, a computer program is provided that, when run by one or more processors, implements:

[0028] depending on determining that the captured audio scene has unacceptable detected noise,

[0029] searching both for different groups of one or more microphones of a microphone arrangement and for different physical rotation angles of the microphone arrangement to find a combination of a first group of one or more microphones of the microphone arrangement and a first physical rotation angle of the microphone arrangement that captures an audio scene with acceptable detected noise; and

[0030] controlling the physical rotation angle of the microphone arrangement to be the first physical rotation angle of the microphone arrangement and using the combination of the first group of one or more microphones of the microphone arrangement and the first physical rotation angle of the microphone arrangement to capture the audio scene.

[0031] According to various but not necessarily all embodiments, a method is provided that includes:

[0032] determining that a captured audio scene captured using a microphone arrangement including a plurality of microphones has unacceptable detected noise;

[0033] Depending on determining that the captured audio scene has unacceptable detection noise, searching both different sets of one or more microphones of the microphone arrangement and different physical rotation angles of the microphone arrangement to find a combination of a first set of one or more microphones of the microphone arrangement and a first physical rotation angle of the microphone arrangement that captures an audio scene having acceptable detection noise; and

[0034] Controlling the physical rotation angle of the microphone arrangement to be the first physical rotation angle of the microphone arrangement, and using the combination of the first set of one or more microphones of the microphone arrangement and the first physical rotation angle of the microphone arrangement to capture an audio scene.

[0035] According to various but not necessarily all embodiments, examples are provided as claimed in the appended claims.

[0036] According to various but not necessarily all embodiments, an apparatus is provided that includes components for:

[0037] Capturing an audio scene using a microphone arrangement that includes a plurality of microphones;

[0038] Determining that the captured audio scene has unacceptable detection noise;

[0039] Depending on determining that the captured audio scene has unacceptable detection noise,

[0040] Searching both different sets of one or more microphones of the microphone arrangement and different physical rotation angles of the microphone arrangement to find a combination of a first set of one or more microphones of the microphone arrangement and a first physical rotation angle of the microphone arrangement that captures an audio scene having acceptable detection noise; and

[0041] Controlling the physical rotation angle of the microphone arrangement to be the first physical rotation angle of the microphone arrangement, and using the combination of the first set of one or more microphones of the microphone arrangement and the first physical rotation angle of the microphone arrangement to capture an audio scene.

[0042] According to various but not necessarily all embodiments, a method is provided that includes:

[0043] Capturing an audio scene;

[0044] Controlling the physical rotation of a mounted camera that includes an image sensor and a microphone to reposition the image sensor and the microphone, wherein repositioning the microphone reduces detection noise at the microphone;

[0045] Automatically moving a logical sub - portion of the image sensor that is used to capture an image of a visual scene across the image sensor to keep a selected visual object within the captured visual scene. Description of the Drawings

[0046] Some examples will now be described with reference to the drawings, in which:

[0047] Figures 1A to 1E Different views of an example of the device are illustrated;

[0048] Figure 2 An example of the device in use is illustrated;

[0049] Figure 3 Is an example illustrating a physical rotation angle;

[0050] Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D Another example of the device is illustrated;

[0051] Figure 5 An example of the device and its control architecture is illustrated;

[0052] Figure 6 An example of a method is illustrated;

[0053] Figure 7 An example of a search is illustrated;

[0054] Figure 8 An example of a search is illustrated;

[0055] Figure 9A 、 Figures 9B to 16A to Figure 16B Illustrate an image sensor ( Figures 9A to 16A ) and an example of a corresponding visual scene captured by the image sensor ( Figures 9B to 16B );

[0056] Figure 17A and Figure 17B An example of reducing the effect of wind noise is illustrated;

[0057] Figure 18A and Figure 18B Another example of reducing the effect of wind noise is illustrated;

[0058] Figure 19 Illustrate a controller; and

[0059] Figure 20 Illustrate a computer program. Detailed Description of the Invention

[0060] The following description and drawings relate to an example of a device 20 that includes components for the following operations:

[0061] Use a plurality of microphones 2 to capture an audio scene 4;

[0062] Determine that the captured audio scene 4 has unacceptable detection noise;

[0063] Depending on determining that the captured audio scene 4 has unacceptable detection noise, search both different groups of one or more microphones 2 and different physical rotation angles 44 of the microphones 2 to find a combination of a first group of one or more microphones 2 and a first physical rotation angle 44 of the microphones 2 for the audio scene 4 that captures with acceptable detection noise; and

[0064] Control the physical rotation angle 44 of the microphone 2 to be the first physical rotation angle 44 of the microphone 2, and use the combination of the first group of one or more microphones 2 and the first physical rotation angle 44 of the microphones 2 to capture the audio scene 4.

[0065] Search to find an acceptable combination of the physical rotation angle 44 and the microphones 2 used.

[0066] For example, acceptable detection noise can be noise that does not exceed a threshold. For example, unacceptable detection noise can be noise that does exceed the threshold.

[0067] The threshold can be, for example, the maximum response of the microphone 2. Acceptable detection noise can be, for example, noise that does not exceed the maximum response of the microphone 2, such as not exceeding the maximum pressure level of any microphone 2. Unacceptable detection noise can be, for example, noise that does exceed the maximum response of the microphone, such as does exceed the maximum pressure level of the microphone.

[0068] In some embodiments, the threshold can be a threshold for the level difference between two microphone signals. The level can be calculated as the signal energy over a short time period (e.g., 20 ms) and / or a narrow frequency band (width between 20 Hz - 2 kHz).

[0069] For example, the threshold can be a measure of the energy within one or more frequency bands. For example, wind noise can be detected as a high-intensity signal (gust) at lower frequencies and / or a high-intensity signal (wind shear) at higher frequencies.

[0070] Figures 1A to 1E and Figure 2 An example of the device 20 is shown. The device 20 includes a plurality of microphones 2 for capturing the audio scene 4.

[0071] The microphones 2 can have different physical rotation angles 44. The physical rotation angle 44 of the microphones 2 can be electrically controlled. The microphones 2 can be electrically controlled to move between different physical rotation angles 44.

[0072] In this example, the microphone 2 forms a microphone arrangement 10. In this example, the microphone arrangement 10 can have different physical rotation angles 44. The physical rotation angle 44 of the microphone arrangement 10 can be electrically controlled. The microphone arrangement 10 can be electrically controlled to move between different physical rotation angles 44.

[0073] Accordingly, in these examples, the apparatus 20 includes components for:

[0074] using a microphone arrangement 10 including a plurality of microphones 2 to capture an audio scene 4;

[0075] determining that the captured audio scene 4 has unacceptable detection noise;

[0076] depending on determining that the captured audio scene 4 has unacceptable detection noise,

[0077] searching both for different groups of one or more microphones 2 of the microphone arrangement 10 and for different physical rotation angles 44 of the microphone arrangement 10 to find a combination of a first group of one or more microphones 2 of the microphone arrangement 10 and a first physical rotation angle 44 of the microphone arrangement 10 that captures the audio scene 4 with acceptable detection noise; and

[0078] controlling the physical rotation angle 44 of the microphone arrangement 10 to be the first physical rotation angle 44 of the microphone arrangement 10 and using the combination of the first group of one or more microphones 2 of the microphone arrangement 10 and the first physical rotation angle 44 of the microphone arrangement 10 to capture the audio scene 4.

[0079] In at least some examples, the plurality of microphones 2 have fixed relative offsets and / or fixed orientations. The microphone arrangement 10 of the microphones can thus be a fixed arrangement of the microphones 2.

[0080] In some examples, one microphone 2 can be moved or oriented relative to another microphone 2.

[0081] The apparatus 20 is an audiovisual apparatus. It includes not only the microphone 2 as described above, but also at least one camera 22. The at least one camera 22 includes an image sensor 24 for capturing a visual scene 6.

[0082] Figure 1A is a left side view of the apparatus 20. Figure 1B is a bottom view of the apparatus 20. Figure 1C is a front view of the apparatus 20. Figure 1D is a right side view of the apparatus 20. Figure 1E is a top view of the apparatus 20.

[0083] In this illustrated example, but not necessarily in all examples, the camera 22 is located in front of the device 20 and the microphones 2 have different orientations. In this example, the microphones 2 have different orientations because they are associated with faces having different orientations. In this example, there is a microphone 2 on the left side face ( Figure 1A ), there is a microphone 2 on the bottom face ( Figure 1B ), there is a microphone 2 on the front face ( Figure 1C ), there is a microphone 2 on the right side face ( Figure 1D ) and there is a microphone 2 on the top face ( Figure 1E ).

[0084] In this example, the multiple microphones 2 have a fixed relative offset and a fixed orientation with respect to the camera 22.

[0085] The physical rotation angle 44 of the device 20 is also the physical rotation angle 44 of the microphones 2 and of the camera 22.

[0086] The device 20 can have different physical rotation angles 44. The physical rotation angle 44 of the device 20 can be electrically controlled and the device 20 can be electrically controlled to move between different physical rotation angles 44.

[0087] When the device 20 has different physical rotation angles 44, the microphone arrangement 10 (and the camera 22) has different physical rotation angles 44. When the physical rotation angle 44 of the device 20 is electrically controlled, the physical rotation angle 44 of the microphone arrangement 10 (and the camera 22) is electrically controlled. When the device 20 is electrically controlled to move between different physical rotation angles 44, the microphone arrangement 10 (and the camera 22) moves between different physical rotation angles 44.

[0088] The orientation direction (viewing angle) of the device 20 is illustrated by the orientation vector 30.

[0089] The camera 22 is mounted on the device 20. In this example, the device 20 moves on the support 40. The support 40 includes a plurality of joints 42. One or more of the joints are movable. The physical rotation angle 44 of the device 20 and thus the direction of the orientation vector 30 can be electrically controlled by electrically controlling the angles at one or more of the joints 42. This can be achieved using a motor (e.g., a servo motor).

[0090] Thus it should be understood that one or more components can be used to define the physical rotation angle 44, for example one for each joint angle.

[0091] Figure 3 An example of the physical rotation angle 44 defining the orientation vector 30 is illustrated. In this example, the physical rotation angle 44 consists of two spherical coordinate components, the elevation angle and the azimuth angle θ. The azimuth angle corresponds to panning, and the elevation angle corresponds to tilting.

[0092] In other examples, the physical rotation angle 44 that defines the orientation vector 30 can include three components: a tilt rotation (rotation about a lateral horizontal axis), a yaw rotation (rotation about a vertical axis), and a roll rotation (rotation about a longitudinal horizontal axis). Yaw corresponds to the plane.

[0093] Accordingly, the different physical rotation angles 44 of the microphone 2 being searched can include multiple components. Each component is a degree of freedom of rotation.

[0094] When the microphone 2 includes multiple components, searching for the different physical rotation angles 44 of the microphone 2 includes searching for different combinations of the values of the components of the physical rotation angle 44 of the microphone 2.

[0095] Figure 4A Another example of the device 20 as described above is illustrated.

[0096] In this example, one or more microphones 2 can be moved or oriented relative to one or more other microphones 2.

[0097] In this example, the device 20 rotates within the support 40 via the joint 42. A gimbal arrangement can be used to achieve the rotation of the device 20 and its mounted camera 22 in multiple directions.

[0098] In this example, one or more microphones 2 are positioned on the support 40. Accordingly, some but not all of the multiple microphones 2 do not rotate with the camera 22 and do not have a fixed relative offset and fixed orientation relative to the camera. The camera 22 can rotate relative to one or more microphones 2 (such as one or more microphones on the support 40).

[0099] Figure 4B Another example of the device 20 as described above is illustrated.

[0100] In this example, the device 20 is similar to the device described in the reference Figure 4A However, in this example, the device 20 includes multiple cameras 22 having different orientation vectors 30 (different perspectives). In some examples, the fields of view of the cameras 22 can overlap. The orientation vectors 20 of the cameras 22 can have a fixed relationship relative to each other.

[0101] Figure 4C Another example of the device 20 as described above is illustrated.

[0102] In this example, the apparatus 20 includes a plurality of cameras 22, which are arranged such that the fields of view of adjacent cameras 22 overlap. The effective field of view composed of the overlapping fields of view of the cameras can be 360 degrees. In this example, the images from the cameras 22 can be stitched together to obtain a panoramic image. For example, the panoramic image can be a 360-degree annular segment of a sphere at the equator (azimuth plane) of the sphere.

[0103] The orientation vectors 20 of the cameras 22 can have a fixed relationship relative to each other. Thus, the panoramic image can be invariant to 2D rotation of the apparatus 20 in the azimuth plane.

[0104] In this example, the fixed arrangement of the cameras 22 is rotationally symmetric. However, the fixed arrangement 10 of the microphones 2 ( Figure 4C not shown in the figure) can be asymmetric. Thus, noise reduction can be achieved by rotating the apparatus 20 (changing the orientation of the fixed microphone arrangement 10) without adversely affecting the captured visual scene 6.

[0105] The above example can be extended to the case where the cameras 22 are arranged in three dimensions such that the fields of view of adjacent cameras 22 overlap. The effective field of view composed of the overlapping fields of view of the cameras can be 360 degrees (or less) in the azimuth plane and can be between +90 degrees and -90 degrees in the polar plane. In this example, the images from the cameras 22 can be stitched together to obtain a spherical panoramic image. For example, the panoramic image can be the spherical surface. The panoramic image is invariant to 3D rotation of the apparatus 20.

[0106] Figure 4D Another example of the apparatus 20 as described above is illustrated. In this example, the apparatus 20 includes a fixed, asymmetric arrangement 10 of microphones 22. At time t1, most of the microphones 22 are exposed to noise, such as wind noise 80. However, after moving the apparatus 20, at time t2, most of the microphones 22 are not exposed to noise, for example, they are positioned on the leeward side of the apparatus 20 and are shielded from the wind noise 80 by the apparatus 20.

[0107] In some examples, the apparatus 20 rotates until an optimal rotation is obtained. In other examples, the apparatus 20 rotates by an amount estimated to be optimal.

[0108] Figure 5 An example of the apparatus 20 including a controller 70 is illustrated, which is configured to capture an audio scene 4 using a plurality of microphones 2.

[0109] The controller 70 is configured to process the input from the microphones 2 to detect noise in the captured audio scene 4 and is configured to determine whether the detected noise is acceptable or unacceptable.

[0110] The controller 70 is configured to perform a search in case the detected noise is unacceptable. The purpose of the search is to achieve capturing a new audio scene 4 with less noise, and preferably, the captured new audio scene 4 will have an acceptable detected noise.

[0111] The search is for both different groups of one or more microphones 2 and different physical rotation angles 44 of the microphones 2 to find a combination of a first group of one or more microphones 2 and a first physical rotation angle 44 of the microphones 2 of the microphone arrangement 10 that captures an audio scene 4 with an acceptable detected noise.

[0112] The controller 70 is additionally configured to control the physical rotation angle 44 of the microphones 2 to be the first physical rotation angle 44 of the microphones 2.

[0113] The controller 70 is additionally configured to capture the audio scene 4 using a combination of a first group of one or more microphones 2 and a first physical rotation angle 44 of the microphones 2.

[0114] Figure 6 An example of the method 100 is illustrated. In this example, the method 100 is a method for capturing an audio scene 4.

[0115] At block 102, the method 100 includes: capturing an audio scene 4 using a plurality of microphones 2.

[0116] At block 104, the method 100 includes determining that the captured audio scene 4 has an unacceptable detected noise.

[0117] If it is determined at block 104 that the captured audio scene 4 has an unacceptable detected noise, the method moves to block 106. If it is determined at block 104 that the captured audio scene 4 has an acceptable detected noise, the method stops, or if it is operating continuously, it returns to block 102 (with or without a delay).

[0118] At block 106, the method 100 includes searching for both

[0119] i) different groups of one or more microphones 2 of the microphone arrangement 10, and

[0120] ii) different physical rotation angles 44 of the microphone arrangement 10

[0121] to find a combination of a first group of one or more microphones 2 of the microphone arrangement 10 and a first physical rotation angle 44 of the microphone arrangement 10 that captures an audio scene 4 with an acceptable detected noise.

[0122] At block 106, method 100 includes controlling the physical rotation angle 44 of microphone arrangement 10 to be a first physical rotation angle 44 of microphone arrangement 10.

[0123] Method 100 then includes capturing audio scene 4 using a combination of a first set of one or more microphones 2 of microphone arrangement 10 and the first physical rotation angle 44 of microphone arrangement 10.

[0124] Method 100 may stop, or if it is operating continuously, may return to block 102 to capture audio scene 4.

[0125] The search may be a search over all combinations of one or more microphones 2 and the physical rotation angle 44 of the microphones 2. The first set of one or more microphones 2 and the first physical rotation angle 44 may, in this example, be the combination of the set of one or more microphones 2 and the physical rotation angle 44 that maximizes (or minimizes) a target cost function.

[0126] In some but not necessarily all examples, the maximum (or minimum) value of the target cost function may be achieved when the least amount of noise is detected in the audio scene.

[0127] In all other examples, the maximum (or minimum) value of the target cost function may be achieved when the audio scene detects noise below a certain threshold and meets some other criterion. Examples of suitable criteria include any one or any combination of the following:

[0128] The largest number of microphones is used;

[0129] The number of microphones used exceeds a threshold;

[0130] The spatial arrangement of the microphones used supports spatial audio;

[0131] The spatial arrangement of the microphones used is non - linear;

[0132] The spatial arrangement of the microphones used is three - dimensional;

[0133] The audio scene has the highest quality;

[0134] The quality of the audio scene is above a threshold;

[0135] The quality of the audio scene after noise reduction processing is the highest;

[0136] The quality of the audio scene after noise reduction processing is above a threshold;

[0137] The audio scene includes a selected audio object;

[0138] The quality of the audio object is the highest;

[0139] The quality of the audio object is higher than the threshold;

[0140] The quality of the audio object after noise reduction processing is the highest;

[0141] The quality of the audio object after noise reduction processing is higher than the threshold;

[0142] The quality of the visual scene is the highest;

[0143] The quality of the visual scene is higher than the threshold;

[0144] The visual scene includes the selected visual object.

[0145] In at least some examples, the audio object and / or the visual object can be selected by the user using the user interface 50.

[0146] The technical audio object means the audio source. It does not necessarily mean object-based encoding of the audio.

[0147] Spatial audio in the capture stage means capturing a three-dimensional sound field by multiple microphones. Spatial audio in the rendering stage means rendering some or all of the three-dimensional sound field. The spatial audio in the capture and rendering stages can be focused on one or more sound sources with three-dimensional positions. This three-dimensional position can change over time. If a single sound source, such as the selected sound source, is received at multiple microphones 2, the spatial arrangement of the microphones used can support spatial audio. This can be determined, for example, by the correlation of the inputs from different microphones 2.

[0148] The objective cost function can depend on one or more parameters controlled by the user, and the one or more parameters change the relative priority between the captured visual scene 6 and the captured audio scene 4.

[0149] Therefore, searching for different groups of one or more microphones 2 and different physical rotation angles 44 of the microphones 2 to find a combination of the first group of one or more microphones 2 and the first physical rotation angle 44 of the microphones 2 for the audio scene 4 depends on one or more parameters controlled by the user. This combination captures acceptable detection noise, and the one or more parameters change the relative priority between the captured visual scene 6 and the captured audio scene 4.

[0150] The search uses an algorithm to find the captured visual scene 6 and the captured audio scene 4 that satisfy the relative priority and maximize / minimize the objective cost function.

[0151] In one example, the relative priority results in the best audio scene 4 regardless of the visual scene 6. In one example, the relative priority results in the best visual scene 6 regardless of the audio scene 4. In one example, the relative priority results in the best available audio scene 4 while maintaining the target visual scene 6. In one example, the relative priority results in the best available visual scene 6 while maintaining the target audio scene 4.

[0152] In one example, the objective cost function is a weighted sum of the "distance" between the captured audio scene 4 and the best or acceptable audio scene and the "distance" between the captured visual scene 6 and the best or acceptable visual scene. The relative priority between the captured visual scene 6 and the captured audio scene 4 can be controlled by changing the weight ratio used in the sum of the "distance" between the captured audio scene 4 and the best or acceptable audio scene and the "distance" between the captured visual scene 6 and the best or acceptable visual scene.

[0153] In some examples, a particular configuration of the detected noise of the microphone 2 (e.g., the noise detected at certain microphones 2 when at a certain physical rotation angle 44, the noise level, or the noise type) can be associated with a combination of a defined group of one or more microphones 2 or a defined physical rotation angle 44 of the microphone 2.

[0154] When a particular configuration of the detected noise of the microphone 2 is detected (e.g., the noise detected at certain microphones 2 when at a certain physical rotation angle 44, the noise level, or the noise type), the associated combination of the defined group of one or more microphones 2 and the defined physical rotation angle 44 of the microphone 2 can be used to capture the audio scene 4 without performing a search 106, or can be used as the starting position of the search 106.

[0155] In some examples, the defined group of one or more microphones 2 is the first group and the defined physical rotation angle 44 of the microphone 2 is the first physical rotation angle 44, and is found as a result of searching different groups of one or more microphones 2 and different physical rotation angles 44 of the microphone 2 to find a combination of the first group of one or more microphones 2 and the first physical rotation angle 44 of the microphone 2 that captures an acceptable detected noise for the audio scene 4. The search can be performed depending on determining that the captured audio scene 4 has an unacceptable detected noise. The detected noise can be recorded in a lookup database as a particular configuration of the detected noise associated with the combination of the defined group of one or more microphones 2 and the defined physical rotation angle 44 of the microphone 2.

[0156] In some examples, a particular configuration sequence of detected noise of microphone 2 (e.g., the noise, noise level, or noise type detected at certain microphones 2 at a certain physical rotation angle 44) can be associated with a (plurality of) microphone sequence / angle combinations. Each (plurality of) microphone / angle combination is a combination of a defined group of one or more microphones 2 and a defined physical rotation angle 44 of microphone 2.

[0157] Thus, a spatial variation pattern of the noise can be associated with a (plurality of) different microphone / angle combinations for capturing the audio scene 4 without performing the search 106 or as a starting point for the search 106. Each (plurality of) microphone / angle combination can have the same or different groups of microphones 2 and / or the same or different physical rotation angles 44 for use.

[0158] Thus, a temporal variation pattern of the noise can be associated with different (plurality of) microphone / angle combinations for capturing the audio scene 4 without performing the search 106. Each (plurality of) microphone / angle combination can have the same or different microphone groups 2 and / or the same or different physical rotation angles 44.

[0159] In some examples, machine learning can be used to classify the configurations of detected noise and associate different configurations of detected noise with different (plurality of) microphone / angle combinations.

[0160] Figure 7 Schematically illustrates the search 106. The search 106 involves searching 110 different groups of one or more microphones 2 and searching 120 different physical rotation angles 44 of microphone 2. The searches 110 and 120 are performed simultaneously to find as a result a combination of a first group of one or more microphones 2 and a first physical rotation angle 44 of microphone 2 of the audio scene 4 that captures the detected noise with an acceptable level.

[0161] The search 120 for different physical rotation angles 44 of microphone 2 can be a constrained search 122. For example, the physical rotation angle 44 of the mounted camera 22 can be constrained such that a selected visual object is kept within the captured visual scene 6 at the searched physical rotation angle 44 of microphone 2 and / or the physical rotation angle 44 of the mounted camera 22 can be constrained such that a selected audio object is kept within the captured audio scene 4 at the searched physical rotation angle 44 of microphone 2.

[0162] The search 110 for different groups of one or more microphones 2 can be a constrained search 112. For example, noise microphones 2 can be removed from the population of microphones used to populate the different groups of one or more microphones 2 used in the search.

[0163] Figure 8 An example is illustrated where search 106 finds a combination of a first set of one or more microphones 2 and a first physical rotation angle 44 of the microphone 2 as a result, and this combination captures an acceptable detection noise. Search 106 can be prioritized to produce 134 results favorable for spatial audio. Search 106 can be prioritized to produce 136 results favorable for high-performance algorithm noise reduction, such as wind noise reduction. Search 106 can be prioritized to produce 132 results favorable for high-quality mono or stereo audio.

[0164] In some examples, the spatial audio search 134 has priority over the noise reduction search 136, and the noise reduction search 136 has priority over mono or stereo audio.

[0165] The (multiple) microphone / angle combination suitable for spatial audio will be found first. If the (multiple) microphone / angle combination suitable for spatial audio cannot be found through search 134, then the (multiple) microphone / angle combination suitable for noise reduction will be searched for through search 136. If the (multiple) microphone / angle combination suitable for noise reduction cannot be found through search 136, then the (multiple) microphone / angle combination suitable for mono or stereo audio will be searched for through search 132.

[0166] In some examples, the (multiple) microphone / angle combination suitable for spatial audio will be searched for first, and then the (multiple) microphone / angle combination suitable for noise reduction will be searched for before the group of one or more microphones 2 is changed. Thus, for a specific group of one or more microphones 2, all available physical rotation angles 44 of the microphone 2 are searched using the search 134 that prioritizes spatial audio, and then if this search is unsuccessful, all available physical rotation angles 44 of the microphone 2 are searched using the search 136 that prioritizes noise reduction. If this search is unsuccessful, then the group of one or more microphones 2 is changed, for example, the number of microphones is reduced. For example, a noisy microphone can be removed from the group.

[0167] In some examples, search 106 finds a result with the largest number of undistorted microphones 2 or a result that best performs the noise reduction algorithm.

[0168] Figure 9A 、 Figure 10A 、 Figure 11A 、 Figure 12A 、 Figure 13A 、 Figure 14A 、 Figure 15A 、 Figure 16A Each of Figure 16A illustrates an example of the image sensor 24 of the camera 22.

[0169] Figure 9B 、Figure 10B , Figure 11B , Figure 12B , Figure 13B , Figure 14B , Figure 15B , Figure 16B Each of the diagrams in illustrates an example of the visual scene 6 captured by the image sensor 24 corresponding to Figure 9A , Figure 10A , Figure 11A , Figure 12A , Figure 13A , Figure 14A , Figure 15A , Figure 16A .

[0170] The image sensor 24 is an array of sensing elements (sensels). A sensing element is an accepted aggregation of "sensor elements", in a way analogous to a pixel of a "picture element". Each sensing element in the image sensor 24 can provide a pixel of the visual scene 6. The array is a fixed array of N rows by M columns of sensing elements. The sensing elements of the image sensor 24 can be identified by their row I (i = 1, 2... N) and their column j (j = 1, 2, 3... M).

[0171] The logical sub - part 26 of the image sensor 24 is a sub - array of sensing elements. The array is an array of n rows by m columns of sensing elements. The sensing elements of the sub - part can be identified by their row (i = a, a + 1,... a + n where a ≥ 1, N ≥ a + n) and their column (j = b, b + 1,... b + m where b ≥ 1, M ≥ b + m). Thus, the logical sub - part 26 is a rectangle of sensing elements with vertices (i, j) at (a, b), (a + n, b), (a, b + m), (a + n, b + m). The sub - part 26 is smaller than the entire image sensor 24.

[0172] The logical sub - part 26 can be used to generate the visual scene 6. That is, the outputs of the sensing elements from the logical sub - part 26 can be used to provide the pixels of the visual scene 6.

[0173] The logical sub - part 26 can be moved by changing (a, b), that is, by changing a and / or b. This can be logically achieved by the controller 70 by controlling which sensing elements of the image sensor 24 are accessed during the image sensor read - out period.

[0174] The size of the logical sub - part 26 can be readjusted by changing n and / or m. The parameters n and / or m can be variable, and in some but not necessarily all examples, the ratio n / m (aspect ratio) is locked. The controller 70 can logically achieve the readjustment of the size by controlling which sensing elements of the image sensor 24 are accessed during the sensor read - out period.

[0175] In at least some examples, the visual scene 6 that can be captured by the image sensor 24 can include a visual object 28, such as the "T" in these figures. In some examples, the visual object 28 is illustrated on the image sensor 24 and also in the visual scene 6 captured by the image sensor 24.

[0176] In some examples, the device 20 is configured to automatically track a selected visual object 28 to keep the selected visual object 28 within the captured visual scene 6. In some examples, the automatic tracking can include the physical movement of the camera 22 as described above. In some examples, the automatic tracking can include the virtual movement of a sub - portion 26 of the image sensor 24 that is used to capture the visual scene 6.

[0177] A user can select the visual object 28.

[0178] Figure 9A and Figure 9B 、 Figure 10A and Figure 10B 、 Figure 11A and Figure 11B Illustrate digital zoom.

[0179] Figure 9A Illustrate the sub - portion 26 of the image sensor 24 that is used to capture Figure 9B the visual scene 6 illustrated in Figure 10A Illustrate the sub - portion 26 of the image sensor 24 that is used to capture Figure 10B the visual scene 6 illustrated in Figure 11A Illustrate the sub - portion 26 of the image sensor 24 that is used to capture Figure 11B the visual scene 6 illustrated in

[0180] The controller 70 causes the magnification or reduction of the selected visual object 28 by readjusting the size of the logical sub - portion 26 of the image sensor 24 that is used to capture the visual scene 6. The smallest sub - portion 26 ( Figure 10A )) produces the maximum magnification ( Figure 10B ). The largest sub - portion 26 ( Figure 11A ) produces the maximum reduction ( Figure 11B ).

[0181] In at least some examples, the controller 70 can automatically readjust the size of the logical sub - portion 26 to keep the selected visual object 28 within the captured visual scene 6.

[0182] Figure 9A and Figure 9B 、 Figure 12A and Figure 12B 、 Figure 13A and Figure 13B Illustrate digital pan (horizontal and vertical).

[0183] Figure 9A illustrates a sub - part 26 of an image sensor 24 that is used to capture Figure 9B the image of the visual scene 6 illustrated in Figure 12A illustrates a sub - part 26 of an image sensor 24 that is used to capture Figure 12B the image of the visual scene 6 illustrated in Figure 13A illustrates a sub - part 26 of an image sensor 24 that is used to capture Figure 13B the image of the visual scene 6 illustrated in

[0184] The controller 70 causes a pan shot of the visual scene 6 by moving a logical sub - part 26 of the image sensor 24 that is used to capture the visual scene 6 across the image sensor 24.

[0185] In at least some examples, the controller 70 can automatically move the logical sub - part 26 to keep a selected visual object 28 within the captured visual scene 6.

[0186] Figure 9A and Figure 9B 、 Figure 10A and Figure 10B 、 Figure 11A and Figure 11B The sequence of Figure 9A and Figure 9B 、 Figure 12A and Figure 12B 、 Figure 13A and Figure 13B illustrates the tracking of a selected visual object 28 using digital zoom. Figure 14A and Figure 14B 、 Figure 15A and Figure 15B 、 Figure 16A and Figure 16B illustrates the tracking of a selected visual object 28 using digital zoom and / or digital pan (horizontal and / or vertical).

[0187] Figure 14A illustrates a sub - part 26 of an image sensor 24 that is used to capture the visual scene 6 at time t1. The captured visual scene 6 is illustrated in Figure 14B Figure 15A illustrates a logical sub - part 26 of an image sensor 24 that is used to capture the visual scene 6 at a later time t2. The captured visual scene 6 is illustrated in Figure 15B Figure 16A shows a logical sub - part 26 of an image sensor 24 that is used to capture the visual scene 6 at an even later time t3. The captured visual scene 6 is in Figure 16B ​​is illustrated in

[0188] The controller 70 pans and / or zooms the visual scene 6 by moving and / or resizing a logical sub-portion 26 of the image sensor 24 that is used to capture the visual scene 6.

[0189] In at least some examples, the controller 70 can automatically move and / or resize the logical sub-portion 26 to keep a selected visual object 28 within the captured visual scene 6.

[0190] In some examples, such as where the device 20 includes multiple cameras 22 with different viewing angles (e.g., see Figure 4B ), then each of the respective image sensors 24 of the respective cameras can be used as described above. In some examples, the controller 70 is configured to use the first camera 22 or the second camera 22 to keep a selected visual object 28 within the captured visual scene 6. That is, the controller 70 can switch from using the first camera 22 to using the second camera 22 to keep a selected visual object 28 within the captured visual scene 6.

[0191] Figure 17A and Figure 17B and Figure 18A and Figure 18B Schematically illustrate different examples of using the device 20. The device 20 can be as described above.

[0192] In Figure 17A , Figure 17B , Figure 18A , Figure 18B example, the wind 80 causes the output from the microphone 2 D to be distorted by wind noise to an unacceptable level, and the output from the beam steering microphone 2 G is good (not distorted by wind noise to an unacceptable level).

[0193] In Figure 17A , the captured audio scene 4 has unacceptable detected wind noise. As a result of determining that the captured audio scene 4 has unacceptable detected wind noise, the device 20 searches both different sets of one or more microphones 2 and different physical rotation angles 44 of the microphones 2 to find a combination of a first set of one or more microphones 2 and a first physical rotation angle 44 of the microphones 2 for the audio scene 4 that captures with acceptable detected wind noise. As illustrated in Figure 17B , the device 20 then controls the physical rotation angle 44 of the microphone 2 to be the first physical rotation angle 44 of the microphone 2 and uses the combination of the first set of one or more microphones and the first physical rotation angle 44 of the microphone 2 to capture the audio scene 4. In Figure 17AAmong them, two of the three microphones 2 have unacceptable wind noise. In Figure 17B Among them, two of the three microphones 2 have acceptable wind noise or no wind noise. The search has minimized the number of unusable microphones 2.

[0194] In Figure 18A Among them, the captured audio scene 4 has unacceptable detected wind noise. As a result of determining that the captured audio scene 4 has unacceptable detected wind noise, the device 20 searches both different groups of one or more microphones 2 and different physical rotation angles 44 of the microphones 2 to find a combination of a first group of one or more microphones 2 and a first physical rotation angle 44 of the microphones 2 that captures acceptable detected wind noise. As Figure 18B As illustrated in, the device 20 then controls the physical rotation angle 44 of the microphone 2 to be the first physical rotation angle 44 of the microphone 2 and uses the combination of the first group of one or more microphones and the first physical rotation angle 44 of the microphone 2 to capture the audio scene 4. In Figure 18A Among them, one of the two microphones 2 has unacceptable wind noise. In Figure 18B Among them, two of the two microphones 2 have acceptable wind noise or no wind noise. The search has minimized the number of unusable microphones 2.

[0195] Figure 17A And Figure 17B Illustrates the first physical rotation angle 44 as a yaw rotation. Figure 18A And Figure 18B Illustrates the first physical rotation angle 44 as a roll rotation. In some examples, the first physical rotation angle 44 found by the search can be a combination of both a yaw rotation and a roll rotation.

[0196] For example, wind noise can be detected through an audio clip.

[0197] In some cases, the dynamic response of the microphone 2 saturates due to its response to noise such as wind noise. For example, the maximum sound pressure level may be exceeded. In such a case, signals other than (wind) noise cannot be detected.

[0198] For example, wind noise can be detected by digitally processing the output from the microphone 2. A wind noise detection (WND) algorithm can be used to detect unacceptable detected wind noise.

[0199] In some examples, the WND algorithm can detect wind noise by detecting high energy and / or high variability at low frequencies (<50 Hz).

[0200] The controller 70 may be configured to digitally process the audio scene 4 captured using a combination of a first set of one or more microphones 2 and a first physical rotation angle 44 of the microphone 2 using a wind noise reduction algorithm (WNR).

[0201] In some examples, a band-pass filter or other frequency-selective filter may be used to attenuate wind noise.

[0202] In any of the foregoing examples, the camera 22 or the camera 22 may be configured to capture still images. In any of the foregoing examples, the camera 22 or the camera 22 may be configured to capture sequential images (video).

[0203] In any of the foregoing examples, the device 20 may be a security device.

[0204] In any of the foregoing examples, the device 20 may be a monitoring device.

[0205] In any of the foregoing examples, the device 20 may be a fixed-position device.

[0206] In any of the foregoing examples, the device 20 may be a mobile device.

[0207] In any of the foregoing examples, the arrangement 10 of the microphones 2 may be a non-linear microphone arrangement 10. A linear microphone arrangement is an arrangement of the microphones 2 in which the microphones 2 are aligned along a line.

[0208] A non-linear microphone arrangement is an arrangement of the microphones 2 that is not a linear microphone arrangement. For example, a non-linear microphone arrangement may have the microphones 2 distributed over an area or volume.

[0209] In any of the foregoing examples, the microphone arrangement in the microphone group used to capture the audio scene may be a non-linear microphone arrangement 10.

[0210] In any of the foregoing examples, the device may include any number of cameras 22, including no cameras 22. In some examples, the device 20 may be an audio device without a camera.

[0211] The above conditional search may be permanently operable or may be operable as a result of user input. For example, Figure 6 the method may become operable in response to user input at the user interface 50. In some examples, the user interface 50 may provide the user with live audio and the user may cause Figure 6 the method to be operable in real time to improve the captured audio.

[0212] Figure 19An example of the controller 70 is illustrated. The implementation of the controller 70 can be a controller circuit system. The controller 70 can be implemented solely in hardware, have certain aspects of software that include only firmware, or can be a combination of hardware and software (including firmware).

[0213] As Figure 19 illustrated, the controller 70 can be implemented using instructions that enable hardware functionality, such as by using a medium (disk, memory, etc.) of executable instructions of a computer program 76 in a general - purpose or special - purpose processor 72, and the executable instructions can be stored on a computer - readable memory for execution by such a processor 72.

[0214] The processor 72 is configured to read from and write to the memory 74. The processor 72 may also include an output interface and an input interface. The processor 72 outputs data and / or commands via the output interface, and data and / or commands are input into the processor 72 via the input interface.

[0215] The memory 74 stores the computer program 76, which includes computer program instructions (computer program code). When loaded into the processor 72, the computer program instructions (computer program code) control the operation of the device 20. The computer program instructions of the computer program 76 provide the logic and routines that enable the device to perform the methods illustrated and / or described. The processor 72 can load and execute the computer program 76 by reading the memory 74.

[0216] The device 20 thus includes:

[0217] At least one processor 72; and

[0218] At least one memory 74 including computer program code,

[0219] The at least one memory 74 and the computer program code are configured to, together with the at least one processor 72, cause the device 20 to at least perform:

[0220] Depending on determining that the captured audio scene has unacceptable detection noise,

[0221] Searching both different groups of one or more microphones of the microphone arrangement and different physical rotation angles of the microphone arrangement to find a combination of a first group of one or more microphones of the microphone arrangement for the audio scene and a first physical rotation angle of the microphone arrangement, the combination of which captures audio with acceptable detection noise; and

[0222] Controlling the physical rotation angle of the microphone arrangement to be the first physical rotation angle of the microphone arrangement and using the combination of the first group of one or more microphones of the microphone arrangement and the first physical rotation angle of the microphone arrangement to capture the audio scene.

[0223] As Figure 20 illustrated, the computer program 76 can reach the device 20 via any suitable delivery mechanism 78. The delivery mechanism 78 can be, for example, a machine-readable medium, a computer-readable medium, a non-transitory computer-readable storage medium, a computer program product, a memory device, a recording medium such as a compact disc read-only memory (CD-ROM) or a digital versatile disc (DVD) or a solid-state memory, including or tangibly embodying the article of manufacture of the computer program 76. The delivery mechanism can be a signal configured to reliably transmit the computer program 76. The device 20 can propagate or transmit the computer program 76 as a computer data signal.

[0224] Computer program instructions for causing the device to perform at least the following operations or for performing at least the following operations:

[0225] Depending on determining that the captured audio scene has unacceptable detection noise,

[0226] Search different groups of one or more microphones of the microphone arrangement and different physical rotation angles of the microphone arrangement to find a combination of a first group of one or more microphones of the microphone arrangement for the audio scene and a first physical rotation angle of the microphone arrangement, the combination of which captures audio with acceptable detection noise; and

[0227] Control the physical rotation angle of the microphone arrangement to be the first physical rotation angle of the microphone arrangement, and use the combination of the first group of one or more microphones of the microphone arrangement and the first physical rotation angle of the microphone arrangement to capture the audio scene.

[0228] The computer program instructions can be included in a computer program, a non-transitory computer-readable medium, a computer program product, a machine-readable medium. In some but not necessarily all examples, the computer program instructions can be distributed over more than one computer program.

[0229] Although the memory 74 is illustrated as a single component / circuit system, it can be implemented as one or more separate component / circuit systems, some or all of which can be integrated / removable and / or can provide permanent / semi-permanent / dynamic / cache storage.

[0230] Although the processor 72 is illustrated as a single component / circuit system, it can be implemented as one or more separate component / circuit systems, some or all of which can be integrated / removable. The processor 72 can be a single-core or multi-core processor.

[0231] References to "computer-readable storage media", "computer program products", "tangibly embodied computer programs", etc. or "controllers", "computers", "processors", etc. should be understood to cover not only computers with different architectures, such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures, but also dedicated circuits, such as field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), signal processing devices, and other processing circuitry. References to computer programs, instructions, code, etc. should be understood to cover software for programmable processors or firmware, such as, for example, the programmable content of a hardware device, whether instructions for a processor or configuration settings for a fixed-function device, gate array, or programmable logic device, etc.

[0232] As used in this application, the term "circuitry" may refer to one or more or all of the following:

[0233] (a) A pure hardware circuit implementation (such as an implementation solely in analog and / or digital circuitry) and

[0234] (b) A combination of hardware circuitry and software, such as (where applicable):

[0235] (i) A combination of (one or more) analog and / or digital hardware circuitry and software / firmware, and

[0236] (ii) Any portion of (one or more) hardware processors (including (one or more) digital signal processors), software, and (one or more) memories with software that work together to cause a device such as a mobile phone or server to perform various functions, and

[0237] (c) (One or more) hardware circuits and / or (one or more) processors that require software (such as firmware) to operate, such as (one or more) microprocessors or a portion of (one or more) microprocessors, but the software may not be present when not needed for operation.

[0238] This definition of circuitry applies to all uses of the term in this application, including all uses in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of only hardware circuitry or a processor and its (or their) accompanying software and / or firmware. By way of example and where applicable to a particular claim element, the term circuitry also covers a baseband integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or networking device.

[0239] Blocks illustrated and / or described in the figures may represent steps in a method and / or segments of code in a computer program 76. The illustration of a particular order of blocks does not necessarily imply a required or preferred order of the blocks, and the order and arrangement of the blocks may be changed. Additionally, some blocks may be omitted.

[0240] The foregoing description also describes a class of apparatuses, where the apparatus 20 includes components for:

[0241] using a microphone arrangement 10 including a plurality of microphones 2 to capture an audio scene 4;

[0242] determining that the captured audio scene 4 has unacceptable detection noise;

[0243] depending on determining that the captured audio scene 4 has unacceptable detection noise,

[0244] searching both different groups of one or more microphones 2 of the microphone arrangement 10 and different physical rotation angles 44 of the microphone arrangement 10 to find a combination of a first group of one or more microphones 2 of the microphone arrangement 10 and a first physical rotation angle 44 of the microphone arrangement 10 that captures the audio scene 4 with acceptable detection noise; and

[0245] controlling the physical rotation angle 44 of the microphone arrangement 10 to be the first physical rotation angle 44 of the microphone arrangement 10 and using the combination of the first group of one or more microphones 2 of the microphone arrangement 10 and the first physical rotation angle 44 of the microphone arrangement 10 to capture the audio scene 4.

[0246] The foregoing description also describes a class of methods, where the method includes:

[0247] capturing an audio scene 4;

[0248] controlling the physical rotation of a mounted camera 22 including an image sensor 24 and a microphone 2 to reposition the image sensor 24 and the microphone 2, where repositioning the microphone 2 reduces the detection noise at the microphone 2;

[0249] automatically moving a logical sub - part 26 of the image sensor 24 that is used to capture an image of a visual scene 6 on the image sensor 24 to keep a selected visual object within the captured visual scene 6.

[0250] Where structural features have been described, they may be replaced by components that perform one or more functions of the structural features, whether or not those functions are explicitly or implicitly described.

[0251] The recording of data can include only temporary records, or it can include permanent records, or it can include both temporary and permanent records. Temporary records mean temporary recordings of data. This can occur, for example, during sensing or image capture, at dynamic memory, at buffers such as circular buffers, registers, caches, and the like. Permanent records mean that the data takes the form of an addressable data structure that can be retrieved from an addressable memory space and can thus be stored and retrieved until it is deleted or overwritten, but long-term storage may or may not occur. The use of the term "capture" related to images or audio is related to the temporary recording of image data. Whenever an image or audio is captured, it can subsequently be stored. The use of the term "store" relates to the permanent recording of image or audio data.

[0252] Systems, devices, methods, and computer programs can use machine learning, which can include statistical learning. Machine learning is a field of computer science that enables computers to learn without being explicitly programmed. A computer learns from experience E about a class of tasks T and a performance measure P whether its performance (measured by P) on the tasks in T improves with experience E. A computer can often learn from previous training data to make predictions about future data. Machine learning includes fully or partially supervised learning and fully or partially unsupervised learning. It can enable discrete outputs (e.g., classification, clustering) and continuous outputs (e.g., regression). For example, machine learning can be implemented using different methods such as cost function minimization, artificial neural networks, support vector machines, and Bayesian networks. For example, cost function minimization can be used for linear and polynomial regression and K-means clustering. Artificial neural networks (e.g., with one or more hidden layers) model complex relationships between input vectors and output vectors. Support vector machines can be used for supervised learning. Bayesian networks are directed acyclic graphs that represent the conditional independence of multiple random variables.

[0253] The above examples can be used as enabling components for the following components:

[0254] Automotive systems; telecommunications systems; electronic systems, including consumer electronics; distributed computing systems; media systems for generating or presenting media content, where the media content includes audio, visual, and audiovisual content as well as mixed, mediated, virtual, and / or augmented reality; personal systems, including personal health systems or personal fitness systems; navigation systems; user interfaces, also known as human-machine interfaces; networks, including cellular, non-cellular, and optical networks; ad-hoc networks; the Internet; the Internet of Things; virtualized networks; and related software and services.

[0255] The term "comprising" as used in this document has an inclusive rather than an exclusive meaning. That is, any reference to X comprising Y indicates that X can include only one Y or can include more than one Y. If it is intended to use "comprising" with an exclusive meaning, it will be stated in the context by referring to "comprising only one.." or by using "consisting of".

[0256] In this description, various examples have been referred to. The description of a feature or function related to an example indicates that those features or functions exist in that example. The use of the term "example" or "for example" or "may" or "might" in the text indicates that, whether explicitly stated or not, these features or functions exist at least in the described example, whether described as an example or not, and they may but do not necessarily exist in some or all other examples. Thus, "example" or "for example", "may" or "might" refer to a particular instance within a class of examples. The properties of an instance can be properties of only that instance or properties of the class or properties of a subclass of the class that includes some but not all instances of the class. Thus, features described with reference to one example rather than another are implicitly disclosed and can, where possible, be used as part of a working combination in that other example, but do not necessarily have to be used in their entirety in that other example.

[0257] Although examples have been described with reference to various examples in the preceding paragraphs, it should be understood that the given examples can be modified without departing from the scope of the claims.

[0258] The features described in the foregoing description can be used in combinations other than those explicitly described above.

[0259] Although functions have been described with reference to certain features, those functions can also be performed by other features, whether or not they are described.

[0260] Although features have been described with reference to certain examples, those features can also exist in other examples, whether or not they are described.

[0261] The term "a" or "the" as used in this document has an inclusive rather than an exclusive meaning. That is, unless the context clearly indicates to the contrary, any reference to X including (a / the) Y indicates that X can include only one Y or can include more than one Y. If it is intended to use "a" or "the" with an exclusive meaning, it will be stated explicitly in the context. In some cases, "at least one" or "one or more" can be used to emphasize the inclusive meaning, but the absence of these terms should not be taken as inferring any exclusive meaning.

[0262] The presence of a feature (or combination of features) in a claim is a reference to that feature or (combination of features) itself, and also to features (equivalent features) that achieve substantially the same technical effect. Equivalent features include, for example, features that are variants and that achieve substantially the same result in substantially the same way. Equivalent features include, for example, features that perform substantially the same function in substantially the same way to achieve substantially the same result.

[0263] In this specification, various examples have been referred to using adjectives or adjective phrases to describe the characteristics of the examples. Such a description of a characteristic associated with an example indicates that the characteristic is present in some examples exactly as described and is present in other examples substantially as described.

[0264] Although efforts have been made in the foregoing specification to draw attention to those features considered to be important, it should be understood that the applicant may seek protection by means of the claims for any patentable feature or combination of features mentioned above and / or shown in the drawings, whether or not emphasized above.

Claims

1. An apparatus, comprising components for the following operations: Using a plurality of microphones to capture an audio scene; Determining that the captured audio scene has unacceptable detection noise; Depending on determining that the captured audio scene has unacceptable detection noise, Searching multiple combinations of the plurality of microphones and multiple physical rotation angles of the plurality of microphones to determine a combination of a first set of one or more microphones and a first physical rotation angle of the one or more microphones, the combination capturing the audio scene with acceptable detection noise, wherein determining that the one or more microphones capture the audio scene with acceptable detection noise is based on a cost function; And Controlling the physical rotation angle of the microphone to the first physical rotation angle of the microphone, and capturing the audio scene through the combination of the first set of one or more microphones and the first physical rotation angle of the one or more microphones.

2. The apparatus according to claim 1, wherein the component for controlling the physical rotation angle of the microphone comprises: Components for electrically controlling the physical rotation angle of a mounted camera, the mounted camera including an image sensor for capturing a visual scene and at least some of the plurality of microphones.

3. The apparatus according to claim 2, wherein the apparatus comprises: Components for automatically tracking a selected visual object to keep the selected visual object within the captured visual scene.

4. The apparatus according to claim 2, wherein the apparatus comprises components for the following operation: Automatically moving a logical sub - part of the image sensor that is used to capture an image of the visual scene across the image sensor to keep the selected visual object within the captured visual scene.

5. The apparatus according to claim 2, wherein the apparatus comprises components for the following operation: Automatically adjusting the size of a logical part of the image sensor that is used to capture an image of the visual scene to keep the selected visual object within the captured visual scene.

6. The apparatus according to claim 3, comprising components for enabling a user to select the visual object.

7. The apparatus according to claim 2, further comprising an additional camera, the additional camera including an image sensor for capturing a visual scene, in which the camera and the additional camera have different perspectives, the apparatus comprising components for using the camera or the additional camera to keep the selected visual object within the captured visual scene.

8. The apparatus according to claim 2, wherein searching the multiple combinations of the plurality of microphones and the multiple physical rotation angles of the plurality of microphones to determine the combination of the first set of one or more microphones and the first physical rotation angle of the one or more microphones depends on one or more parameters controlled by a user, the one or more parameters changing the relative priority between the captured visual scene and the captured audio scene, the combination capturing the audio scene with acceptable detection noise.

9. The apparatus according to claim 1, wherein the component for controlling the physical rotation angle of the microphone comprises a component for electrically controlling the physical rotation angle of a mounted camera, the mounted camera comprising an image sensor for capturing a visual scene and the microphone, wherein the search for different physical rotation angles of the microphone is a constrained search, and wherein the physical rotation angle of the mounted camera is constrained such that a selected visual object remains within the captured visual scene at the first physical rotation angle of the microphone.

10. The apparatus according to claim 1, wherein the search for the plurality of combinations of the plurality of microphones and the plurality of physical rotation angles of the plurality of microphones is a constrained search, and wherein noise microphones are removed from the different groups of one or more microphones.

11. The apparatus according to claim 1, wherein the different physical rotation angles of the microphone being searched comprise a plurality of components, and wherein searching for the plurality of physical rotation angles of the microphone comprises searching for different combinations of values of the components of the physical rotation angle of the microphone.

12. The apparatus according to claim 1, wherein searching for the plurality of combinations of the plurality of microphones and the plurality of physical rotation angles of the plurality of microphones to find as a result a combination of a first set of one or more microphones and a first physical rotation angle of the one or more microphones is a search to find a result having the largest number of undistorted microphones or a result that optimally performs a noise reduction algorithm, the combination capturing the audio scene having an acceptable detected noise.

13. The apparatus according to any one of the preceding claims, comprising a component for detecting unacceptable detected noise using a wind noise detection algorithm.

14. A computer program product, the computer program product comprising a computer program which, when executed by a processor of a device, causes the device to: depending on determining that the captured audio scene has unacceptable detected noise, search for a plurality of combinations of a plurality of microphones of a microphone arrangement and a plurality of physical rotation angles of the plurality of microphones of the microphone arrangement to determine a combination of a first set of one or more microphones of the microphone arrangement and a first physical rotation angle of the one or more microphones of the microphone arrangement, the combination capturing the audio scene having an acceptable detected noise, wherein determining that the one or more microphones capture the audio scene having an acceptable detected noise is based on a cost function; and control the physical rotation angle of the one or more microphones of the microphone arrangement to the first physical rotation angle of the one or more microphones of the microphone arrangement, and capture the audio scene using the combination of the first set of one or more microphones of the microphone arrangement and the first physical rotation angle of the one or more microphones of the microphone arrangement.

15. A method, comprising: Determine that the captured audio scene has unacceptable detection noise, where the captured audio scene is captured using a microphone arrangement including a plurality of microphones; Depending on determining that the captured audio scene has unacceptable detection noise, search for a plurality of combinations of the plurality of microphones of the microphone arrangement and a plurality of physical rotation angles of the plurality of microphones of the microphone arrangement to determine a combination of a first set of one or more microphones of the microphone arrangement and a first physical rotation angle of the one or more microphones of the microphone arrangement, where the combination captures the audio scene having acceptable detection noise, and where determining that the one or more microphones capture the audio scene having acceptable detection noise is based on a cost function; And Control the physical rotation angles of the plurality of microphones of the microphone arrangement to the first physical rotation angle of the one or more microphones of the microphone arrangement, and capture the audio scene using the first set of the one or more microphones of the microphone arrangement and the combination of the first physical rotation angle of the one or more microphones of the microphone arrangement.

Images