Mobile device microphone position optimization

Abstract

An apparatus for capturing spatial audio, the apparatus comprising: at least four microphones; at least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets are located to define a three-dimensional four-sided capturing geometry for spatial audio, wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

Description

MOBILE DEVICE MICROPHONE POSITION OPTIMIZATIONFIELD

[0001] The present application relates to a mobile device with microphone position optimization and furthermore methods, apparatus, system and computer programs for designing the mobile device with microphone position optimization, and in particular but not exclusively for a mobile device with microphone position optimization suitable for spatial phone calls.BACKGROUND

[0002] 3GPP IVAS (Immersive Voice and Audio Services) is a new conversational audio codec standardized in Rel-18 and being enhanced in Rel-19. This is a versatile codec with many supported input formats and output formats. The main supported input formats for IVAS are stereo, multichannel (MC), objectbased audio (ISM), scene-based audio (SBA), and Metadata-assisted spatial audio (MASA). In addition, the following combinations are supported: Objects with MASA (OMASA) and Objects with SBA (OSBA). IVAS furthermore includes the EVS codec for mono input operation. The IVAS output formats include mono, stereo, multi-channel (including custom loudspeaker layouts), FOA, HOA2, HOA3, and binaural. In addition, so-called pass-through operation is possible allowing, e.g., MASA output for MASA input. As a spatial audio codec supporting at least three degrees of rotation freedom (yaw, pitch, roll) for all spatial inputs, the IVAS codec is expected to be used in a variety of scenarios, all of which cannot be known beforehand.

[0003] IVAS codec algorithm is described in 3GPP TS 26.253 (Codec for Immersive Voice and Audio Services; Detailed Algorithmic Description incl. RTP payload format and SDP parameter definitions) with IVAS codec floating-point C code is provided in 3GPP TS 26.258. IVAS codec fixed-point C code is currently being standardized in Rel-19.

[0004] 3GPP DaCAS (Diverse audio CApturing System for UEs) is a current 3GPP SA4 work item that aims to define immersive audio capture example solutions for converting raw / compensated microphone signals into the relevant IVAS codec input formats listed above. This will be based on definition of a set of target devices or target device sets with descriptions of the overall microphone configurations. For example, number of microphones, their relative positions in the device integration, etc. should be defined.

[0005] Relevant IVAS and DaCAS audio use cases relate, e.g., to immersive voice calls (including immersive experience sharing) and immersive audio conferencing. For example, use cases can comprise an immersive audio capture capable smartphone in portrait (including handset mode capture) and landscapeorientation audio capture, switching between capture orientations, and an ad-hoc telco capture with device placed on table surrounded by talkers (telco participants in same room).

[0006] The microphone positions in a mobile device should be designed such that good spatial capture performance can be achieved in all targeted use cases. For example, the orientation of the device may vary, user may keep the device in landscape or portrait orientation in his / her hands, or the device may be located, e.g., on table. The microphone setup should support spatial capture such that the direction of sounds originating from around the device can be identified, captured and encoded in correct orientation.SUMMARY

[0007] According to a first aspect there is provided an apparatus for capturing spatial audio, the apparatus comprising: at least four microphones; at least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets are located to define a three-dimensional four-sided capturing geometry for spatial audio, wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

[0008] The three-dimensional four-sided capture geometry may be a skew quadrilateral.

[0009] The three-dimensional four-sided capture geometry may maximize an aperture of a microphone array comprising the at least four microphones.

[0010] The three-dimensional four-sided capture geometry may define an aperture of a microphone array comprising the at least four microphones which exceeds a threshold value.

[0011] The aperture of the microphone array comprising the at least four microphones may be defined by: defining a first vector with respect to a first pair of the at least four microphones, the first pair of the at least four microphones being located towards one end of the apparatus; defining a second vector with respect to a second pair of the at least four microphones, the second pair of the at least four microphones being located towards an opposite end of the apparatus; and defining a cross-product based on one of: the first vector and the second vector; and a projection of the first vector to a first plane and a projection of the second vector to the first plane.

[0012] The apparatus may comprise a structure within which the at least four microphone inlets are located, the structure may comprise: a first surface, the first surface comprising a display; a second surface, the second surface located opposite the first surface; a third surface, the third surface connecting the first surface and second surface, and comprising a charging port; a fourth surface, the fourth surface connecting the first surface and the second surface and located opposite the third surface; a fifth surface, the fifth surface connecting the first surface, the second surface, the third surface and the fourth surface; and a sixth surface, the sixth surface connecting the first surface, the second surface, the third surface and the fourth surface and located opposite the fifth surface.

[0013] At least one of the surfaces may comprise a beveling or curving towards at least one of the other surfaces.

[0014] The second surface may comprise a camera module protrusion projecting from the second surface.

[0015] A first of the at least four microphone inlets and a second of the at least four microphone inlets may be located on the second surface, wherein the first microphone inlet may be located substantially diagonally opposite the second microphone inlet.

[0016] A first of the at least four microphone inlets may be located at a first location on the second surface and a second of the at least four microphone inlets may be located at a second location on the second surface, wherein the distance between the first and second locations is substantially maximized with respect to a defined apparatus dimension.

[0017] One of the first microphone inlet or the second microphone inlet may be located on the camera module protrusion.

[0018] A third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0019] A third of the at least four microphone inlets may be located at a third location on the third surface, wherein the distance between the third location and one of the first and second locations may be substantially maximized with respect to a defined apparatus dimension.

[0020] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0021] A fourth of the at least four microphone inlets may be located at a fourth location on the fourth surface, wherein the distance between the fourth location and one of: the first; second or third locations may be substantially maximized.

[0022] A first of the at least four microphone inlets may be located on the second surface.

[0023] A second and a third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and the third microphone inlet may be located substantially diagonally opposite the second microphone inlet on the third surface.

[0024] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0025] A fourth of the at least four microphone inlets may be located on the fourth surface, and located distant from the first microphone inlet and the second microphone inlet.

[0026] Pairs of the at least four microphone inlets may be aligned with respect to axes defined by the surfaces.

[0027] The first microphone inlet may be one of: located on a camera module protrusion; and located on or substantially close to a centre line of the second surface.

[0028] A fourth of the at least four microphone inlets may be located on the first surface, and located at an earpiece opening.

[0029] Afirst and a second of the at least four microphone inlets may be located on the third surface, wherein the first microphone inlet is located substantially opposite the second microphone inlet on the third surface.

[0030] An orientation of the first microphone inlet may differ from the orientation of the second microphone inlet.

[0031] A third of the at least four microphone inlets may be located on the second surface.

[0032] The third microphone inlet may be located on the camera module protrusion.

[0033] A fourth, fifth and sixth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth, fifth and sixth microphone inlets may be distributed along the fourth surface.

[0034] An orientation of the fourth microphone inlet may differ from the orientation of the fifth and sixth microphone inlets.

[0035] According to a second aspect, there is provided means for designing an apparatus for capturing spatial audio, the apparatus comprising: at least four microphones; at least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets is located to define a three-dimensional four-sided capturing geometry for spatial audio, wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

[0036] The three-dimensional four-sided capture geometry may be a skew quadrilateral.

[0037] The three-dimensional four-sided capture geometry may maximize an aperture of a microphone array comprising the at least four microphones.

[0038] The three-dimensional four-sided capture geometry may define an aperture of a microphone array comprising the at least four microphones which exceeds a threshold value.

[0039] The aperture of the microphone array comprising the at least four microphones may be defined by: defining a first vector with respect to a first pair of the at least four microphones, the first pair of the at least four microphones being located towards one end of the apparatus; defining a second vector with respect to a second pair of the at least four microphones, the second pair of the at least four microphones being located towards an opposite end of the apparatus; and defining a cross-product based on one of: the first vector and the second vector; and a projection of the first vector to a first plane and a projection of the second vector to the first plane.

[0040] The apparatus may comprise a structure within which the at least four microphone inlets are located, the structure may comprise: a first surface, the first surface comprising a display; a second surface, the second surface located opposite the first surface; a third surface, the third surface connecting the first surface and second surface, and comprising a charging port; a fourth surface, the fourth surface connecting the first surface and the second surface and located opposite the third surface; a fifth surface, the fifth surface connecting the first surface, the second surface, the third surface and the fourth surface; and a sixth surface, the sixth surface connecting the first surface, the second surface, the third surface and the fourth surface and located opposite the fifth surface.

[0041] At least one of the surfaces may comprise a beveling or curving towards at least one of the other surfaces.

[0042] The second surface may comprise a camera module protrusion projecting from the second surface.

[0043] A first of the at least four microphone inlets and a second of the at least four microphone inlets may be located on the second surface, wherein the first microphone inlet may be located substantially diagonally opposite the second microphone inlet.

[0044] A first of the at least four microphone inlets may be located at a first location on the second surface and a second of the at least four microphone inlets may be located at a second location on the second surface, wherein the distance between the first and second locations is substantially maximized with respect to a defined apparatus dimension.

[0045] One of the first microphone inlet or the second microphone inlet may be located on the camera module protrusion.

[0046] A third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0047] A third of the at least four microphone inlets may be located at a third location on the third surface, wherein the distance between the third location and one of the first and second locations may be substantially maximized with respect to a defined apparatus dimension.

[0048] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0049] A fourth of the at least four microphone inlets may be located at a fourth location on the fourth surface, wherein the distance between the fourth location and one of: the first; second or third locations may be substantially maximized.

[0050] A first of the at least four microphone inlets may be located on the second surface.

[0051] A second and a third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and the third microphone inlet may be located substantially diagonally opposite the second microphone inlet on the third surface.

[0052] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0053] A fourth of the at least four microphone inlets may be located on the fourth surface, and located distant from the first microphone inlet and the second microphone inlet.

[0054] Pairs of the at least four microphone inlets may be aligned with respect to axes defined by the surfaces.

[0055] The first microphone inlet may be one of: located on a camera module protrusion; and located on or substantially close to a centre line of the second surface.

[0056] A fourth of the at least four microphone inlets may be located on the first surface, and located at an earpiece opening.

[0057] Afirst and a second of the at least four microphone inlets may be located on the third surface, wherein the first microphone inlet is located substantially opposite the second microphone inlet on the third surface.

[0058] An orientation of the first microphone inlet may differ from the orientation of the second microphone inlet.

[0059] A third of the at least four microphone inlets may be located on the second surface.

[0060] The third microphone inlet may be located on the camera module protrusion.

[0061] A fourth, fifth and sixth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth, fifth and sixth microphone inlets may be distributed along the fourth surface.

[0062] An orientation of the fourth microphone inlet may differ from the orientation of the fifth and sixth microphone inlets.

[0063] According to a third aspect, there is provided a method for providing an apparatus for capturing spatial audio, the apparatus comprising: at least four microphones; at least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets is located to define a three-dimensional four-sided capturing geometry for spatial audio, wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

[0064] The three-dimensional four-sided capture geometry may be a skew quadrilateral.

[0065] The three-dimensional four-sided capture geometry may maximize an aperture of a microphone array comprising the at least four microphones.

[0066] The three-dimensional four-sided capture geometry may define an aperture of a microphone array comprising the at least four microphones which exceeds a threshold value.

[0067] The aperture of the microphone array comprising the at least four microphones may be defined by: defining a first vector with respect to a first pair of the at least four microphones, the first pair of the at least four microphones being located towards one end of the apparatus; defining a second vector with respect to a second pair of the at least four microphones, the second pair of the at least four microphones being located towards an opposite end of the apparatus; and defining a cross-product based on one of: the first vector and the second vector; and a projection of the first vector to a first plane and a projection of the second vector to the first plane.

[0068] The apparatus may comprise a structure within which the at least four microphone inlets are located, the structure may comprise: a first surface, the first surface comprising a display; a second surface, the second surface located opposite the first surface; a third surface, the third surface connecting the first surface and second surface, and comprising a charging port; a fourth surface, the fourth surface connecting the first surface and the second surface and located opposite the third surface; a fifth surface, the fifth surface connecting the first surface, the second surface, the third surface and the fourth surface; and a sixth surface, the sixth surface connecting the first surface, the second surface, the third surface and the fourth surface and located opposite the fifth surface.

[0069] At least one of the surfaces may comprise a beveling or curving towards at least one of the other surfaces.

[0070] The second surface may comprise a camera module protrusion projecting from the second surface.

[0071] A first of the at least four microphone inlets and a second of the at least four microphone inlets may be located on the second surface, wherein the first microphone inlet may be located substantially diagonally opposite the second microphone inlet.

[0072] A first of the at least four microphone inlets may be located at a first location on the second surface and a second of the at least four microphone inlets may be located at a second location on the second surface, wherein the distance between the first and second locations is substantially maximized with respect to a defined apparatus dimension.

[0073] One of the first microphone inlet or the second microphone inlet may be located on the camera module protrusion.

[0074] A third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0075] A third of the at least four microphone inlets may be located at a third location on the third surface, wherein the distance between the third location and one of the first and second locations may be substantially maximized with respect to a defined apparatus dimension.

[0076] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0077] A fourth of the at least four microphone inlets may be located at a fourth location on the fourth surface, wherein the distance between the fourth location and one of: the first; second or third locations may be substantially maximized.

[0078] A first of the at least four microphone inlets may be located on the second surface.

[0079] A second and a third of the at least four microphone inlets may be located on the third surface, wherein the third microphone inlet may be orientated towards the first surface and the third microphone inlet may be located substantially diagonally opposite the second microphone inlet on the third surface.

[0080] A fourth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth microphone inlet may be orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

[0081] A fourth of the at least four microphone inlets may be located on the fourth surface, and located distant from the first microphone inlet and the second microphone inlet.

[0082] Pairs of the at least four microphone inlets may be aligned with respect to axes defined by the surfaces.

[0083] The first microphone inlet may be one of: located on a camera module protrusion; and located on or substantially close to a centre line of the second surface.

[0084] A fourth of the at least four microphone inlets may be located on the first surface, and located at an earpiece opening.

[0085] Afirst and a second of the at least four microphone inlets may be located on the third surface, wherein the first microphone inlet is located substantially opposite the second microphone inlet on the third surface.

[0086] An orientation of the first microphone inlet may differ from the orientation of the second microphone inlet.

[0087] A third of the at least four microphone inlets may be located on the second surface.

[0088] The third microphone inlet may be located on the camera module protrusion.

[0089] A fourth, fifth and sixth of the at least four microphone inlets may be located on the fourth surface, wherein the fourth, fifth and sixth microphone inlets may be distributed along the fourth surface.

[0090] An orientation of the fourth microphone inlet may differ from the orientation of the fifth and sixth microphone inlets.

[0091] In the above, many different embodiments have been described. It should be appreciated that further embodiments may be provided by the combination of any two or more of the embodiments described above. DESCRIPTION OF FIGURES

[0092] Embodiments will now be described, by way of example only, with reference to the accompanying Figures in which:

[0093] Fig.1 a shows schematically example immersive audio capture scenarios relevant for IVAS / DaCAS;

[0094] Fig.1 b shows a generic spatial audio processing front-end;

[0095] Fig.1c shows typical parts of a device microphone being integrated and sealed against an outer shell of the device body;

[0096] Fig .1 d shows an example signal path from sound source s(w) in space to microphone array output;

[0097] Fig.1 e shows a block diagram of the example signal path shown in Fig.ld;

[0098] Figs.2a to 2e show example apparatus with example design constraints for microphone inlet positions according to some embodiments

[0099] Fig.3 shows example tested sound source directions which can be analyzed independent of orientation of the apparatus for designing

[0100] Figs. 4 to 9 show example apparatus and example microphone inlet positions within the example design constraints according to some embodiments;

[0101] Fig.10 shows a generalised skew quadrilateral microphone inlet position geometry according to some embodiments;

[0102] Fig.11 shows an example graph of audio focusing performance for selected target directions for various example microphone inlet positions as shown in Figs.4 to 9;

[0103] Fig.12 shows an example graph of average audio focusing performance for the selected target directions for various example microphone inlet positions as shown in Figs.4 to 9;

[0104] Fig.13 shows example graphs of directivity (polar pattern) at 984 Hz towards a front direction for various example microphone inlet positions as shown in Figs.4 to 9;

[0105] Fig.14 shows example graphs of directivity (polar pattern) for selected frequencies towards a front direction for various example microphone inlet positions as shown in Figs.4 to 9;

[0106] Fig.15 shows example graphs of focused audio capture performance (directivity index) towards various focus directions for the example microphone inlet positions as shown in Figs.4 to 9;

[0107] Fig.16 shows an example graph of relative spatial analysis performance for the example microphone inlet positions as shown in Figs.5 to 9 compared to the positions shown in Fig.4;

[0108] Fig.17 shows example graphs of direction analysis error for DOA angle spatial analysis accuracy for the example microphone inlet positions as shown in Figs.4 to 9;

[0109] Fig.18 shows example graphs of Directivity variability in standard deviation (STD) in different directions for the example microphone inlet positions as shown in Figs.4 to 9; and

[0110] Fig.19 shows example apparatus and example microphone inlet positions within the example design constraints for a six-microphone placement according to some embodiments.

[0111] Fig.20 shows simulation results of 14993 microphone inlet placement configurations where the value of the optimization cost function defined the x-axis and y-axis is the normalized volume of the skew quadrilateral prism. Normalization is implemented by dividing the volume of the prism with volume of the device body (L'W'D). The 30 arrays with smallest optimization cost are highlighted with “o” circle symbols;

[0112] Fig. 21 shows the microphone inlet positions corresponding to the subset of 30 simulated microphone inlet configurations with lowest optimization cost as an example on device surface. It can be observed that microphone inlets at corner areas are concentrated on smaller surface areas compared to more flat surface areas.;

[0113] Fig.22 shows simulation results of 14993 array configurations where an effective aperture is estimated using sum of microphone port vector lengths as a sum cross products vectors lengths |AB x CD | + |AC x BD| + |AD X BC|; and

[0114] Fig.23 shows simulation results of 14993 array configurations where effective aperture is estimated using sum of microphone port areas as a sum cross products vectors lengths |PXZ(AB) X PXZ(CD)| .DETAILED DESCRIPTION

[0115] The concept, as discussed in further detail in the embodiments herein, is related to apparatus (or also devices, or user equipment, such as a smartphone or mobile phone) and methods to provide immersive audio capture for multi-mode immersive communication audio coding, transmission, and rendering on a multimicrophone consumer audio capture device.

[0116] Fig.1a shows example uses of an immersive audio capture capable smartphone. For example, on the left 101 the apparatus or device is shown in portrait (including handset mode capture) and landscape orientation audio capture, switching between capture orientations, and on the right 103 an ad-hoc telco capture with the apparatus or device placed on table surrounded by talkers (telco participants in same room).

[0117] Fig.1b shows a generic spatial audio processing front-end. The audio source 121 acoustic waves 122 are received by the microphones 123 which generate multi-microphone input signals 124. The figure also shows spatial audio capture algorithms implemented by a spatial audio processor 125 are configured to the process multi-microphone input signals 124 to generate a desired audio output 132 response and in parallel the microphone signals 124 are analyzed in a spatial analyzer 127 to estimate spatial audioparameters, such as direction of arrival (DOA) and direct-to-total energy ratio, which can be used to create Metadata-assisted spatial audio (MASA) metadata m(n) 126 and control metadata 128. The control metadata 128 can then be passed to a controller 129 to generate control signals 130 to control the spatial audio processor 125 for supported input formats for IVAS y(n) with optional source separation or audio focusing towards estimated azimuth e and elevation p. angles. It is clear that spatial audio processing performance and spatial analysis (DOA) accuracy are dependent on device electroacoustics and in particularly the shape of the device and position of integrated microphone inlets, which restrict the spatial audio sampling capabilities of the device.

[0118] The aim of the embodiments described herein is an apparatus comprising microphone (or more specifically microphone sound inlet) positions and / or orientations (for example for a mobile device such as a smartphone) which is designed or configured such that good spatial capture performance can be achieved in all targeted use cases.

[0119] Fig.1c shows typical parts of device microphone integrated and sealed against the outer shell of the device body. Integrated microphone assembly includes a sound port 149 passing acoustic pressure from a microphone sound inlet 141 to a microphone transducer diaphragm within a MEMS microphone capsule 145. A microphone sound port 149 can pass through multiple material layers supporting the microphone integration, which can lead to the microphone sound port 149 having cross-section variations and changes in acoustic impedance of different layers along the length of the sound port from device surface to the (MEMS) microphone transducer diaphragm. For example as shown in Fig.1c the sound port can pass through a printed circuit board (PCB) 153 on which the bottom port MEMS microphone capsule 145 is mounted. As shown in Fig.1c the MEMS microphone capsule 145 can be mounted on the ‘underside’ of the PCB 153. The sound port 149 furthermore can be protected by a dust filter 151 mounted on the ‘overside’ of the PCB 153 and furthermore sealed with respect to the outer shell 143 of the device via a suitable gasket 155. The sound port 149 is acoustically coupled to the exterior of the device via a sound inlet 141 (which is a suitable hole of acoustical port in the outer shell 143). The MEMS microphone capsule 145 is configured to generate a suitable microphone output signal 147. Additionally is shown in Fig.1c is a reference point 157 which is ‘above’ and central to the sound inlet 141. The reference point 157 is a point very close to sound inlet 141, where sound pressure at sound inlet 141 can be reliably measured using for example a probe microphone.

[0120] Acoustic sound port resonances are common in commercial device integrated microphone characteristics, which can limit the spatial audio capture and spatial analysis (DOA) accuracy especially around resonance frequencies.

[0121] Hereafter, the sound inlet 141 can be referred simply as inlet and sound port 149 can be referred as a port for simplicity. Fig.ld shows an example signal path model from a sound or audio source s(w) 121 in space to a spatial output audio signal 187. In this example model there is a transfer function for the ithintegrated microphone e. cp^l] which defines the acoustic signal path from the audio or sound source 121 point to sound inlet reference point 157 on the device surface.

[0122] Furthermore there is shown in Fig.ld an integrated microphone frequency response181 which defines a signal path from sound inlet reference point 157 to microphone transducer output xt(n) 183, which can be an analog or digital signal (though in the examples herein is a digital signal generated via digital microphones). Each microphone signal xt(n) 183 is then processed using spatial algorithms Hg ppz) 185 in response to sound source angles [0, <p] to produce a spatial output signal y(n) 187.

[0123] Fig .1 e furthermore shows the example signal path model as represented by a block diagram showing a total end-to-end signal path for the audio or sound source s(w) 121 to the spatial audio algorithm output 187. The spatial processing algorithm H0^(z) 185 processes the microphone input signals x£(n), where i = [1, for N microphones to produce the desired spatial audio output signal y(n) 187. In this example the integrated microphone signal path blocks are shown in Fig.1e only for ithintegrated microphone. A good spatial capture performance should be possible in the examples described hereafter even where the orientation of the device varies, a user changes or keeps the device in landscape or portrait orientation in his / her hands, or the device location varies or is maintained, e.g., left on a table or moved from table to table.

[0124] The microphone setup or configuration should therefore be able to support spatial capture such that the direction of sounds originating from around the device can be identified, captured and encoded in correct orientation.

[0125] In some embodiments, the setup or configuration of microphones should furthermore be able to implement a range of spatial processing analysis levels. For example, in some embodiments the configuration would be enough that directions are identified in a limited range or orientations e.g. in horizontal level only, whereas in some other cases a full 3D analysis is implemented.

[0126] The introduction of IVAS standard furthermore raises new requirements for mobile devices (UEs). The smartphone form factor is particularly challenging for spatial audio capture because the mobile device is typically a slim device, with at least one axis which is short relative to the wavelength of sound. Integration of multiple microphones in the smartphone highly influenced by acoustic design constraints of <10mm device thickness.

[0127] The embodiments as described herein show examples which attempt to solve the design problem of designing microphone inlet positions for spatial audio recording that enable synchronized capture of acoustic waveform so that audio processing algorithms can solve two parallel problems:a) detect the direction of arrival of the captured sound source that can be mapped to IVAS MASA parameters (or similar spatial representations) andb) enable spatial audio processing supported input formats for IVAS encoder and complementary audio processing with spatially selective audio capture, a.k.a. audio focusing or sound source separation.

[0128] When an array of microphones is integrated into acoustically non-transparent product mechanics the product shape becomes an active part of the acoustic processing chain. The signal path from the acoustic sound source to the input of audio algorithms include acoustic waveform radiation in air, which is a locally homogeneous medium, to device surface. The radiated waveform can be described by the Helmholtz equation and divided into incident wave and scattered wave, where the incident wave represents free field radiation component and scattered wave describes the waveform perturbation caused by physical body of the device. Therefore, each microphone inlet on the device surface receives a non-isotropic acoustic signal with spatial response defined by frequency-wavenumber response F^ , 0, cp) such as disclosed in Trees, H. V. (2004). Optimum Array Processing (1st ed.). Using the array processing terminology, array of microphone sound inlets defines continuous-time array aperture where microphone array sound inlets define the spatial sampling of the acoustic waveform. Thus, the spatial extent of microphone inlet is referred as an aperture of the microphone array. When the acoustic pressure signals are further processed by the integrated microphone frequency responses Hm.(a>), we have a definition for spatial signals received by the microphone array. In the array processing literature, a mathematical representation of the spatial response of the array at a specific direction of arrival (DOA) is called as array manifold vector.

[0129] In the case of integrated microphone arrays the definition of this property requires application of multiple techniques. Spatial frequency responses for microphone signals can be measured in anechoic laboratory environments simply by playing sound from different directions [9, <p] or by rotating the device between measurements. This is feasible for the characterization of the existing microphone array, but if the design question is to define the microphone array and find positions for microphone inlets, physical prototyping can become infeasible and slow. Instead, it is possible to measure an integrated microphone response Hm.(a>) using a probe microphone using IMPro method, as disclosed in J. Cozens , M. Hamalainen , and M. Pekkarinen, IMPro - Method for Integrated Microphone Pressure Frequency Response Measurement Using a Probe Microphone, (to be published in) AES 158th Convention 2025, May 22-24, from an earlier product with comparable microphone integration or a relevant microphone prototype. Then it is possible to apply numerical acoustic methods to simulate acoustic scattering of the product mechanics from sound source to pressure on device surface at microphone inlet positions and this way get a numerical solution for 0, <p).

[0130] Reliable integration of microphone transducers to a commercial hand-held device has several requirements, related to industrial design and protection of the transducer against dirt, water and other environmental conditions and therefore microphone transducers are not integrated on the surface of the device.

[0131] Referring back to Fig.1c is shown a simplified model of integrated microphone, where a short tube (or acoustic gravity) or sound port is passing acoustic pressure from the sound inlet on device surface to themicrophone transducer diaphragm. This sound port for integrated microphone can include acoustic filters or acoustic membranes (such as the dust filter 151) for environmental protection that have acoustic impedance. Acoustic port geometry and used acoustic materials define the frequency response for the acoustic port. In many cases, this frequency response of the port can be commonly estimated as linear one-dimensional frequency response model as described above.

[0132] Focused audio capture can be implemented using a range of different techniques such as time invariant Filter and Sum Beamforming (FSB), Adaptive MVDR beamforming as well as advanced application of Deep Neural Networks (DNNs). All these focused capturing technologies are constrained by a spatial resolution of the microphone array and since design time optimization of spatial audio capture resolution can be considered as signal independent property of focused audio capture, the acoustic problem can be addressed by considering time invariant beamforming methods.

[0133] Any improvements for microphone sound inlet placement improving spatial audio resolution can also benefit other spatial audio processing algorithms. As such these example microphone inlet placements can be employed to improve the performance of other spatial audio processing applications.

[0134] A microphone inlet position optimization tool can in some embodiments be employed to attempt to solve these two design problems in parallel because a suitable device will be configured with only one set of microphones and microphone inlets to support all audio algorithms.

[0135] Unfavorable microphone inlet positions can therefore compromise the performance of all spatial audio algorithms in the device and therefore it is very important to design microphone inlet positions carefully for target uses of the device. This is particularly important for devices with small number of microphones (N<4) since there is no redundancy in microphone inlet placement to compensate suboptimal spatial sampling of acoustic waveform algorithmically, which would be possible with large microphone arrays.

[0136] With the DaCAS work item, 3GPP is for the first time in mobile communications history planning to specify audio capture methods, specifically for immersive capture targeting the IVAS codec, IVAS encoder input formats, and immersive services and use cases.

[0137] The main problem to be solved is to achieve the above constraints in a way that allows for specified device example designs and capture solutions (specified in TS 26.533) to provide guidance and conformant implementations for manufacturers and operators to for example launch large-scale IVAS services.

[0138] As such the embodiments described herein describe a method that is proposed for specifying microphone acoustic inlet location geometry on a capture device outer shell for immersive audio capture. This results in the design of a suitable device or apparatus which comprises microphone acoustic inlet location geometry on a capture device outer shell for immersive audio capture.

[0139] The following example configurations show a 4-microphone capture device configuration. However, in some embodiments a microphone capture device with more than 4 microphones can be equally designedbased on the following placement designs. The design of less than 4 microphone device can also be done with similar principles, but target responses should be aligned with the limited spatial capabilities of the device.

[0140] In the following examples the microphone or microphone inlet designs are such that there are placed (or located or positioned / oriented) two pairs of microphone inlets in substantially opposite ends of the physical device. In some embodiments the placement of one of the microphone inlets is set in a camera module protrusion or camera bump (or camera bulge) to utilize the maximum thickness of the device.

[0141] In some embodiments the placement of the microphone inlets is such that lines between the two microphone pairs on both ends of the device are non-parallel and the angle between the lines is substantially or practically maximized while attempting to maximize the size of the array aperture in all 3D directions.

[0142] Thus, in some embodiments the placement of the microphone inlets is such that there is a placement of two microphones close to the ‘charging port’ end corners of the device and a four-microphone geometry that can be described as skew quadrilateral geometry where there is a minimum distance between diagonal lines between microphone pairs located close to opposite corners which is maximized.

[0143] In such examples there is an attempt to optimize immersive audio 3D capture and audio focus performance in both landscape and portrait orientations and immersive audio planar capture performance for tabletop placement.

[0144] The microphone inlet placement method described above is shown with respect to the example device and microphone inlet positions shown in the Figs and described herein as a practical DaCAS audio capture target device configuration provided for generation of IVAS encoder input formats, including Metadata-assisted spatial audio (MASA).

[0145] The above design additionally follows and fulfills the following design approach criteria:(a) Defining a ‘typical’ smartphone shape and dimensions with multiple microphone locations; (b) Setting up a simulation for modeling receiving sounds from surrounding directions to the various locations on the surface of a mobile device being simulated;(c) Defining a cost function which is able to, based on the simulation, measure performance of the selected spatial audio capture method for the defined microphone setup over selected target source directions;(d) Setting design constraints for the positions of possible microphone inlet locations such that locations not possible due to device’s mechanical structure or locations that are not practical considering device use cases are excluded from the search of optimal microphone inlet locations; and(e) Using optimization algorithm for finding optimal microphone inlet locations from the allowed device surface area for given number of microphones using defined cost function.

[0146] 3GPP TR 26.933 (Study on diverse audio capturing system for end-user devices) includes statistics about smartphone size evolution over several years and in these examples the device chosen has an average device size for the most recent year starting from year 2020 to 2024.

[0147] However, it would be appreciated that some embodiments can be employed in other device sizes as the method described herein is relatively insensitive for device size. For example, typically (smart)phone device dimensions are limited by usability constraints so that the device can be carried and put into a pocket conveniently, which as discussed above results in at least one of device dimensions being small compared to the wavelength of the acoustic waves (audio bandwidth) of the sounds especially in the direction of device thickness.

[0148] For example, these embodiments describe example traditional smartphone designs that have a large display on one side or face or surface and a main camera on the other side (or face or surface) of the device. In the following description the terms side, face and surface are interchangeable. The proposed methods and configurations as described herein by the following embodiments are also applicable for alternative form factors such as folded devices. In the case of folded devices, relevant use cases can include variations in the physical shape of the device and in some cases multiple displays which leads to multiple acoustic configurations and different design constraints.

[0149] Thus, for example, in the case of folded devices a similar design could also be applied, but with knowledge or consideration that different mechanical configurations (for example display folded vs. display open) can produce different behaviors for example as the physical device configuration causes occlusions which affect acoustic waveform detection or determination. Thus, the occlusions affect the transmission or propagation of acoustic pressure around the device.

[0150] Practical smartphone design has many practical constraints that can vary from device to device. However, there are several common limitations in typical smartphone devices which the following examples take into consideration. For example, the following embodiment examples employ a device with defined surface areas where microphone inlets can be placed and other surface areas where microphone sound inlets are not permitted either for technical reasons or usability reasons.

[0151] For example, technical limitations for placement of microphone inlets described herein can include a display module (on one or more face) and device charging port (positioned typically on the bottom edge or side). In some examples the camera lenses and associated mechanics are defined as forbidden regions for the placement of the microphone inlets however the following examples employ a simplified device rather than commercial device, as there may be different numbers and configurations of cameras and other optical sensors.

[0152] An example device 200 which is used as a base designs for the embodiments described herein is shown in Fig.2a to 2e.

[0153] The device 200 shown in Fig.2a to 2c is shown with the mesh or lighter shaded portions 212 where microphone inlets can be placed and darker shaded portions 210 where microphone inlets cannot be placed.

[0154] The device 200 shown in Fig.2d is shown with the mesh or lighter shaded portions 212 where microphone inlets can be placed and refined darker shaded portions 210 additionally restricts microphone placement on lower portion of the device back cover. This additional limitation is justified by unpredictable user hand grip when holding the device especially in portrait orientation. User hand can shadow and blocking the microphone inlet and therefore significantly change the audio waveform received by the microphone covered by user’s hand. User grip is difficult to predict in practice and therefore an additional design constraint has been introduced.

[0155] The device 200 is shown with a first surface, side or face 201 , which is oriented for these examples in an z-y axis, with a first dimension (length, or longest dimension) in the y-axis and second dimension (width) in the z-axis. In the examples shown herein the first side or face comprises the display module which covers a significant portion of the face and the placement of microphone sound inlets over the display module is not allowed or forbidden. In some embodiments the first side or face comprises an acoustic outlet for earpiece transducer, slot or portion approximately at a ‘top’ area and which is suitable for microphone inlet placement.

[0156] The device 200 is further shown with a second surface, side or face 203, which is also oriented on the z-y axis and opposite the first side or face 201 with an offset in the x-axis, the offset defining a third dimension (thickness or shortest dimension). The second side or face 203 can comprise the camera module and lens(es). In some embodiments the camera module or lens(es) are at least partially contained in a camera module protrusion (bulge or bump) which is further offset on the x-axis from the rest of the second side. The size, location and shape of the camera module protrusion (camera bump or bulge) 202 can differ from design to design, but in the following examples is an approximate cuboid located at the ‘top-right’ portion of the second side (relative to the point of view of the main camera module when the device is in landscape mode).

[0157] The device 200 is further shown with a third surface, side or face 205, which is oriented for these examples in an z-x axis, between the first side and second side and is located at the ‘left’ of the device (when in landscape mode facing the camera module). The third side 205 can comprise the charging port 204 which in this example covers a central portion of the face and the placement of microphone sound inlets or ports through the display module is not allowed or forbidden.

[0158] Additionally, the device 200 is shown with a fourth surface, side or face 207, which is oriented for these examples in z-x axis between the first side and second side and is located at the ‘right’ of the device and opposite the third side or face 205. The fourth side or face 207 is offset on the y-axis from the third side or face. The fourth side or face 207 is generally available for microphone outlet placement.

[0159] Furthermore, the device 200 is further shown with a fifth surface, side or face 209, which is oriented for these examples in an y-x axis, between the first side and second side and is located at the ‘top’ of the device (when in landscape mode facing the camera module). The fifth side 209 can comprise the antenna over which the placement of microphone inlets or ports is not allowed or forbidden.

[0160] The device 200 is shown with a sixth surface, side or face 211, which is oriented for these examples in y-x axis between the first side and second side and is located at the ‘bottom’ of the device and opposite the fifth side or face. The sixth side or face 211 is offset on the z-axis from the fifth side or face. Similar to the fifth side or face 209 the sixth side or face 211 can comprise the antenna and is generally forbidden for microphone outlet placement.

[0161] The following examples show well defined surfaces (sides or faces) it would be understood that in some embodiments at least two of the surfaces can be joined by a blending surface, for example a beveling or smoothing or curvature. Furthermore, in some embodiments the following examples define a placement of the microphone (inlet) on a surface or side where the placement of the microphone (inlet) is generally orientated to a tangent of the surface. For example, this can mean that the rim of the inlet or port opening on the device surface is orientated according to a tangent of the surface.

[0162] As discussed above 3GPP IVAS use cases are focused on spatial audio capture and playback capabilities. However, the mobile devices will not have IVAS specific electroacoustic components, and the same set of components support all smartphone use cases including legacy telephone services, media capture and playback devices.

[0163] This examples as shown herein focusing on audio capture technologies and loudspeaker technologies and microphone inlet placement to aim for quality capture and playback, however such microphone inlet placement can be integrated in some locations with loudspeaker outlets to reuse the mechanical openings, holes, etc. in the outer shell of the device.

[0164] In some embodiments the microphone placement is as described in the following examples for a 4-microphone configuration.

[0165] In the following examples the microphone (or microphone inlet) placement can be seen to define a three-dimensional four-sided capturing geometry for spatial audio. A size of the three-dimensional four-sided capture geometry can furthermore be provided according to a geometry of the apparatus.

[0166] Furthermore, the microphone (inlet) placement results in a polygon where the sides of the polygon are not coplanar. This results in a quadrilateral which does not lie on a single flat plane (with respect to the apparatus dimension axes) and the polygon vertices do not all exist within the same 2D plane. In other words, the placement defines a non-planar quadrilateral or a skew quadrilateral geometry. In some embodiments, the sides and angles can vary, and the quadrilateral does not have parallel sides or any symmetry. The polygon having non-coplanar sides means the four vertices of a skew quadrilateral do not lie on the sameplane. Non-coplanar can also mean the opposite sides do not intersect as they would in a typical quadrilateral.

[0167] With respect to the three-dimensional four-sided capture geometry provided according to a geometry of the apparatus placement it would be understood that an example (target) device utilizes a smartphone form factor with dimension of 8.6 mm, 160.6 mm, and 75.4 mm, and a four-microphone array designed to maximize array aperture by defining a three-dimensional four-sided capturing geometry, and optionally with skew quadrilateral shape maximizing the microphone array aperture.

[0168] A size of the acoustic aperture can be estimated by geometrical dimensions of the skew quadrilateral geometry. The volume ofthe skew quadrilateral prism can be calculated by = - 6Ah, where A is the volume of the base and h is the height or with vectors V = |AD - (AB x AC)| . The size of this volume can be increased when one of the microphone inlets is placed on an elevated camera bump (or camera module protrusion) increasing the effective size of the aperture. However, free field geometry cannot fully describe the effective size of the discrete acoustic aperture defined by microphone inlets that are integrated into acoustically non-transparent product shell.

[0169] The aperture of an array of microphones integrated into product mechanics can, in some embodiments, be defined as a surface or volume, near or on the microphone array for the purpose of computing an effective capture area or vector. The aperture could for example be taken as that portion of a plane surface near the array through which an acoustic wave is captured.

[0170] In some embodiments the placement of the microphones or microphone inlets can be defined by two vectors of top-end (AB) microphone pairs and bottom-end (CD) microphone pairs (as shown in Fig.10 and described in further detail later) and generate a cross product between these two vectors.

[0171] The cross product can be used as a descriptor indicating how far these microphones are away from each other and how “skewed” the geometry is.

[0172] For example, the cross product (and the cross product where these vectors are first projected in xz-plane before cross product) can be employed to indicate a target performance.

[0173] For example, in some embodiments the microphone (inlet) placement can be determined based on matching a criteria or threshold value (threshold) based on this cross product. Thus, for example, in some embodiments the microphone (inlet) placement is such that:|PXZ( / 1B) x PXZ(CD)| / W > threshold

[0174] Where P(AB) is the projection in the xz-plane ofthe AB vector, P(CD) is the projection in xz-plane of the CD vector, W the width ofthe device and threshold the threshold value which the design is attempting to overcome. Threshold value corresponds to geometry criteria above according to which the microphone inlet placement is considered to be feasible and have sufficiently small cost for target use case. Threshold valuecan be mapped to y-axis value in Figs. 20 and 23. Threshold value can be applied to select the most feasible configuration(s) for practical consideration against non-acoustic design considerations such as industrial design or antenna or other product hardware considerations.

[0175] The following examples show placement embodiments based on the above rules. The rules and the design are based on simulation results combining audio focusing performance and sound source direction analysis. Both these audio DSP algorithms utilize synchronized sampling of microphone signals from the device, which enables spatial audio signal processing.

[0176] In some embodiments the microphone inlet placement is analyzed for typical smartphone use cases where capture algorithms need to support spatial analysis of captured sound sources as well as ability to focus on desired directions to emphasize captured signals.

[0177] Fig.2d furthermore shows a further example test device with respect to casing and shows the width 295, length 291 and thickness 271 of the device. Additionally is shown a camera module protrusion (also known as a camera bump or bulge) thickness 283, camera module protrusion width 251, camera module protrusion height 257 and camera module protrusion x-location 255. The figure additionally shows curvature radiuses such as the camera module protrusion corner radius 253, camera module face edge radius 263, display face edge radius 261 and corner radius 281.

[0178] For example, as shown in Fig.3, there is shown an example test device 200 and selected focus directions approximated spherical shape obtained using a subdivision of an icosahedron to approximately equal size triangles which provides in this example an approximately uniform representation of a sphere with 92 uniform directions 301.

[0179] With respect to focused audio capture, a typical capture scenario is recording of monophonic or stereophonic audio. This is considered in this example design by defining a number spatial directions where focused capture performance is evaluated as a part of design criteria. In this example results the applied criteria has been one of mono capture directed towards an approximated main camera direction, stereo capture both in landscape and portrait device orientation as well as mono capture directed towards a display direction for selfie camera use case. These can also be described using typical audio channel naming convention as center, left, right, portrait left, portrait right and center for back direction. For selfie capture the device is typically close to the user and performance requirements may differ from main camera use cases where the source distance from the device can be large. In some embodiments an additional limitation is that as described above the first side or display surface limits the placement of the microphones. It would be appreciated that this geometry is an example only and different geometries could be employed based on a target use case.

[0180] With respect to Fig.4 is shown a first example microphone or microphone inlet placement according to some embodiments. The microphone (inlet) placement (designated #40) is shown in Fig.4 with respect to microphone #1 401, microphone #2403, microphone #3405, and microphone #4407.

[0181] In this configuration a first pair of the microphones, microphone #1 401, microphone #2 403, are located at the ‘top’ end of the device and a second pair of microphones, microphone #3405, and microphone #4407 are located at the ‘bottom’ end of the device. Furthermore, the microphone #1 401, microphone #3 405 are located at the ‘left’ end of the device and microphone #2403, microphone #4407 are located at the ‘right’ end of the device.

[0182] In this example microphone #2403 and microphone #3405 are located on the second side or face of the device and are substantially at opposite diagonals of the face. In other words, approximately similar displacements from respective edges of the sides on the z-axis and y-axis. In this example microphone #2 403 is located on the camera module protrusion portion to provide a z-axis displacement relative to microphone #3405.

[0183] Furthermore, in this example microphone #1 401 is placed on the third side and microphone #4407 placed on the first side respectively. The microphone #1 401 and microphone #4 407 are substantially diagonally opposite with a similar z-axis displacement relative to the top and bottom side edges of the third and fourth sides.

[0184] In such a manner the geometry defined by the four microphones can be described as skew quadrilateral geometry where there is a (minimum) distance between diagonal lines between microphone pairs located close to opposite corners. This minimum distance is furthermore maximized.

[0185] For example the placement of microphone #1 401 and microphone #2 402 creates a first distance AA, the placement of microphone #1 401 and microphone #3 405 creates is a second distance BB, the placement of microphone #1 401 and microphone #4407 creates is a third distance CC, the placement of microphone #2 403 and microphone #3 405 creates is a fourth distance DD, the placement of microphone #2403 and microphone #4407 creates is a fifth distance EE, and the placement of microphone #3405 and microphone #4 407 creates is a sixth distance FF. In this example the configuration attempts to maximize the minimum version of this distances (and thus as shown in Fig.4 the placement of microphone #2403 and microphone #3405 is not located substantially close to the third and fourth sides to increase the third distance CC and sixth distance FF at the cost of reducing the other distances.

[0186] Fig.5 shows a further example configuration. This design satisfied design constraints shown in Fig.2d, which prevents microphone inlet placement 405 on Fig.4 to be used. Fig.5 microphone (inlet) placement (designated #41) retains the microphone placement for microphone #1 501, microphone #2503, and microphone #4 507, but differs from the example as shown in Fig.4 with respect to the placement ofmicrophone #3505 (inlet) where the microphone is placed on the third side rather than the second side where microphone #1 501 and microphone #3505 are placed diagonally opposite on the third side.

[0187] Following example configurations Fig.6 to Fig.9 satisfy design constraints shown in Fig.2, including additional constraints shown in Fig.2d, but only design constraints Fig.2a to Fig.2c are shown in figures for improved visual clarity of the document.

[0188] Fig.6 shows another example configuration. Fig.6 microphone (inlet) placement (designated #42) retains the microphone placement as the placement shown in Fig.5 for microphone #1 601, microphone #2 603, and microphone #3 605, but differs from the example as shown in Fig.5 with respect to the placement of microphone #4607 (inlet) where the microphone is placed on the third side rather than the first side where microphone #1 601 and microphone #4607 are placed diagonally opposite with respect to the third side and fourth side and the z-axis.

[0189] Fig.7 shows a further example configuration. Fig.7 microphone (inlet) placement (designated #43) retains the microphone placement as the placement shown in Fig.5 for microphone #1 701, microphone #2 703, and microphone #3 705, but differs from the example as shown in Fig.5 with respect to the placement of microphone #4707 (inlet) where the microphone is placed on the first side at the location of the earpiece (and thus sharing resources with the earpiece outlet or port) rather than having a further port for the microphone sound inlet. Furthermore, the microphone #1 701 and microphone #4707 are placed diagonally opposite with respect to the z-axis.

[0190] Fig.8 shows a further example configuration. Fig.8 microphone (inlet) placement (designated #44) retains the microphone placement as the placement shown in Fig.5 for microphone #1 801, microphone #2 803, and microphone #3 805, but differs from the example as shown in Fig.5 with respect to the placement of microphone #4807 (inlet) where the microphone is placed on the first side and that microphone inlet pairs microphone #1 801, microphone #3805 and microphone #2803, microphone #4807 are aligned with z and y axis respectively.

[0191] Fig.9 shows a further example configuration. Fig.9 microphone (inlet) placement (designated #45) retains the microphone placement as the placement shown in Fig.5 for microphone #1 901 and microphone #3905, but differs from the example as shown in Fig.5 with respect to the placement of microphone #2 903 and microphone #4 907 (inlet) where the microphone #2 903 is placed on the second side and close to centerline (and the camera module protrusion is not utilized) and the microphone #4907 placed on the fourth side also close to the centreline.

[0192] With respect to Fig.10 is shown a generalized view of the placement of the four microphones. In this example the device 200 comprises microphone A 1005 located on the first side, a microphone B 1007 located on the camera module protrusion of the second side, where the diagonal between the microphone A 1005 and the microphone B 1007 is at an angle a 1006 relative to the y-z plane.

[0193] Furthermore, microphone C 1001 placed on the third side, where the diagonal from the microphone C 1002 and the microphone B 1007 is at an angle g 1002 relative to the y-z plane and from the microphone B 1007 and the microphone C 1002 is at an angle b 1008 relative to the y-z plane.

[0194] Finally microphone D 1003 is shown placed on the third side, where the diagonal from the microphone D 1002 and the microphone C 1001 is at an angle d 1003 relative to the y-z plane.

[0195] As described above these placements can be tested by a suitable microphone position optimization method developed for mobile application. The method supports simulation models and measurement results enabling realistic virtual prototypes of real HW products.

[0196] The example microphone placements shown in Figs.4 to 9 produces results which are shown in Fig.11. Fig 11 for example show example graphs of audio focusing performance for selected target directions for various example microphone inlet positions as shown in Figs.41101, Fig.5 1103, Fig.6 1105, Fig.7 1107, Fig.8 1109 and Fig.9 1111.

[0197] Fig.12 furthermore shows example graph of average audio focusing performance for the selected target directions for various example microphone inlet positions as shown in Figs.4 1201 , Fig.5 1203, Fig.6 1205, Fig .7 1207, Fig .8 1209 and Fig.9 1211.

[0198] Fig.13 furthermore shows example graphs of directivity (polar pattern) at 984Hz towards a front direction for various example microphone inlet positions as shown in Figs.4 1301, Fig.5 1303, Fig.6 1305, Fig.7 1307, Fig.8 1309 and Fig.9 1311.

[0199] Fig.14 shows example graphs of directivity (polar pattern) for selected frequencies towards a front direction for various example microphone inlet positions as shown in Figs.4 1401, Fig.5 1403, Fig.6 1405, Fig.7 1407, Fig.8 1409 and Fig.9 1411.

[0200] Fig.15 shows example graphs of focused audio capture performance (directivity index) towards various focus directions for the example microphone inlet positions as shown in Figs.4 1501, Fig.5 1503, Fig.6 1505, Fig.7 1507, Fig.8 1509 and Fig.9 1511.

[0201] Fig.16 shows an example graph of relative spatial analysis performance for the example microphone inlet positions as shown in Fig.5 1603, Fig.6 1605, Fig.7 1607, Fig.8 1609 and Fig.9 1611 compared to the positions shown in Fig.4.

[0202] Fig.17 shows example graphs of direction analysis error for DOA angle spatial analysis accuracy for the example microphone inlet positions as shown in Figs.4 1701, Fig.5 1703, Fig.6 1705, Fig.7 1707, Fig.8 1709 and Fig.9 1711.

[0203] Fig.18 shows example graphs for Directivity variability in different directions for the example microphone inlet positions as shown in Figs.41901 , Fig.51903, Fig.61905, Fig.71907, Fig.81909 and Fig.9 1911.

[0204] Fig.20 shows simulation results of 14993 array configurations where the value of the optimization cost function defined the x-axis and y-axis is the normalized volume of the skew quadrilateral prism. Normalization is implemented in this example by dividing the volume of the prism with a volume of the device body (L'W'D). Array aperture volume correlates positively with optimization results since smartphone devices are small compared to audio wavelengths of interest and reliable direction analysis (DOA) estimation at low frequencies and spatial capture directivity at low frequencies are both relevant for audio capture use and both benefit from large array aperture. However, too large separation between microphone inlets spatial undersampling, which leads to spatial aliasing. Spatial aliasing creates ambiguity in direction analysis and introduce grating lobes in array algorithms. Spatial aliasing problem can be addressed by reducing the size of the array aperture or by adding the number of microphones. The 30 arrays with lowest optimization cost are highlighted with “o” circle symbols. The geometric interpretation of the aperture volume differs from acoustic properties of the array and therefore there are examples where smaller geometrical aperture can provide close to similar performance compared to larger geometrical aperture. Likewise there are examples where two similar size geometrical apertures have different performance and cost. Accurate performance estimation of microphone arrays integrated into physical objects require analysis in acoustic domain, where microphone inlet distances are evaluated as acoustic distances between microphone inlets. Geometric design criteria can provide accurate performance estimates in tree-field conditions, which can be helpful in limiting the search space for feasible array configurations integrated into physical objects.

[0205] Fig. 21 shows the microphone port positions corresponding to subset of 30 simulated microphone inlet configurations with lowest optimization cost as an example on device surface. It can be observed that microphone inlets at corner areas are concentrated on smaller surface areas compared to more flat surface areas.

[0206] Fig. 22 shows simulation results of 14993 array configurations where effective aperture is estimated using sum of microphone port vector lengths as a sum cross products vectors lengths |AB x CD | + |AC x BD| + |AD x BC|.

[0207] .Fig..23 shows simulation results of 14993 array configurations where effective aperture is estimated using sum of microphone port areas as a sum cross products vectors lengths |PXZ(AB) X PXZ(CD)|.

[0208] With respect to focused audio capture, it is typical to aim for recording of monophonic or stereophonic audio. This is considered in this example design by defining a number of spatial directions where focused capture performance is evaluated as a part of design criteria. In this example the applied criteria has been mono capture towards approximated main camera direction, stereo capture both in landscape and portrait device orientation as well as mono capture towards display direction for selfie camera use case. This can also be described using typical audio channel naming convention as center, left, right, portrait left, portrait right and center for back direction.

[0209] Microphone position optimization is active using two cost functions for audio focusing and direction accuracy for spatial capture.

[0210] The above Figs thus show relative audio focusing (beamforming) performance difference between studied optimized designs and practical design variations. It can be seen that smartphone concept does not have equal performance in all directions mainly due to lack of microphones on the display side. Audio focusing performance is illustrated in the above Figs showing good performance for all of the example embodiments on all presented design concepts while design concept #40 is shown by the example simulation results to produce the best results.

[0211] With respect to Fig.19 shows a six-microphone configuration based on the above placement methods. In this example the charging port is located on the fifth side rather than the third side and is configured with casing with a radius or bevel with respect to the third, fourth, fifth and sixth sides. For example the bevel can define approximately a normal edge, first inclined edge relative to the first side (for example 45 degrees) and second inclined edge relative to the second side (for example 45 degrees). There are furthermore other example configurations over which a six-microphone example placement can be implemented.

[0212] In this example a first microphone 2005 is placed on the camera module protrusion and is oriented approximately on the normal of the second side.

[0213] Furthermore, a second microphone 2001 is placed on the third side (away from the camera module protrusion side) towards the fifth side part of the third side and is oriented at a first orientation (towards the second side).

[0214] Furthermore, a third microphone 2003 is placed on the third side (away from the camera module protrusion side) towards the sixth side part of the third side and is oriented at a second orientation (towards the first side). In some embodiments the orientation of the second microphone 2001 or third microphone 2003 can be other orientations - for example the same or different.

[0215] Additionally a fourth microphone 2007, fifth microphone 2009, and sixth microphone 2011 are placed on the fourth side (closer to the camera module protrusion side) towards the fifth side part of the fourth side, substantially central on the fourth side and towards the sixth side part of the fourth side respectively.

[0216] In some embodiments the orientations of the fourth microphone 2007, fifth microphone 2009, and sixth microphone 2011 can differ, for example as shown in Fig.19 where the fourth microphone 2007 is orientated to the second side, fifth microphone 2009 to the first side, and sixth microphone 2011 approximately tangential to the fourth side.

[0217] It is also noted herein that while the above describes example embodiments, there are several variations and modifications which may be made to the disclosed solution without departing from the scope of the present invention.

[0218] As used herein, “at least one of the following: ” and “at least one of ” and similar wording, where the list of two or more elements are joined by “and” or “or”, mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements.

[0219] In general, the various embodiments may be implemented in hardware or special purpose circuitry, software, logic or any combination thereof. Some aspects of the disclosure may be implemented in hardware, while other aspects may be implemented in firmware or software which may be executed by a controller, microprocessor or other computing device, although the disclosure is not limited thereto. While various aspects of the disclosure may be illustrated and described as block diagrams, flow charts, or using some other pictorial representation, it is well understood that these blocks, apparatus, systems, techniques or methods described herein may be implemented in, as non-limiting examples, hardware, software, firmware, special purpose circuits or logic, general purpose hardware or controller or other computing devices, or some combination thereof.

[0220] As used in this application, the term “circuitry” may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry) and(b) combinations of hardware circuits and software, such as (as applicable):(c) a combination of analog and / or digital hardware circuit(s) with software / firmware and (i) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and(ii) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0221] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example and if applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

[0222] The embodiments of this disclosure may be implemented by computer software executable by a data processor of the mobile device, such as in the processor entity, or by hardware, or by a combination of software and hardware. Computer software or program, also called program product, including softwareroutines, applets and / or macros, may be stored in any apparatus-readable data storage medium and they comprise program instructions to perform particular tasks. A computer program product may comprise one or more computer-executable components which, when the program is run, are configured to carry out embodiments. The one or more computer-executable components may be at least one software code or portions of it.

[0223] Further in this regard it should be noted that any blocks of the logic flow as in the Figures may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on such physical media as memory chips, or memory blocks implemented within the processor, magnetic media such as hard disk or floppy disks, and optical media such as DVD and the data variants thereof, CD. The physical media is a non-transitory media.

[0224] The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM).

[0225] The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as semiconductor based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. The data processors may be of any type suitable to the local technical environment, and may comprise one or more of general purpose computers, special purpose computers, microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASIC), FPGA, gate level circuits and processors based on multi core processor architecture, as non-limiting examples.

[0226] Embodiments of the disclosure may be practiced in various components such as integrated circuit modules. The design of integrated circuits is by and large a highly automated process. Complex and powerful software tools are available for converting a logic level design into a semiconductor circuit design ready to be etched and formed on a semiconductor substrate.

[0227] The scope of protection sought for various embodiments of the disclosure is set out by the independent claims. The embodiments and features, if any, described in this specification that do not fall under the scope of the independent claims are to be interpreted as examples useful for understanding various embodiments of the disclosure.

[0228] The foregoing description has provided by way of non-limiting examples a full and informative description of the exemplary embodiment of this disclosure. However, various modifications and adaptations may become apparent to those skilled in the relevant arts in view of the foregoing description, when read in conjunction with the accompanying drawings and the appended claims. However, all such and similar modifications of the teachings of this disclosure will still fall within the scope of this invention as defined inthe appended claims. Indeed, there is a further embodiment comprising a combination of one or more embodiments with any of the other embodiments previously discussed.

Claims

CLAIMS1. An apparatus for capturing spatial audio, the apparatus comprising:at least four microphones; andat least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets are located to define a three-dimensional four-sided capturing geometry for spatial audio, and wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

2. The apparatus as claimed in claim 1 , wherein the three-dimensional four-sided capture geometry is a skew quadrilateral.

3. The apparatus as claimed in any of claims 1 or 2, wherein the three-dimensional four-sided capture geometry performs at least one of:maximises an aperture of a microphone array comprising the at least four microphones; defines an aperture of a microphone array comprising the at least four microphones which exceeds a threshold value.

4. The apparatus as claimed in any of claim 3, wherein the aperture of a microphone array comprising the at least four microphones is defined by:defining a first vector with respect to a first pair of the at least four microphones, the first pair of the at least four microphones being located towards one end of the apparatus;defining a second vector with respect to a second pair of the at least four microphones, the second pair of the at least four microphones being located towards an opposite end of the apparatus; and defining a cross-product based on one of:the first vector and the second vector; anda projection of the first vector to a first plane and a projection of the second vector to the first plane.

5. The apparatus as claimed in any of claims 1 to 4, comprising a structure within which the at least four microphone inlets are located, the structure comprising:a first surface, the first surface comprising a display;a second surface, the second surface located opposite the first surface;a third surface, the third surface connecting the first surface and second surface, and comprising a charging port;a fourth surface, the fourth surface connecting the first surface and the second surface and located opposite the third surface;a fifth surface, the fifth surface connecting the first surface, the second surface, the third surface and the fourth surface; anda sixth surface, the sixth surface connecting the first surface, the second surface, the third surface and the fourth surface and located opposite the fifth surface.

6. The apparatus as claimed in claim 5, wherein at least one of the surfaces comprises a beveling or curving towards at least one of the other surfaces.

7. The apparatus as claimed in claim 5 or 6, wherein the second surface comprises a camera module protrusion projecting from the second surface.

8. The apparatus as claimed in any of claims 5 to 7, wherein a first and a second of the at least four microphone inlets are located on the second surface, wherein the first microphone inlet is located substantially diagonally opposite the second microphone inlet.

9. The apparatus as claimed in any of claims 5 to 7, wherein a first of the at least four microphone inlets is located at a first location on the second surface and a second of the at least four microphone inlets is located at a second location on the second surface, wherein the distance between the first and second locations is substantially maximized with respect to a defined apparatus dimension.

10. The apparatus as claimed in claim 8 or 9 when dependent on claim 5, wherein one of the first microphone inlet or the second microphone inlet is located on the camera module protrusion.

11. The apparatus as claimed in any of claims 8 to 10, wherein a third of the at least four microphone inlets is located on the third surface, wherein the third microphone inlet is orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

12. The apparatus as claimed in any of claims 8 to 10, wherein a third of the at least four microphone inlets is located at a third location on the third surface, wherein the distance between the third location and one of the first and second locations is substantially maximized.

13. The apparatus as claimed in claim 11 or 12, wherein a fourth of the at least four microphone inlets is located on the fourth surface, wherein the fourth microphone inlet is orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

14. The apparatus as claimed in claim 11 or 12, wherein a fourth of the at least four microphone inlets is located at a fourth location on the fourth surface, wherein the distance between the fourth location and one of: the first; second or third locations is substantially maximized.

15. The apparatus as claimed in any of claims 4 or 5, wherein a first of the at least four microphone inlets is located on the second surface.

16. The apparatus as claimed in claim 15, wherein a second and a third of the at least four microphone inlets are located on the third surface, wherein the third microphone inlet is orientated towards the first surface and the third microphone inlet is located substantially diagonally opposite the second microphone inlet on the third surface.

17. The apparatus as claimed in claim 16, wherein a fourth of the at least four microphone inlets is located on the fourth surface, wherein the fourth microphone inlet is orientated towards the first surface and located distant from the first microphone inlet and the second microphone inlet.

18. The apparatus as claimed in claim 16, wherein a fourth of the at least four microphone inlets is located on the fourth surface, and located distant from the first microphone inlet and the second microphone inlet.

19. The apparatus as claimed in any of claims 12 to 15, wherein pairs of the at least four microphone inlets are aligned with respect to axis defined by the surfaces.

20. A method for providing an apparatus for capturing spatial audio, the apparatus comprising:at least four microphones; andat least four microphone inlets, each microphone inlet associated with a respective microphone from the at least four microphones, wherein the at least four microphone inlets is located to define a three-dimensional four-sided capturing geometry for spatial audio, and wherein a size of the three-dimensional four-sided capture geometry is provided according to a geometry of the apparatus.

Images