setting parameter values

Abstract

An apparatus, method, and computer program product for receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal, receiving an instruction to set the audio parameter to a first parameter value, in response to determining that the first parameter value satisfies at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value.

Description

TECHNICAL FIELD

[0001] The present application relates generally to spatial audio. More specifically, the present application relates to setting parameter values for controlling spatial audio. BACKGROUND

[0002] The amount of multimedia content is constantly increasing. Users create and consume multimedia content and it plays an important role in modern society. SUMMARY

[0003] Various aspects of examples of the application are set out in the claims. The scope of the protection sought to be afforded by the various embodiments of the present application, therefore, is defined by the claims. Examples and features described in this specification that are not within the scope of the claims if any, are to be interpreted as examples useful for understanding the various embodiments of the present application.

[0004] According to a first aspect of the present application, there is provided an apparatus comprising means for: receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal, receiving an instruction to set the audio parameter to a first parameter value, in response to determining that the first parameter value meets at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value.

[0005] According to a second aspect of the present application, there is provided a method comprising: receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal, receiving an instruction to set the audio parameter to a first parameter value, in response to determining that the first parameter value meets at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value.

[0006] According to a third aspect of the present application, there is provided a computer program comprising instructions for causing an apparatus to perform at least the following: receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal, receiving an instruction to set the audio parameter to a first parameter value, in response to determining that the first parameter value meets at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value.

[0007] According to a fourth aspect of the invention, an apparatus is provided, comprising at least one processor and at least one memory, the at least one memory including computer program code, the at least one memory and the computer program code being configured together with the at least one processor to cause the apparatus to at least: receive captured spatial audio information, the captured spatial audio information including at least one audio signal and at least one audio parameter for controlling the at least one audio signal, receive an instruction to set the audio parameter to a first parameter value, and, in response to determining that the first parameter value satisfies at least one criterion, set the audio parameter to a second parameter value and change the audio parameter from the second parameter value to the first parameter value.

[0008] According to a fifth aspect of the invention, a non-transient computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following operations: receiving captured spatial audio information, the captured spatial audio information including at least one audio signal and at least one audio parameter for controlling the at least one audio signal; receiving an instruction to set the audio parameter to a first parameter value; and, in response to determining that the first parameter value satisfies at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value.

[0009] According to a sixth aspect of the present invention, a computer-readable medium is provided, the computer-readable medium including program instructions for causing a device to perform at least the following operations: receiving captured spatial audio information, the captured spatial audio information including at least one audio signal and at least one audio parameter for controlling at least one audio signal; receiving instructions to set the audio parameter to a first parameter value; and, in response to determining that the first parameter value satisfies at least one criterion, setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value. Attached Figure Description

[0010] To gain a more complete understanding of exemplary embodiments of the present invention, reference is now made to the following description taken in conjunction with the accompanying drawings, in which:

[0011] Figure 1 A block diagram of an example apparatus is shown, in which examples of the disclosed embodiments can be applied;

[0012] Figure 2 A block diagram of another example device to which examples of the disclosed embodiments can be applied is shown;

[0013] Figure 3A and Figure 3B The illustration shows an example of changing audio focus;

[0014] Figure 4A , Figure 4B and Figure 4CThis illustration shows another example of changing audio focus;

[0015] Figure 5A , Figure 5B and Figure 5C This illustration shows yet another example of changing audio focus;

[0016] Figure 6 An example method is illustrated;

[0017] Figure 7 Another example method is illustrated;

[0018] Figure 8 Another example method is illustrated. Detailed Implementation

[0019] The following embodiments are exemplary. Although the specification may refer to "a," "an," or "some" embodiments in several places in the text, this does not necessarily mean that each reference refers to the same embodiment(s), or that one or more particular features apply only to a single embodiment. Individual features of different embodiments may also be combined to provide other embodiments.

[0020] The example embodiments relate to receiving captured spatial audio information, which includes at least one audio signal and at least one audio parameter for controlling the at least one audio signal. The example embodiments also relate to providing perceptible feedback to a user who has captured the spatial audio information.

[0021] Spatial audio can include full spherical surround sound to mimic how people perceive audio in real life. Spatial audio can include audio emanating from the user's location to be assigned a direction and / or distance. Spatial audio can include audio created by sound sources, ambient audio, or a combination thereof. For example, ambient audio can include audio that may not be identifiable in relation to the sound source (such as traffic hum, wind, or waves). Therefore, the perceived audio can change as the user moves or rotates. Full spherical surround sound can include a spatial audio field around a reference point.

[0022] According to one example embodiment, the reference point corresponds to the user's location. According to another example embodiment, the reference point corresponds to the location of the capturing device when capturing spatial audio information. According to yet another example embodiment, the reference point corresponds to the center point of the spatial audio field. For example, the location of the capturing device can be considered the center of the spatial audio field.

[0023] For example, spatial audio can be captured using a capture device that includes multiple microphones configured to capture audio signals around the capture device. In addition to capturing audio signals, the capture device can also be configured to capture different types of information, such as one or more parameters related to the captured audio signals and / or visual information. The captured parameters can be stored along with the captured audio or in separate files. The capture device can be, for example, a camera, a video recorder, or a smartphone.

[0024] Spatial audio can include one or more parameters, such as audio focus parameters and / or audio scaling parameters. Audio parameters can include parameter values ​​with respect to a reference point, such as the user's location or the location of the capture device. Modifying spatial audio parameter values ​​can cause changes in the spatial audio perceived by the listener.

[0025] Audio focus features allow users to focus on audio in a desired direction when capturing and / or playing back content. Therefore, audio focus features also allow users to at least partially eliminate background noise. When capturing content, in addition to capturing the audio, the direction of the sound is also captured. The direction of the sound can be defined relative to a reference point. For example, the direction of the sound can include an angle or discrete direction relative to a reference point, such as front, back, left, right, up, and / or down relative to the reference point, or combinations thereof. The reference point can correspond to, for example, a value of 0 degrees or no audio focus direction, in which case the audio consists of surround sound without audio focus at the reference point. Audio focus parameters can also include one or more further levels of detail, such as horizontal focus direction and / or vertical focus direction.

[0026] Audio scaling features allow users to amplify sound. Amplifying sound involves adjusting the amount of audio gain associated with a specific direction. Therefore, audio scaling parameters correspond to sensitivity to the direction of sound. Audio scaling can be performed using audio beamforming, which allows users to control, for example, the size, shape, and / or direction of the audio beam. Performing audio scaling can include controlling audio signals from a specific direction while attenuating audio signals from other directions. For example, audio scaling features can allow control over audio gain. Audio gain can include the amount of gain set for an audio input signal from a certain direction. Audio scaling parameter values ​​can be defined relative to a reference point. For example, audio scaling parameters can be percentage values, and the reference point can correspond to, for example, a 0% value, in which case, at the reference point, the audio includes surround sound without audio scaling. As another example, audio scaling features can allow different microphone signals to be delayed differently and then summed, thereby achieving spatial filtering of the audio.

[0027] Audio scaling can be associated with scaling visual information. For example, if a user records a video and zooms in on an object, the audio can also be amplified on the object, for instance, emphasizing the sound produced by the object while attenuating other sounds. In other words, spatial audio parameters can be controlled by controlling video scaling.

[0028] Providing users with the ability to control audio parameters, such as manually focusing audio in a specific direction, can be confusing in some situations. For example, a user might assume that in response to focusing audio in a desired direction, there should be an audible change in the perceived audio. However, in some cases, the audio focus may already be in the desired direction, resulting in no audible change. Due to this lack of feedback, the user might mistakenly believe, for example, that the audio focus is not working, or that the user has entered the wrong command. This can lead to a negative user experience and may also cause the user to enter repetitive commands to focus the audio.

[0029] According to an example embodiment, an apparatus is configured to receive captured spatial audio information, the captured spatial audio information including at least one audio signal and at least one audio parameter for controlling the at least one audio signal. The at least one audio parameter may include an audio parameter corresponding to the direction of the audio relative to a reference point. The at least one audio parameter may include, for example, an audio focus parameter. The apparatus is also configured to receive an instruction to set the audio parameter to a first parameter value and to determine whether the first parameter value satisfies at least one criterion. The at least one criterion may include an active parameter value or an effect caused by an active parameter value. The apparatus is further configured to, in response to determining that the first parameter value satisfies at least one criterion, set the audio parameter to a second parameter value and change the audio parameter from the second parameter value to the first parameter value. The apparatus may be configured to set the audio parameter to the second parameter value over a time period. The apparatus may be configured to change the audio parameter from the second parameter value to the first parameter value while outputting the captured spatial information.

[0030] The captured content can be stored in volatile or non-volatile memory. For example, when content is captured, it can be saved as a file on the capture device or a file located on a cloud service. When playing back the captured content, the saved file can be downloaded to the playback device and opened. The captured content can also be edited on the capture and / or playback device. Storing the captured content can include storing the captured content in one or more files. The captured content can include spatial audio information and / or captured visual information.

[0031] Figure 1This is a block diagram depicting an apparatus 100 operating according to an exemplary embodiment of the present invention. Apparatus 100 may be, for example, an electronic device, such as a chip or chipset. Apparatus 100 includes one or more control circuitry systems, such as at least one processor 110 and at least one memory 160, including one or more algorithms, such as computer program code 120, wherein at least one memory 160 and computer program instructions are configured, together with at least one processor 110, to cause the apparatus to perform any of the exemplary functions described below.

[0032] exist Figure 1 In the example, processor 110 is a control unit operatively connected to read from and write to at least one memory 160. Processor 110 may also be configured to receive control signals via an input interface and / or processor 110 may be configured to output control signals via an output interface. In the example embodiment, processor 110 may be configured to translate received control signals into appropriate commands for use in controlling the function of the device.

[0033] At least one memory 160 stores computer program code 120, which controls the operation of device 100 when loaded into processor 110, as described below. In other examples, device 100 may include more than one memory 160 or different types of storage devices.

[0034] Computer program code 120, or a portion thereof, used to implement exemplary embodiments of the present invention, may be loaded onto device 100 by the manufacturer of device 100, by a user of device 100, or by device 100 itself based on a download program, or may be pushed to device 100 via an external device. The computer program code may reach device 100 via electromagnetic carrier signals or be copied from a physical entity such as a computer program product, storage device, or recording medium such as an optical disc (CD), optical disc read-only memory (CD-ROM), digital versatile optical disc (DVD), or Blu-ray disc).

[0035] Figure 2 This is a block diagram depicting an apparatus 200 according to an exemplary embodiment of the present invention. Apparatus 200 may be an electronic device, such as a handheld portable device, mobile phone or personal digital assistant (PDA), personal computer (PC), laptop computer, desktop computer, tablet computer, wireless terminal, communication terminal, game console, music player, e-book reader, positioning device, digital camera, home appliance, CD player, DVD player or Blu-ray player, or media player. In the following examples, it is assumed that apparatus 200 is a mobile computing device or part thereof.

[0036] exist Figure 2In an example embodiment, device 200 is illustrated as including device 100, one or more microphones 210, one or more speakers 230, and a user interface 220 for interacting with device 200 (e.g., a mobile computing device). Device 200 may also include a display configured to act as user interface 220. For example, the display may be a touchscreen display. In the example embodiment, the display and / or user interface 220 may be external to device 200, but in communication with it.

[0037] Additionally or alternatively, the user interface 220 may also include manually operable controls such as buttons, keys, touchpads, joysticks, styluses, pens, scroll wheels, joysticks, keypads, keyboards, or any suitable input mechanism for inputting and / or accessing information. Other examples include cameras, voice recognition systems, eye-motion recognition systems, and input systems based on acceleration, tilt, and / or motion. Therefore, the device 200 may also include various types of sensors, such as one or more gyroscope sensors, accelerometers, magnetometers, positioning sensors, and / or tilt sensors.

[0038] Figure 2 The example device 200 can also be configured to establish radio communication with another device using, for example, Bluetooth, WiFi, radio frequency identification (RFID), or near field communication (NFC) connections.

[0039] According to an example embodiment, device 200 is configured to receive captured spatial audio information from a microphone array including at least two microphones 210. Device 200 is also configured to apply one or more beamforming operations to the captured spatial audio information to emphasize audio signals from a particular direction and / or attenuate sounds from other directions.

[0040] Beamforming operations can include forming an audio beam by selecting a specific microphone arrangement for capturing spatial audio information from a first direction and / or attenuating sound from a second direction and processing the received audio information. In other words, a microphone array can be used to form a spatial filter configured to extract signals from a specific direction and / or reduce contamination from signals from other directions.

[0041] According to an example embodiment, device 200 is configured to change the direction of an audio beam. Changing the direction of the audio beam may include changing the direction of the audio beam relative to a reference point in a spatial audio field. For example, changing the direction of the audio beam may include changing the direction of the audio beam from a first direction to a second direction. When the audio beam is pointed in the first direction, the audio signal from that direction is emphasized, and when the audio beam is pointed in the second direction, the audio signal from that direction is emphasized.

[0042] The apparatus 200 can be configured to change the direction of the audio beam by switching from a first microphone arrangement to a second microphone arrangement, by using algorithms with different parameters, and / or by using different algorithms for processing the captured audio information. For example, in the case of delay-and-beamformer, beam steering can be achieved by adjusting the value of the steering delay so that signals from a particular direction are aligned before summation. As another example, in the case of parametric spatial audio processing that analyzes the spatial audio field as a directional component with an arrival direction parameter and an ambient component without an arrival direction, beam steering can be achieved by changing the arrival direction on which the directional signal component is enhanced.

[0043] According to an example embodiment, device 200 is configured to activate an audio beam. Activating an audio beam may include activating an audio beam that has been formed but is not activated until it is activated. For example, device 200 may be configured to form a first beam and a second beam but activate one beam at a time. Activating an audio beam may include activating the first audio beam and deactivating the second audio beam. For example, activating an audio beam may include causing the deactivation of the currently activated audio beam.

[0044] According to an example embodiment, device 200 is configured to receive captured spatial audio information, which includes at least one audio signal and at least one audio parameter for controlling the at least one audio signal. Device 200 may be configured to receive the captured spatial audio information, for example, by receiving a file from a separate device, by opening a file included by device 200, or by receiving the spatial audio information as streaming content.

[0045] Receiving captured spatial audio information may include receiving spatial audio information by capturing spatial audio information and / or by playing back spatial audio information.

[0046] According to an example embodiment, device 200 is configured to capture spatial audio information. Device 200 may be configured to capture spatial audio information using, for example, a microphone array including two or more microphones 210.

[0047] According to an example embodiment, device 200 is configured to play back captured spatial audio information. Device 200 may be configured to use, for example, one or more speakers to play back the spatial audio information.

[0048] Capturing spatial audio information may also include capturing other types of information. For example, assuming device 200 includes a video recorder, capturing spatial audio information may also include capturing visual information.

[0049] Audio parameters can be included in spatial audio information in different ways. For example, audio parameters can be included in spatial audio information during or after its capture. Including audio parameters in spatial audio information during capture can include, for example, manually adjusting the audio parameters or applying automatic sound source tracking techniques, such as visual object tracking or audio-based sound source tracking. Including audio parameters in spatial audio information after its capture can include, for example, editing the audio parameters using editing tools or by applying automatic sound source tracking techniques, such as visual object tracking or audio-based sound source tracking techniques.

[0050] At least one audio parameter may include one or more spatial audio parameters. Spatial audio parameters may include parameters used to control specific spatial audio characteristics. Spatial audio parameters may include parameters defined relative to a reference point. Spatial audio parameters may also include combinations of multiple parameters. For example, spatial audio parameters may include the direction of the audio the user is interested in and the amount of audio gain in that direction. The different audio parameters included by the spatial audio may have a common reference point.

[0051] Spatial audio parameters can include parameters used to control the characteristics of spatial audio. Characteristics can include, for example, volume, effects volume, or translation in 3D space.

[0052] According to the example embodiment, spatial audio parameters include spatial audio parameter values. Spatial audio parameter values ​​can be defined relative to a reference point in the spatial audio field. For example, a spatial audio parameter value can be defined as an angle or direction relative to the reference point. If a spatial audio parameter value corresponds to a reference point, it can be interpreted that the spatial audio parameter value is zero or that the spatial audio signal has not been modified by the spatial audio parameters.

[0053] According to an example embodiment, at least one audio parameter includes an audio focus parameter corresponding to the direction of the audio relative to a reference point. For example, a user may perceive the audio as being located to the user's left or right. The direction of the audio may include a direction relative to a reference point, such as left, right, front, back, up, down, or any combination thereof. The direction may also include an auditory angle relative to a reference point, such as 30°, 80°, 120°, or 240°, or any other angle between 0° and 360°.

[0054] According to an example embodiment, the reference point includes the center of the spatial audio field. The reference point may include the user's location or the location of the capturing device.

[0055] According to an example embodiment, device 200 is configured to output captured spatial audio information. Device 200 may be configured to output the captured spatial audio information using, for example, one or more speakers 230.

[0056] According to an example embodiment, device 200 is configured to receive an instruction to set an audio parameter to a first parameter value. Device 200 may be configured to receive the instruction from a user, a device, or a software module.

[0057] According to an example embodiment, device 200 is configured to receive the instruction from a user. For example, the user may be a user of device 200.

[0058] According to an example embodiment, device 200 is configured to receive instructions from an application. The application may include an application executed by device 200 or an application communicating with device 200. The application may include, for example, a camera application or a voice recording application, with the camera application including video recording features. For example, device 200 may be configured to receive instructions from a video recording application to automatically set audio focus parameters when the video recording application is launched. In this case, the user may not be aware that the audio focus feature has been activated.

[0059] The first parameter value can be indicated by a numerical value, a discrete value, a selection, or an input corresponding to a numerical value. For example, the first parameter can be indicated by a number, direction, a selected area on the user interface, a gesture, or any other suitable input.

[0060] According to an example embodiment, device 200 is configured to perform an action in response to receiving an instruction to set an audio parameter to a first parameter value based on an active parameter value. In other words, device 200 can be configured to perform a first action if the active parameter has a first value, and a second action if the active parameter has a second value. The active parameter value can refer to the value of the audio parameter that has been set before receiving the instruction to set the audio parameter to the first parameter value. In other words, the active parameter value can include the current value of the parameter. For example, if the active parameter value is equal to the first parameter value, device 200 can be configured to perform an action different from the received instruction, instead of setting the audio parameter to the first value.

[0061] The active parameter value may include the parameter value that is active when the device 200 receives the instruction, a predetermined parameter value, or a default parameter value. Therefore, the device 200 can be configured to dynamically determine the action to be performed in response to receiving an instruction to set the audio parameters to a specific value.

[0062] According to an example embodiment, device 200 is configured to determine whether a first parameter value satisfies at least one criterion. Determining whether the first parameter satisfies at least one criterion may include, for example, comparing the first parameter value with an active parameter value.

[0063] As another example, determining whether a first parameter satisfies at least one criterion may include, for example, comparing the effect caused by the first parameter value with the effect caused by the active parameter value. Comparing the effect caused by the first parameter value with the effect caused by the active parameter value may include comparing the amount, type, and / or characteristics of the effect caused by the first parameter value with the amount, type, and / or characteristics of the effect caused by the active parameter value accordingly. The effect caused by the parameter may include an effect perceptible to the user. According to an example embodiment, at least one criterion includes the active parameter value or the effect caused by the active parameter value.

[0064] Determining whether the first parameter meets the criteria may include determining whether the value of the first parameter is equal to or substantially equal to the value represented by the criteria, higher than or lower than the threshold represented by the criteria, or within the range defined by the criteria.

[0065] According to an example embodiment, the device 200 is configured to set an audio parameter to a second parameter value in response to determining that a first parameter satisfies at least one criterion.

[0066] Setting the audio parameter to a second parameter value may include changing the direction of the audio beam or activating the audio beam such that audio signals from the direction corresponding to the second parameter value are emphasized. According to an example embodiment, the device is configured to set the audio parameter to a second parameter value by changing the direction of the audio beam or activating the audio beam.

[0067] The device 200 can be configured to determine a second parameter value based on a first parameter value. For example, if the first parameter value corresponds to an audio focus in a first direction relative to a reference point in the spatial audio field, the device 200 can be configured to determine a second parameter value such that the second parameter value corresponds to a direction relative to the reference point that is not connected to the first direction.

[0068] Device 200 can be configured to determine the second parameter dynamically or based on predetermined settings. Device 200 can be configured to determine the second parameter based on information related to the spatial audio field (such as information about the location of sound objects and / or ambient sounds). Dynamically determining the second parameter may include, for example, determining the second parameter such that it corresponds to a direction where no sound object is present. Predetermined settings may include parameter values ​​corresponding to non-focal parameter values ​​or parameter values ​​associated with a predetermined audio focal point.

[0069] According to an example embodiment, the second value differs from the first value. Setting the audio parameter to the second parameter value can include modifying the audio beam used to capture spatial audio. Modifying the audio beam can include, for example, modifying the shape and / or size of the beam, pointing the audio beam in a specific direction, activating the audio beam, or deactivating the audio beam.

[0070] The device 200 is configured to temporarily set an audio parameter to a second parameter value. According to an example embodiment, the device 200 is configured to set the audio parameter to the second parameter value for a period of time in response to determining that a first parameter value satisfies at least one criterion. This period of time may include a predetermined period of time or a dynamic period of time, such as a period of time used to output audible differences when outputting captured spatial audio information.

[0071] According to an example embodiment, the second parameter value includes a parameter value corresponding to a reference point in the spatial audio field. As described above, the reference point may correspond to the user's location, the location of the capturing device when capturing spatial audio information, or the center point of the spatial audio field. Furthermore, the reference point may correspond to an audio parameter value that substantially does not cause modification to the audio signal.

[0072] According to an example embodiment, the first parameter value includes a value corresponding to the autofocus direction, and the second parameter value includes a value corresponding to the direction associated with the ambient sound. The autofocus direction may include the direction in which the audio beam is automatically focused. The device 200 may be configured to determine the autofocus direction by applying automatic sound source tracking technology (such as visual object tracking or audio-based sound source tracking). For example, the use of autofocus functionality may be activated by a user or an application (such as a video / audio recording application).

[0073] According to an example embodiment, device 200 is configured to change an audio parameter from a second parameter value to a first parameter value. In other words, device 200 can be configured to temporarily set the audio parameter to a second value, and then change the audio parameter from the second parameter value to the first parameter value. Changing the audio parameter from the second parameter value to the first parameter value may include, for example, replacing the second parameter value with the first parameter value.

[0074] Changing an audio parameter from a second parameter value to a first parameter value may include changing the direction of the audio beam or activating the audio beam such that audio signals from the direction corresponding to the first parameter value are emphasized. According to an example embodiment, the device is configured to change the audio parameter from a second parameter value to a first parameter value by changing the direction of the audio beam or activating the audio beam.

[0075] Without limiting the scope of the claims, the advantage of setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to a first parameter value is as follows: if the user instruction device sets the audio parameter to an already activated value, the user can receive feedback on the instruction even if the parameter value has not changed.

[0076] According to an example embodiment, the device 200 is configured to set an audio parameter to a second parameter value in response to determining that a first parameter value satisfies at least one criterion, and to change the audio parameter from the second parameter value to the first parameter value while outputting the captured spatial audio information.

[0077] Without limiting the scope of the claims, the advantage of setting the audio parameter to a second parameter value and changing the audio parameter from the second parameter value to the first parameter value is that the user can perceive the change in the audio output.

[0078] According to an example embodiment, apparatus 200 includes components for performing features of the claimed invention, wherein the components for performing the features include at least one processor 110 and at least one memory 160, the at least one memory 160 including computer program code 120, the at least one memory 160 and the computer program code 120 being configured to, together with at least one processor 110, cause performance of apparatus 200. The components for performing features of the claimed invention may include components for receiving captured spatial audio information, the captured spatial audio information including at least one audio signal and at least one audio parameter for controlling the at least one audio signal, components for receiving an instruction to set the audio parameter to a first parameter value, components for determining whether the first parameter value satisfies at least one criterion, components for setting the audio parameter to a second parameter value in response to determining that the first parameter value satisfies at least one criterion, and components for changing the audio parameter from the second parameter value to the first parameter value.

[0079] The component for receiving instructions may include a component for receiving instructions from a user or a component for receiving instructions from an application.

[0080] The component for setting the audio parameter to a second parameter value may include a component for changing the direction of the audio beam or a component for activating the audio beam. The component for changing the audio parameter from a second parameter value to a first parameter value may include a component for changing the direction of the audio beam or a component for activating the audio beam.

[0081] The components for implementing the features of the claimed invention may further include components for outputting the captured spatial audio signal and / or components for setting audio parameters to a second parameter value and for changing the audio parameters from the second parameter value to a first parameter value while outputting the captured spatial audio information. The device may also include components for setting the audio parameters to the second parameter value over a time period. The device may also include components for changing the direction of the audio beam and / or activating the audio beam.

[0082] Figure 3A and Figure 3BThe illustration shows an example of changing audio focus. Figure 3A and Figure 3B The device 300 in the example is a mobile computing device. The mobile computing device 300 includes an array of microphones and one or more speakers. The device 300 (e.g., the mobile computing device) is configured to capture spatial audio information, which includes at least one audio signal and at least one audio parameter for controlling the at least one audio signal. The mobile computing device 300 is considered to be located at a reference point in the spatial audio field. Figure 3A and Figure 3B In the example, at least one audio parameter includes an audio focus parameter that causes the audio signal to be emphasized in a particular direction.

[0083] exist Figure 3A In the example, a first person 301 and a second person 302 are illustrated. The mobile computing device 300 captures spatial audio information such that the audio focus is directed at the second person 302. For example, when an autofocus function is activated, the focus can be automatically directed at the second person 302. Audio focus capture produces a mono signal in which specific spatial sectors are emphasized. The audio focus can be directed at the second person 302 using an audio beam 303, which is created using one or more beamforming techniques. When the audio focus is directed at the second person 302, the output of the captured audio includes, for example, the speech of the second person 302, which is louder than the speech of the first person 301.

[0084] exist Figure 3B In the example, mobile computing device 300 receives instruction 304 from a user of mobile computing device 300 to set the audio focus toward a first person 301. In response to receiving this instruction, the audio focus changes from a second person 302 to the first person 301. Changing the audio focus from the second person 302 to the first person 301... Figure 3B In the example, this is indicated by arrow 305. The previous direction of the audio beam 303 is... Figure 3B The image is shown as a dashed line. Audio focus can be directed at a first person 301 using audio beam 306, which is created using one or more beamforming techniques. When the audio focus is directed at the first person 301, the captured audio output includes, for example, the voice of the first person 301, which is louder than the voice of the second person 302.

[0085] As a result of the change, the user of the mobile computing device 300 can perceive the change in the audio output. For example, firstly, the user can hear the voice of the second person 302 louder, and in response to the change in audio focus, the user can hear the voice of the first person 301 louder.

[0086] Figure 4A , Figure 4B and Figure 4CThis illustration shows another example of changing audio focus. Figure 4A In the example, assume that the mobile computing device 300 has automatically set the audio focus 402 to the first person 301. Therefore, the mobile computing device 300 captures spatial audio such that the audio focus 402 is directed towards the first person 301. Figure 4A In the example, mobile computing device 300 receives instruction 401 from a user of mobile computing device 300 to set audio focus 402 toward a first person 301 who has already had audio focus set. Mobile computing device 300 is configured to determine that audio focus has been set to the first person 301.

[0087] exist Figure 4B In the example, in response to determining that the audio focus 402 is pointing at the first person 301, the mobile computing device 300 moves the audio focus from the first person 301 to the reference point. In other words, the audio focus is removed, and the audio output is unfocused at 404. Moving the audio focus to the reference point is indicated by arrow 403, and... Figure 4B In the example, the previous audio focus 402 is illustrated by a dashed line. The audio focus can be moved to the reference point within a time period.

[0088] exist Figure 4C In the example, the mobile computing device 300 moves the audio focus from the reference point (unfocused 404) back to the first person 301 (audio focus 402). In other words, the mobile computing device 300 moves the audio parameter value from the unfocused value to a parameter value corresponding to the direction of the first person 301. In this example of changing the audio focus, the user perceives the first person 301's voice as becoming quieter relative to the rest of the audio field, and then as becoming louder relative to the rest of the audio field. The mobile computing device can also be configured to narrow the audio focus 402 toward the first person 301 and / or increase its gain to emphasize the effect perceived by the user. In this way, the user receives feedback about the instruction to set the audio focus toward the first person 301, thereby improving the user experience.

[0089] Figure 5A , Figure 5B and Figure 5C This illustration shows another example of changing audio focus. Figure 5A , Figure 5B and Figure 5C In the example, the mobile computing device 300 is configured to form a first audio beam 501 and a second audio beam 502. For example, the first audio beam 501 is directed toward a moving sound object and the second audio beam 502 is directed away from any moving sound object, such as being directed in a direction that primarily contains ambient noise.

[0090] exist Figure 5AIn the example, the mobile computing device 300 has automatically set the audio focus to a first person 301 that is an active sound object using a first audio beam 501. The mobile computing device 300 is also configured to form a second audio beam 502 directed toward ambient sound. Figure 5A In the example, mobile computing device 300 receives instruction 503 from the user of mobile computing device 300 to set the audio focus to the first person 301 that it has already set.

[0091] exist Figure 5B In the example, the mobile computing device 300 sets the audio focus toward ambient sound, in which case the user can perceive the first person 301 as quieter relative to the rest of the audio field. Figure 5B In the example, the audio focus is changed from the first person 301 to ambient sound, indicated by arrow 504.

[0092] exist Figure 5C In the example, the mobile computing device 300 sets the audio focus back to the first person 301. This causes the voice of the first person 301 to become louder relative to the rest of the audio field. Similar to... Figure 4A , Figure 4B and Figure 4C For example, the user receives feedback on instructions to set the audio focus to the first person 301, thereby improving the user experience.

[0093] Figure 6 An example method 600 incorporating aspects of previously disclosed embodiments is illustrated. More specifically, example method 600 illustrates changing an audio parameter from a second parameter to a first parameter.

[0094] The method begins by receiving 605 spatial audio information, which includes at least one audio signal and at least one audio parameter for controlling the at least one audio signal. The at least one audio parameter may include an audio focus parameter corresponding to the direction of the audio relative to a reference point.

[0095] The method continues to receive an instruction from 610 to set the audio parameter to the first parameter value. For example, this instruction can be received from a user.

[0096] The method then proceeds to determine whether the first parameter value (615) satisfies at least one criterion. At least one criterion may include the active parameter value or the effect caused by the active parameter value. Determining whether the first parameter value satisfies at least one criterion may include comparing the first parameter value with the active parameter value, or comparing the effect caused by the first parameter value with the effect caused by the active parameter value.

[0097] The method further proceeds in response to determining that the first parameter value satisfies at least one criterion, setting the audio parameter to 620 as the second parameter value. The first parameter value and the second parameter value are different.

[0098] The method further changes the audio parameter from the second parameter value to the first parameter value by 625.

[0099] Figure 7 An example method 700 incorporating aspects of previously disclosed embodiments is illustrated. More specifically, example method 700 illustrates determining whether a first parameter satisfies at least one criterion. Figure 7 In the example, it is assumed that the device executing method 700 has already been connected to Figure 6 Similarly, the device 200 receives captured spatial audio information, which includes an audio signal and at least one audio parameter for controlling the audio signal, and the device 200 has received an instruction to set the audio parameter to a first parameter value.

[0100] The method begins by determining the 705 active parameter value. The active parameter value may include the parameter value that is active when device 200 receives the instruction, a predetermined parameter value, or a default parameter value.

[0101] The method continues by comparing the first parameter value with the active parameter value 710. If, based on the comparison, it is determined that the first parameter value is substantially equal to the active parameter value, then the second parameter value is determined 715. The second parameter value can be determined dynamically, or a predetermined value can be selected.

[0102] Figure 8 An example method 800 incorporating aspects of previously disclosed embodiments is illustrated. More specifically, example method 800 illustrates another method for determining whether a first parameter satisfies at least one criterion. Figure 8 In the example, assume the device executing method 800 is... Figure 6 and Figure 7 The example method similarly receives captured spatial audio information, which includes an audio signal and at least one audio parameter for controlling the audio signal, and the device 200 has received an instruction to set the audio parameter to a first parameter value.

[0103] The method begins by determining the active parameter value of 805. The active parameter value may include the parameter value that is active when device 200 receives the instruction, a predetermined parameter value, or a default parameter value.

[0104] The method continues to compare the effect caused by the first parameter value with the effect caused by the active parameter value 810. If, based on the comparison, it is determined that the effect caused by the first parameter value is substantially equal to the effect caused by the active parameter value, then the second parameter value is determined 815. The second parameter value can be determined dynamically, or a predetermined value can be selected.

[0105] Without limiting the scope, interpretation, or application of the following claims in any way, the technical effect of one or more example embodiments disclosed herein is to provide better feedback to the user, thereby making the use of the device more efficient. Another technical effect of one or more example embodiments is that different beamforming techniques can be used to provide feedback to the user.

[0106] As used in this application, the term "circuit system" may refer to one or more or all of the following: (a) a hardware circuit implementation only (such as an implementation only in analog and / or digital circuit systems) and (b) a combination of hardware circuits and software, such as (if applicable): (i) a combination of (multiple) analog and / or digital hardware circuits with software / firmware and (ii) any part of (multiple) hardware processors having software (including (multiple) digital signal processors), software, (multiple) memories, which work together to enable a device (such as a mobile phone or a server) to perform various functions) and (c) (multiple) hardware circuits and / or (multiple) processors, such as (multiple) microprocessors or a portion of (multiple) microprocessors, which require software (e.g., firmware) for operation, but may be absent when operation does not require the software.

[0107] The definition of "circuit system" applies to all uses of the term in this application, including in any claim. As another example, as used in this application, the term "circuit system" also covers implementations of only hardware circuitry or a processor (or multiple processors) or a portion thereof and its accompanying software and / or firmware. For example, if applicable to a particular claim element, the term "circuit system" also covers baseband integrated circuits or processor integrated circuits for mobile devices, or similar integrated circuits in servers, cellular network devices, or other computing or network devices.

[0108] Embodiments of the present invention can be implemented in software, hardware, application logic, or a combination of software, hardware, and application logic. The software, application logic, and / or hardware may reside on a device, a separate device, or multiple devices. If desired, a portion of the software, application logic, and / or hardware may reside on a device, a portion of the software, application logic, and / or hardware may reside on a separate device, and a portion of the software, application logic, and / or hardware may reside on multiple devices. In example embodiments, the application logic, software, or instruction set is maintained on any one of a variety of conventional computer-readable media. In the context of this document, "computer-readable media" can be any medium or component that can contain, store, communicate, propagate, or transmit instructions for use by or in connection with an instruction execution system, device, or apparatus (such as a computer), wherein... Figure 2An example of a computer is described and depicted. A computer-readable medium may include a computer-readable storage medium, which may be any medium or component that can contain or store instructions for use by or in connection with an instruction execution system, apparatus, or device, such as a computer.

[0109] If necessary, the different functions discussed in this document can be executed in different orders and / or concurrently with each other. Furthermore, if necessary, one or more of the functions described above can be optional or can be combined.

[0110] Although various aspects of the invention are set forth in the independent claims, other aspects of the invention include other combinations of features from the described embodiments and / or dependent claims with features of the independent claims, and not only the combinations expressly listed in the claims.

[0111] It will be apparent to those skilled in the art that the concept of this invention can be implemented in various ways as technology advances. The invention and its embodiments are not limited to the examples described above, but can be varied within the scope of the claims.

Claims

1. An apparatus for controlling spatial audio, comprising: means for receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal; means for receiving an instruction to set an audio parameter to a first parameter value, wherein the audio parameter is comprised in the at least one audio parameter for controlling the at least one audio signal; means for determining whether the first parameter value fulfils at least one criterion based on whether the first parameter value is equal to or within a predetermined range of an active parameter value, or an effect caused by the first parameter value is equal to an effect caused by the active parameter value, wherein the active parameter value comprises a parameter value active at the time of receiving the instruction; means for setting the audio parameter to a second parameter value for a time period in response to determining that the first parameter value fulfils the at least one criterion; and means for changing the audio parameter from the second parameter value to the first parameter value after the time period.

2. The apparatus according to claim 1, comprising means for outputting the captured spatial audio information.

3. The apparatus according to claim 2, comprising means for setting the audio parameter to the second parameter value and changing the audio parameter from the second parameter value to the first parameter value while outputting the captured spatial audio information.

4. The apparatus according to claim 1 or 2, wherein the audio parameter comprises at least one audio parameter corresponding to a direction of audio relative to a reference point.

5. The apparatus according to claim 1 or 2, wherein the time period is predetermined or is a dynamic time period.

6. The apparatus according to claim 1 or 2, wherein the at least one criterion comprises an active parameter value or an effect caused by the active parameter value.

7. The apparatus according to claim 1 or 2, wherein the second parameter value comprises a parameter value corresponding to a reference point in a spatial audio field.

8. The apparatus according to claim 1 or 2, wherein the second parameter value is different from the first parameter value.

9. The apparatus according to claim 1 or 2, wherein the first parameter value comprises a value corresponding to an auto-focus direction and the second parameter comprises a value corresponding to a direction associated with an ambient sound.

10. The apparatus according to claim 1 or 2, wherein the means for setting the audio parameter to the second parameter value and / or the means for changing the audio parameter from the second parameter value to the first parameter value comprises means for changing a direction of an audio beam.

11. The apparatus according to claim 1 or 2, wherein the means for setting the audio parameter to the second parameter value and / or the means for changing the audio parameter from the second parameter value to the first parameter value comprises means for activating an audio beam.

12. The apparatus according to claim 1 or 2, wherein the apparatus comprises means for receiving the instruction from an application. ​ 13. The apparatus of claim 1 or 2, wherein the apparatus comprises means for receiving the instruction from a user.

14. A method for controlling spatial audio, comprising: receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal; receiving an instruction to set an audio parameter to a first parameter value, wherein the audio parameter is comprised in the at least one audio parameter for controlling the at least one audio signal; determining whether the first parameter value fulfils at least one criterion based on whether the first parameter value is equal to or within a predetermined range of an active parameter value, or an effect caused by the first parameter value is equal to an effect caused by the active parameter value, wherein the active parameter value comprises a parameter value active at the time of receiving the instruction; in response to determining that the first parameter value fulfils the at least one criterion, setting the audio parameter to a second parameter value for a time period; and changing the audio parameter from the second parameter value to the first parameter value after the time period.

15. A computer program product comprising instructions for causing an apparatus to perform at least the following: receiving captured spatial audio information, the captured spatial audio information comprising at least one audio signal and at least one audio parameter for controlling the at least one audio signal; receiving an instruction to set an audio parameter to a first parameter value, wherein the audio parameter is comprised in the at least one audio parameter for controlling the at least one audio signal; determining whether the first parameter value fulfils at least one criterion based on whether the first parameter value is equal to or within a predetermined range of an active parameter value, or an effect caused by the first parameter value is equal to an effect caused by the active parameter value, wherein the active parameter value comprises a parameter value active at the time of receiving the instruction; in response to determining that the first parameter value fulfils the at least one criterion, setting the audio parameter to a second parameter value for a time period; and changing the audio parameter from the second parameter value to the first parameter value after the time period.

Images