API for spatial audio control
Through the operating system, the API is used to receive audio session parameters, adjust and send audio data, the flexibility of audio presentation in the XR environment is solved, and the user's immersion experience and audio control accuracy are improved.
Patent Information
- Application Number
- CN202410698237.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-02
- Filing Date
- 2024-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
Developers lack simple ways to define the spatialization of audio in Extended Reality (XR) environments, resulting in the inflexible and precise presentation of audio in XR environments.
Through the application programming interface (API), the operating system receives audio session parameters, adjusts audio data, and sends it to the speaker for head-tracking, fixed or non-spatial experience audio playback.
It realizes flexible spatialization of audio in XR environment, enhances user immersion experience, and provides more precise audio position and volume control.
Smart Images

Figure CN119071714A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 470,693, filed on June 2, 2023, which is hereby incorporated by reference in its entirety. Technical Field
[0003] The present disclosure generally relates to systems, methods, and devices for controlling the spatial rendering of sound via an application programming interface (API). Background Art
[0004] In various implementations, an extended reality (XR) environment presented by an electronic device including a display includes one or more scenes displayed by an application. In addition, the electronic device includes two or more speakers that play audio provided by the application. However, there is no easy way for developers to define how audio will be spatialized in the XR environment. BRIEF DESCRIPTION OF THE DRAWINGS
[0005] So that the present disclosure may be understood by those of ordinary skill in the art, a more particular description may be had with reference to aspects of some illustrative embodiments, some of which are shown in the accompanying drawings.
[0006] Figure 1 is a block diagram of an exemplary operating environment according to some implementations.
[0007] Figure 2A and Figure 2B An XR environment is shown during various time periods according to some implementations.
[0008] Figure 3 An electronic device according to some implementations is shown.
[0009] Figure 4 is a flowchart representation of a method of displaying an image according to some specific implementations.
[0010] As is common practice, the various features shown in the drawings may not be drawn to scale. Therefore, for clarity, the sizes of the various features may be arbitrarily expanded or reduced. In addition, some drawings may not depict all components of a given system, method, or device. Finally, throughout the specification and drawings, similar reference numerals may be used to represent similar features. Summary of the invention
[0011] Various specific implementations disclosed herein include devices, systems, and methods for playing audio. In various specific implementations, the method is performed by a device that is located in a physical environment, coupled to two or more speakers, and includes one or more processors and non-volatile memory. The method includes executing an operating system and an application. The method includes receiving, by the operating system, an audio session parameter including a spatial experience value from the application via an application programming interface, the audio session parameter providing instructions for spatial playback of audio associated with the application. The method includes receiving, by the operating system, an instruction to play audio data from the application. The method includes adjusting, by the operating system, the audio data based on the spatial experience value. The method includes sending, by the operating system, the adjusted audio data to the two or more speakers.
[0012] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; the one or more programs are stored in the non-volatile memory and are configured to be executed by one or more processors, and the one or more programs include instructions for performing or causing the execution of any of the methods described herein. According to some specific implementations, a non-volatile computer-readable storage medium stores instructions that, when executed by one or more processors of the device, cause the device to perform or cause the execution of any of the methods described herein. According to some specific implementations, a device includes: one or more processors, non-volatile memory, and means for performing or causing the execution of any of the methods described herein. DETAILED DESCRIPTION
[0013] Many details are described in order to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings only illustrate some example aspects of the present disclosure and should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail so as not to obscure more relevant aspects of the example implementations described herein.
[0014] In various embodiments, an extended reality (XR) environment presented by an electronic device including a display includes one or more scenes displayed by an application. In addition, the electronic device includes two or more speakers that play audio provided by the application. Developers can indicate how audio will be spatialized in the XR environment via an application programming interface (API). Specifically, an audio session API is provided that allows application developers to instruct the operating system to instantiate an audio session that plays audio to achieve a head-tracked spatial experience, a fixed spatial experience, or a non-spatial experience.
[0015] Figure 11 is a block diagram of an exemplary operating environment 100 according to some implementations. Although relevant features are shown, those of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein. To this end, as a non-limiting example, the operating environment 100 includes a controller 110 and an electronic device 120.
[0016] In some implementations, the controller 110 is configured to manage and coordinate the user's XR experience. In some implementations, the controller 110 includes a suitable combination of software, firmware, and / or hardware. In some implementations, the controller 110 is a computing device located locally or remotely relative to the physical environment 105. For example, the controller 110 is a local server located within the physical environment 105. In another example, the controller 110 is a remote server (e.g., a cloud server, a central server, etc.) located outside the physical environment 105. In some implementations, the controller 110 is communicatively coupled to the electronic device 120 via one or more wired or wireless communication channels 144 (e.g., Bluetooth, IEEE 802.11x, IEEE 802.16x, IEEE 802.3x, etc.). For example, the controller 110 is included in the housing of the electronic device 120. In some implementations, the functionality of the controller 110 is provided by the electronic device 120 and / or combined with the electronic device.
[0017] In some implementations, the electronic device 120 is configured to provide an XR experience to the user. In some implementations, the electronic device 120 includes a suitable combination of software, firmware, and / or hardware. According to some implementations, the electronic device 120 presents XR content to the user via a display 122 when the user is physically present in a physical environment 105, the physical environment including a table 107 within a field of view 111 of the electronic device 120. In some implementations, the user holds the electronic device 120 in one or both of his / her hands. In some implementations, when providing XR content, the electronic device 120 is configured to display an XR object (e.g., an XR cylinder 109) and implement video pass-through of the physical environment 105 (e.g., including a representation 117 of the table 107) on the display 122.
[0018] According to some implementations, the electronic device 120 provides an XR experience to the user while the user is virtually and / or physically present within the physical environment 105 .
[0019] In some implementations, the user wears the electronic device 120 on his / her head. For example, in some implementations, the electronic device includes a head-mounted system (HMS), a head-mounted device (HMD), or a head-mounted housing (HME). Therefore, the electronic device 120 includes one or more XR displays configured to display XR content. For example, in various implementations, the electronic device 120 surrounds the user's field of view. In some implementations, the electronic device 120 is a handheld device (such as a smart phone or tablet) configured to present XR content, and the user no longer wears the electronic device 120 but holds the device with the display facing the user's field of view and the camera facing the physical environment 105. In some implementations, the handheld device may be placed in a housing that can be worn on the user's head. In some implementations, the electronic device 120 is replaced with an XR cabin, housing, or room configured to present XR content, in which the user no longer wears or holds the electronic device 120.
[0020] Figure 2A and Figure 2B An XR environment 200 is shown, at least in part, displayed by a display of the electronic device, from the perspective of a user of the electronic device. In various implementations, the user's perspective is from the position of an image sensor of the electronic device. For example, in various implementations, the electronic device is a handheld electronic device, and the user's perspective is from the position of the image sensor of the handheld electronic device facing the physical environment. In various implementations, the user's perspective is from the position of the user of the electronic device. For example, in various implementations, the electronic device is a head-mounted electronic device, and the user's perspective is from the position of the user facing the physical environment, which, in the absence of a head-mounted electronic device, typically approximates the user's field of view. In various implementations, the user's perspective is from the position of an avatar of the user. For example, in various implementations, the XR environment 200 is a virtual environment, and the user's perspective is from the position of an avatar or other representation of the user facing the virtual environment.
[0021] Figure 2A and Figure 2B The XR environment 200 is shown during a series of time periods. In various implementations, each time period is an instant, a fraction of a second, a few seconds, a few hours, a few days, or any length of time.
[0022] The XR environment 200 includes multiple objects, including one or more real objects (e.g., sea 211, tree 212, and sun 213) and one or more virtual objects (e.g., virtual digital clock 211 of the clock application, virtual alarm 222 of the clock application, virtual notepad window 223 of the notepad application, virtual audiobook window 224 of the audiobook application, and virtual sphere 225 of the meditation application). In various specific implementations, certain objects (such as real objects, virtual alarm 222, virtual audiobook window 224, and virtual sphere 225) are displayed at locations in the XR environment 200, for example, at locations defined by three coordinates in a three-dimensional (3D) XR coordinate system. Therefore, when the electronic device moves (e.g., changes position and / or orientation) in the XR environment 200, the object moves on the display of the electronic device but maintains its position in the XR environment 200. Such virtual objects that move on the display but maintain their position in the XR environment 200 in response to the motion of the electronic device are referred to as world-locked objects. In various implementations, certain virtual objects (such as the virtual digital clock 221) are displayed at a position on the display such that the object is stationary on the display on the electronic device when the electronic device moves in the XR environment 200. Such virtual objects that maintain their position on the display in response to the motion of the electronic device are referred to as head-locked objects or display-locked objects.
[0023] Figure 2A The XR environment 200 during a first time period is shown. During the first time period, a virtual digital clock 221 is displayed at a digital clock position on the display. During the first time period, a virtual alarm 222 is displayed on the display at a first alarm position corresponding to the alarm position in the XR environment 200 (on the right side of the display). In various specific implementations, during the first time period, the clock application instructs the electronic device to generate a sound from the alarm position in the XR environment 200 (e.g., a ticking sound that increases in volume as the alarm time approaches). Therefore, the user perceives the sound as being generated from the virtual alarm 222 at the first alarm position on the display, for example, on the right side of the user. During the first time period, a virtual notepad window 223 is displayed on the display at a first notepad window position corresponding to the notepad window position in the XR environment 200 (on the right side of the display). In various implementations, during the first time period, the note application instructs the electronic device to generate a sound (e.g., a ringtone indicating that another user has edited a shared note displayed in the virtual note window 223) from a note window location in the XR environment 200. As a result, the user perceives the sound as being generated from the virtual note window 223 at the first note window location on the display, e.g., to the right of the user.
[0024] During the first time period, the virtual sphere 224 is displayed on the display at a first sphere position corresponding to the sphere position in the XR environment 200 (at the center of the display). In various embodiments, the virtual sphere 224 periodically increases and decreases in size and / or luminous intensity to indicate a breathing pattern to the user. In various embodiments, during the first time period, the meditation application instructs the electronic device to produce sounds independent of the ball position or the posture of the electronic device. Specifically, the meditation application provides a left audio stream that will be played unchanged by the left speaker and a right audio stream that will be played unchanged by the right speaker. For example, in various embodiments, the meditation application instructs the device to produce binaural beats that will be reduced or eliminated by spatially positioning the left and right audio streams.
[0025] During the first time period, the virtual audiobook window 225 is displayed on the display at a first audiobook window position (on the left side of the display) that corresponds to the audiobook window position in the XR environment 200. In various specific implementations, during the first time period, the audiobook application instructs the electronic device to generate sound (e.g., the sound of the audiobook being read) from a position in front of the user during the first time period. Specifically, the position in front of the user during the first time period is not the audiobook window position in the XR environment 200, but rather a position in front of the user whose corresponding position in the XR environment 200 changes as the user (and the device) moves.
[0026] Figure 2B The XR environment 200 is shown during a second time period after the first time period. During the second time period, the posture of the electronic device has changed from the first posture to the second posture compared to the first time period. Specifically, the electronic device has moved to the right.
[0027] During the second time period, the virtual digital clock 221 is displayed at the digital clock position of the display. Therefore, between the first time period and the second time period, the virtual digital clock 221 has not moved on the display.
[0028] During the second time period, the virtual alarm 222 is displayed on the display at a second alarm position corresponding to the alarm position in the XR environment 200 (in the center of the display). Therefore, between the first time period and the second time period, the virtual alarm 222 has not moved in the XR environment 200, but has moved to the left on the display. In various specific implementations, during the second time period, the clock application instructs the electronic device to generate a sound (e.g., an alarm) from the alarm position in the XR environment 200. Therefore, the user perceives the sound as being generated from the virtual alarm 222 at the second alarm position on the display, for example, in front of the user. During the second time period, the virtual notepad window 223 is displayed on the display at a second notepad window position corresponding to the notepad window position in the XR environment 200 (in the center of the display). Therefore, between the first time period and the second time period, the virtual notepad window 223 has not moved in the XR environment 200, but has moved to the left on the display. In various implementations, during the second time period, the note application instructs the electronic device to generate a sound (e.g., a ringtone indicating that another user has edited a shared note displayed in the virtual note window 223) from the note window location in the XR environment 200. As a result, the user perceives the sound as being generated from the virtual note window 223 at the second note window location on the display, e.g., in front of the user.
[0029] During the second time period, the virtual sphere 224 is displayed on the display at a second sphere position corresponding to the sphere position in the XR environment 200 (on the left side of the display). In various specific implementations, during the second time period, the meditation application instructs the electronic device to produce sounds independently of the ball position or the posture of the electronic device. Specifically, the meditation application provides a left audio stream that will be played unchanged by the left speaker and a right audio stream that will be played unchanged by the right speaker. For example, in various specific implementations, the meditation application instructs the device to produce binaural beats, which will be reduced or eliminated by spatially positioning the left audio stream and the right audio stream. Therefore, the perceived position (if any) of the sound produced according to the instructions of the meditation application does not change between the first time period and the second time period.
[0030] During the second time period, the virtual audiobook window 225 is not displayed because the corresponding audiobook window position in the XR environment 200 is outside the current field of view. However, in various specific implementations, during the second time period, the audiobook application instructs the electronic device to generate sound (e.g., the sound of the audiobook being read) from the position in front of the user during the second time period. Specifically, the position in front of the user during the second time period is not the audiobook window position in the XR environment 200, nor is it the position in front of the user during the first time period, but rather is a position in front of the user whose corresponding position in the XR environment 200 changes as the user (and the device) moves.
[0031] Figure 3 3 is a functional block diagram of a device 300 according to some specific implementations. The device 300 includes a system layer 310, an application layer 330, and two or more speakers 350. The system layer 310 includes an operating system 320 and one or more drivers 323. The application layer 330 includes an application 340 having an application implementation module 241 and an API call module 341. The operating system 320 includes an API 321 and an OS implementation module 322. In various specific implementations, the API 321 is a software module (e.g., a set of computer-readable instructions) that provides an interface that allows different modules (e.g., the API call module 341) to access and use one or more functions, methods, procedures, data structures, classes, and / or other services provided by the OS implementation module 322 of the operating system 320. The API 321 may define one or more parameters passed between the API call module 341 and the OS implementation module 322. The OS implementation module 322 is an operating system software module (e.g., a set of computer-readable instructions) that is configured to perform operations in response to receiving an API call via the API 321. In some embodiments, the OS implementation module is configured to provide an API response (via API 321) as a result of processing an API call.
[0032] In various specific implementations, API 321 receives audio session parameters including a spatial experience value from API calling module 341, which provide spatialization instructions for spatial playback of audio. In various specific implementations, API 321 includes an audio session API that receives audio session parameters. In response, OS implementation module 322 instantiates an audio session of application 340 based on the audio session parameters. In various specific implementations, API 321 receives instructions to play audio data from API calling module 341. In various specific implementations, API 321 includes an audio playback API that receives instructions to play audio data. In response, OS implementation module 322 plays audio data according to the audio session parameters (including the spatial experience value). In various specific implementations, OS implementation module 322 plays audio data by providing instructions to driver 323 to drive speaker 350.
[0033] Figure 4 4 is a flowchart of a method 400 for playing audio according to some specific implementations. In various specific implementations, the method 400 is performed by an electronic device (such as Figure 1 Electronic equipment 120 or Figure 3 The method 400 is performed by an electronic device 300 of the present invention. In various implementations, the method 400 is performed by a device that is located in a physical environment, coupled to two or more speakers, and includes one or more processors and non-transitory memory. In some implementations, the method 400 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 400 is performed by a processor that executes instructions (e.g., code) stored in a non-transitory computer-readable medium (e.g., memory).
[0034] Method 400 begins in box 410, where the device executes an operating system and an application. Method 400 continues in box 420, where the device receives, by the operating system via an application programming interface, audio session parameters including a spatial experience value from the application, the audio session parameters providing instructions for spatial playback of audio associated with the application. In various specific implementations, the spatial experience value is a string indicating a spatial experience. For example, in various specific implementations, the spatial experience value indicates a head-tracked spatial experience, a fixed (or non-head-tracked) spatial experience, or a non-spatial experience (or ignoring the spatial experience). In various specific implementations, in response to receiving the audio session parameters, the operating system instantiates an audio session for the application according to the audio session parameters.
[0035] The method 400 continues in block 430, where the device receives, by the operating system, an instruction to play the audio data from the application. In various implementations, the operating system receives the instruction to play the audio data via an application programming interface. In various implementations, the application programming interface includes an audio session application programming interface (which receives the audio session parameters in block 420) and an audio playback application programming interface (which receives the instruction to play the audio data in block 430).
[0036] The method 400 continues at block 440 where the device, by the operating system, adjusts the audio data based on the spatial experience value. The method 400 continues at block 450 where the device, by the operating system, sends the adjusted audio data to the two or more speakers.
[0037] In various implementations, the spatial experience value indicates a head tracking spatial experience, and adjusting the audio data includes adjusting the audio data to be played from the location in the physical environment. In various implementations, the audio data is spatialized to be played from the location in the physical environment using binaural rendering, in which a source signal is filtered using two head-related transfer functions (HRTFs) based on a relative position of a user's head to the location in the physical environment (e.g., determined using head tracking), and the resulting signal is played by two speakers proximate to two ears of the user, respectively.
[0038] In various implementations, the audio session parameters also include an anchor value indicating a location in a physical environment. For example, in various implementations, the anchor value indicates a scene having a location in a physical environment. Figure 2A and Figure 2B , the clock application provides a spatial experience value indicating the head tracking spatial experience, and also provides the position of the virtual alarm clock 222 in the XR environment 200. Therefore, the electronic device plays the audio data from the position of the virtual alarm clock 222. In various specific implementations, the anchor value indicates an element of the scene, such as a specific object (e.g., a virtual speaker or a virtual window) or a specific user interface element (e.g., a button of a virtual window). In various specific implementations, the anchor value indicates a default position in the physical environment. For example, in various specific implementations, the anchor value indicates the front of the physical environment. In various specific implementations, the front of the physical environment is specified when the device is powered on. In various specific implementations, the front of the physical environment can be changed by the user during use of the device. In various specific implementations, the anchor value indicates a set of three-dimensional coordinates in the three-dimensional world coordinate system of the physical environment.
[0039] In various implementations, the audio session parameters also include a size value indicating a size of the location in the physical environment, and adjusting the audio data includes adjusting the audio data to play from the location in the physical environment having the size. In various implementations, the size of the audio data is adjusted to play from the location in the physical environment having the size by adjusting virtual speaker size, gain, delay, reverberation, or other audio parameters. In various implementations, the size can be small (which plays as if played from a point source at the location in the physical environment), medium (which plays as a small theater surrounding the location in the physical environment), or large (which plays as an immersive experience centered on the location in the physical environment).
[0040] In various implementations, the audio session parameters further include an attenuation value indicating distance attenuation, and adjusting the audio data includes adjusting the volume of the audio data based on a distance between the device and the location in the physical environment. In various implementations, the attenuation value may indicate disabling distance attenuation, and adjusting the audio data includes adjusting the audio data independent of the distance between the device and the location in the physical environment.
[0041] In various implementations, the spatial experience value indicates a fixed spatial experience, and adjusting the audio data includes adjusting the audio data to be played from a position relative to the device. In various implementations, the audio data is spatialized to be played from a position relative to the device using binaural rendering, in which a source signal is filtered using two head-related transfer functions (HRTFs) based on a position relative to the device (and independent of a posture of the device), and the resulting signal is played by two speakers proximate to two ears of the user, respectively.
[0042] In various implementations, the audio session parameters also include an anchor value indicating a location relative to the device. Figure 2A and Figure 2B In the embodiment of the present invention, the audiobook application (and / or the podcast application) provides a spatial experience value indicating a fixed spatial experience, and also provides a position in front of the electronic device. Therefore, the electronic device plays the audio data from the front of the electronic device. In various specific implementations, the anchor value indicates a default position relative to the device. For example, in various specific implementations, the anchor value indicates a position in front of the device. In various specific implementations, the anchor value indicates a set of three-dimensional coordinates in a three-dimensional device coordinate system of the device. In various specific implementations, the anchor value indicates an angle relative to the device. For example, in various specific implementations, the anchor value indicates a position on the left side of the device, in front of the device, on the right side of the device, or behind the device.
[0043] In various implementations, the audio session parameters also include a size value indicating a size relative to the location of the device, and adjusting the audio data includes adjusting the audio data to play from the location relative to the device having the size. In various implementations, the size of the audio data is adjusted to play from the location relative to the device having the size by adjusting virtual speaker size, gain, delay, reverberation, or other audio parameters. In various implementations, the size can be small (which plays as if played from a point source at the location in the physical environment), medium (which plays as a small theater surrounding the location in the physical environment), or large (which plays as an immersive experience centered on the location in the physical environment).
[0044] In various implementations, the spatial experience value indicates a non-spatial experience, and adjusting the audio data includes adjusting the audio data without performing spatialization. Figure 2A and Figure 2B In the present invention, the meditation application provides a spatial experience value indicating a non-spatial experience, and the electronic device plays the audio data as received.
[0045] In various specific implementations, if the operating system receives an instruction to instantiate an audio session with a spatial experience value, the operating system instantiates the audio session with a spatial experience value set to the received spatial experience value. In various specific implementations, if the operating system receives an instruction to instantiate an audio session without receiving a spatial experience value from an application, the operating system instantiates an audio session with a spatial experience value set to an automatic value, which indicates that the operating system selects a spatial experience value in response to receiving an instruction to play audio data. In various specific implementations, the operating system selects a spatial experience value indicating a head tracking spatial experience with a medium size at the location of the scene displayed by the application by default. However, in various applications, the operating system selects different spatial experience values, different sizes, or different locations. For example, in various specific implementations, the operating system selects a non-head tracking spatial experience at a front position for a long-form audio application (e.g., music, audiobooks, or podcast players). For example, in various specific implementations, the operating system selects a large size for a video application or a local application, or a small size can be selected for a picture-in-picture application. Thus, in various implementations, the operating system selects a spatial experience value, size, and / or location based on other data provided by the application via the application programming interface.
[0046] As described above, in various implementations, the operating system receives an instruction to instantiate an audio session with a spatial experience value, and the operating system instantiates the audio session with the spatial experience value set to the received spatial experience value. In various implementations, the operating system also receives user input from the user, which overwrites the spatial experience value with the user value. For example, in various implementations, the user can provide user input to downgrade the spatial experience to a non-spatial experience. Therefore, in various implementations, the method also includes adjusting the audio data according to the user value.
[0047] As described above, in various implementations, the audio session parameters also include an attenuation value indicating distance attenuation, and adjusting the audio data includes adjusting the volume of the audio data based on the distance between the device and the location in the physical environment. In various implementations, the attenuation value is one of a plurality of preset values (e.g., enabled or disabled). In various implementations, the attenuation value is defined on a sliding scale indicating the amount of distance attenuation.
[0048] In various implementations, the audio session parameters also include a reverberation value indicating a reverberation characteristic, and adjusting the audio data includes adjusting the audio to have the reverberation characteristic. In various implementations, the reverberation characteristic is an amount of reverberation (which may be high, low, or neither). In various implementations, the reverberation characteristic is reverberation accuracy (which may be a full reverberation simulation of a physical environment, an amount of reverberation associated with a physical environment, or neither).
[0049] Although various aspects of specific implementations within the scope of the appended claims are described above, it should be apparent that the various features of the above-mentioned specific implementations can be embodied in a variety of forms, and any specific structures and / or functions described above are merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement a device and / or a method can be practiced. In addition, in addition to or different from one or more aspects set forth herein, other structures and / or functions can be used to implement such a device and / or such a method can be practiced.
[0050] It will also be understood that, although the terms "first", "second", etc. may be used to describe various elements in this article, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description, as long as all occurrences of the "first node" are consistently renamed and all occurrences of the "second node" are consistently renamed. Both the first node and the second node are nodes, but they are not the same node.
[0051] The terms used herein are only for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms of "one", "a kind of" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" used herein refers to and covers any and all possible combinations of one or more items in the associated listed items. It will also be understood that the term "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.
[0052] As used herein, the term “if” may be interpreted to mean “when the antecedent is true” or “when the antecedent is true” or “in response to determining” or “upon determining” or “in response to detecting” that the antecedent is true, depending on the context. Similarly, the phrase “if it is determined that [the antecedent is true]” or “if [the antecedent is true]” or “when [the antecedent is true]” is interpreted to mean “upon determining that the antecedent is true” or “in response to determining” or “upon determining” that the antecedent is true or “when detecting that the antecedent is true” or “in response to detecting” that the antecedent is true, depending on the context.
Claims
1. A method, comprising: At a device located in a physical environment, coupled to two or more speakers, and including one or more processors and non-transitory memory: Execute operating systems and applications; receiving, by the operating system via an application programming interface, audio session parameters including a spatial experience value from the application, the audio session parameters providing instructions for spatial playback of audio associated with the application; The operating system receives an instruction to play audio data from the application program; as well as adjusting, by the operating system, the audio data based on the spatial experience value; as well as The adjusted audio data is sent, by the operating system, to the two or more speakers. 2 . The method of claim 1 , wherein the spatial experience value indicates a head tracking spatial experience, and adjusting the audio data comprises adjusting the audio data to be played from a location in the physical environment.
3. The method of claim 2, wherein the audio session parameters further include an anchor value indicative of the location in the physical environment. The method of claim 3 , wherein the anchor value indicates a scene having a location in the physical environment. The method of claim 3 , wherein the anchor value indicates a forward position in the physical environment.
6. A method according to any one of claims 3 to 5, wherein the audio session parameters also include a size value indicating a size of the location in the physical environment, and adjusting the audio data includes adjusting the audio data to be played from the location having the size in the physical environment.
7. A method according to any one of claims 3 to 6, wherein the audio session parameters also include an attenuation value indicating distance attenuation, and adjusting the audio data includes adjusting the volume of the audio data based on the distance between the device and the location in the physical environment. 8 . The method of claim 1 , wherein the spatial experience value indicates a fixed spatial experience, and adjusting the audio data comprises adjusting the audio data to be played from a position relative to the device.
9. The method of claim 8, wherein the audio session parameters further include an anchor value indicating the position relative to the device.
10. The method of claim 9, wherein the audio session parameters further include a size value indicating a size relative to the position of the device, and adjusting the audio data comprises adjusting the audio data to be played from the position relative to the device having the size.
11. The method of claim 1, wherein the spatial experience value indicates a non-spatial experience, and adjusting the audio data comprises adjusting the audio data without spatialization.
12. The method according to any one of claims 1 to 11, further comprising: receiving, by the operating system, a user input, wherein the user input overwrites the spatial experience value with a user value; as well as The audio data is adjusted according to the user value.
13. A device located in a physical environment and coupled to two or more speakers, the device comprising: non-transitory memory; and One or more processors configured to: Execute operating systems and applications; receiving, by the operating system via an application programming interface, audio session parameters including a spatial experience value from the application, the audio session parameters providing instructions for spatial playback of audio associated with the application; The operating system receives an instruction to play audio data from the application program; as well as adjusting, by the operating system, the audio data based on the spatial experience value; The adjusted audio data is sent, by the operating system, to the two or more speakers.
14. The apparatus of claim 13, wherein the spatial experience value indicates a head tracking spatial experience, and adjusting the audio data comprises adjusting the audio data to play from a location in the physical environment.
15. The apparatus of claim 14, wherein the audio session parameters further comprise an anchor value indicative of the location in the physical environment. 16 . The device of claim 13 , wherein the spatial experience value indicates a fixed spatial experience, and adjusting the audio data comprises adjusting the audio data to be played from a position relative to the device.
17. The device of claim 16, wherein the audio session parameters further comprise an anchor value indicative of the position relative to the device.
18. The apparatus of claim 13, wherein the spatial experience value indicates a non-spatial experience, and adjusting the audio data comprises adjusting the audio data without spatialization.
19. The apparatus according to any one of claims 13 to 18, wherein the one or more processors are further configured to: receiving, by the operating system, a user input, the user input overwriting the spatial experience value with a user value; and The audio data is adjusted according to the user value.
20. A non-volatile memory storing one or more programs which, when executed by one or more processors of a device located in a physical environment and coupled to two or more speakers, cause the device to perform any of the methods of claims 1 to 12.
