CALIBRATION AMPLIFICATION FOR REAL-WORLD SOUND

Abstract

Computer-readable medium containing instructions that configure a computer to: present an application programming interface (API), wherein the API includes a user-specified sound level parameter; receive, via the API, one or more values ​​for the user-specified sound level parameter, which are assigned to a digital audio asset;and incorporating code into a simulated reality application that is currently being assembled, wherein the code determines a loudness correction gain for the digital audio asset by determining a difference between i) the one or more values ​​for the user-specified sound level parameter and ii) a parameter of the playback hardware device that represents a known or predefined sound level that can be produced by a playback hardware device on which the simulated reality application is running, wherein the loudness correction gain compensates for the difference, and the loudness correction gain is then to be applied to the digital audio asset during the runtime of the simulated reality application.

Description

[0001] This non-provisional patent application claims the benefit of the earlier filing date of US provisional application No. 63 / 285,263, filed on December 2, 2021.

[0002] One aspect of this disclosure relates to computer software techniques for adjusting the real-world loudness of virtual objects in a simulated reality application. Other aspects are also described. BACKGROUND

[0003] Authors of simulated reality applications, such as virtual reality and augmented reality apps, have limited programmatic control within the application development software they use to create their apps over how loud a particular virtual object, defined in the application, will sound when played back in the real world. The application could be run by various audio playback hardware devices. However, different hardware devices (e.g., laptop computers, tablet computers, headphones) exhibit varying playback sensitivities due to differences in their audio amplifiers, speaker drivers (also called transducers, e.g., a loudspeaker driver, a micro-speaker driver, a headphone speaker driver), and acoustic design.Thus, the same digital audio asset will be experienced at different playback loudness levels on different hardware devices. Another problem is that an end user of the application can manually adjust a volume control (thereby making a variable, end-user selectable, or "manual" volume setting on their hardware device), which can change the sound level and deviate from the sound level intended by the author. SUMMARY

[0004] From one perspective, a software development kit (SDK) provides an application programming interface (API) and its implementation functions, which provide software code that can be integrated into a simulated reality application. The API configures an overall calibration gain to be applied to the audio signal of a specific digital audio asset, selected by the simulated reality application's author, during runtime. The overall calibration gain includes at least a loudness correction gain, configured by the API based on a user-specified sound level, which is specified by the user (the author) through an API parameter.The desired sound level can be the level at which the author wants an end user (listener) in the simulated reality to hear the audio asset during the app's runtime, at a virtual reference distance from a virtual object to which the audio asset has been assigned. The code for determining the calibration gain is embedded into the simulated reality application (which is currently being assembled) via the SDK. The calibration gain is then automatically determined by the code and applied during runtime (app execution) with each playback of the audio asset. In this way, the audio asset's playback is gain-corrected to reflect the expected real-world sound level desired by the app's author.

[0005] From another perspective, based on the understanding that an audio asset can be associated with a virtual object and that the distance between the virtual object and the virtual listening position changes during the app's runtime, the API and its implementation functions provide software code (intended to be integrated into the simulated reality application via the SDK) that determines a relative distance gain. The relative distance gain is a function of the virtual distance during runtime (app execution), which is the distance between the virtual object and the virtual listening position. During runtime, the relative distance gain can be automatically applied to each playback of the audio asset. The function implementing the relative distance gain is defined or configured by the API based on a set of parameters input to the API.

[0006] The determination of the calibration gain and the gain based on the relative distance are transparent to the author and facilitate the task of assembling the simulated reality application, while ensuring a real-world loudness perception during playback that also meets the author's wishes.

[0007] Another advantage of the solutions described below is that they allow the author to realistically preview a 3D or physical audio mix for the assembled virtual reality scene with regard to sound, even during the simulation of the simulated reality application.

[0008] The foregoing summary does not constitute a complete list of all aspects of the present disclosure. The disclosure is intended to include all practically implementable systems and methods derived from all suitable combinations of the various aspects summarized above, as well as those disclosed in the detailed description below and expressly mentioned in the claims. Such combinations may exhibit certain advantages not specifically stated in the foregoing summary. List of characters

[0009] Various aspects of the revelation presented here are illustrated in an exemplary and non-exhaustive manner in the figures of the accompanying drawings, where identical references denote identical elements. It should be noted that references to “one” aspect in this revelation do not necessarily refer to the same aspect, and they signify at least one. Furthermore, for the sake of brevity and to reduce the total number of figures, a given figure may be used to illustrate the features of more than one aspect of the revelation, and it may not be necessary for all the elements in the figure to represent a given aspect. Fig. Figure 1 is a block diagram showing how to determine an overall calibration gain and use it to adjust the gain of an audio asset in a simulated reality application. Fig. Figure 2 illustrates an example of a relative distance-based gain model for further adjusting the gain of the audio asset in the simulated reality application. Fig. Figure 3 is a flowchart of a process for creating a simulated reality application. DETAILED DESCRIPTION

[0010] Several aspects of the disclosure will now be explained with reference to the accompanying drawings. In cases where the shapes, relative positions, and other aspects of the described parts are not clearly defined, the scope of the disclosure is not limited to the parts shown, which are included solely for illustrative purposes. While numerous details are presented, it is understood that some aspects of the disclosure can be explained without these details. In other cases, well-known circuits, structures, and techniques have not been shown in detail so as not to obscure the understanding of this description.

[0011] An author of a simulated reality application (e.g., an augmented reality application or a virtual reality application) who defines a simulated reality scene might want to control the actual loudness of what an end user or listener hears at a virtual listening position during scene playback. For example, a small bee might not be loud, but it is audible when close to the listening position and then quickly becomes quieter as it flies away. Another example: A distant sound like a waterfall might be moderately loud but change very little even as the listening position changes (because it is far away).The following are several techniques for configuring a computer system, for example, by configuring a processor of the system according to instructions stored in a computer-readable medium such as memory, to support the assembly of the simulated reality app and, in particular, to enable the author to more easily control the playback loudness of the sound sources that the author adds to the simulated reality application (as the playback will be experienced by the end user when the application is run with a given type of hardware device).

[0012] A computer used by the author is configured by application development software, for example, as part of a software development kit (SDK), which is to be executed by the computer's processor. The application development software includes an application programming interface (API) that defines how the simulated reality application being created can access specific functions within a library (which may also be part of the SDK). These functions can be configured by a user, such as the author of the simulated reality app, and then incorporated (e.g., compiled) into the simulated reality application by the SDK.

[0013] From one perspective, the API and its implementation functions provide software code that can be included in the simulated reality application and that determines an overall calibration gain to be applied during the app's runtime to the audio signal of a specific digital audio asset selected by the simulated reality app author. The overall calibration gain includes at least one contribution from a loudness correction gain. The loudness correction gain can be viewed as a correction for the sliding scale between sound pressure levels achieved by different playback hardware devices for a given digital audio signal level. The loudness correction gain is determined based on a user-specified or desired sound level provided by the author via the API.The desired sound level can be the level at which the author wants an end user (listener) of the simulated reality application to hear the audio asset during the app's runtime, at a virtual reference distance from a virtual object to which the audio asset has been assigned. The code for determining the overall calibration gain is included by the SDK in the simulated reality application (which is currently being assembled). The code then automatically determines the overall calibration gain and applies it to each playback of the audio asset during runtime (app execution). This corrects the audio asset's playback gain to reflect the expected real-world sound level specified by the app's author.

[0014] From another perspective, given that the audio asset is associated with a virtual object and that the distance between the virtual object and the virtual listening position changes during the app's runtime, the API and its implementation functions provide software code (to be included in the simulated reality application by the SDK) that determines a relative distance gain. The relative distance gain is a function of the distance between the virtual object and the virtual listening position, which changes in real time (during the app's runtime or execution). Furthermore, the relative distance gain can be automatically applied to each playback of the audio asset during runtime.The function that implements the relative distance amplification is defined or configured based on parameters entered via the API, thus simplifying its use by the author while ensuring a realistic and artifact-free attenuation of the asset's sound, especially as the virtual object moves further away from the virtual listening position.

[0015] The determination of the calibration gain and the gain based on the relative distance are transparent to the author and facilitate the task of assembling the simulated reality application, while ensuring a real-world loudness perception during playback that also meets the author's wishes. Calibration gain

[0016] One aspect of the present disclosure, which enables the author (of the Simulated Reality app) to control the loudness of a real-world sound of a virtual object, is referred to as the Overall Calibration Gain 2. Fig. Figure 1 is a block diagram showing how the overall calibration gain for a given Audio Asset 6 can be determined and then applied to the Audio Asset 6 by an audio rendering engine for playback within the simulated reality application. Some of the parameters and functions shown can be configured based on user / author selections made during app development and then incorporated into the simulated reality app. Other parameters and functions can be calculated or updated at runtime or during the simulation of the app.

[0017] The Overall Calibration Gain 2 is a function that, as shown in the figure, combines the contributions of several gains into an "overall" calibration gain. These contributions (also referred to here as gains) include a loudness correction gain, generated by a loudness correction function (Loudness Correction 3) described below, and several other gains. Inputs to the Overall Calibration Gain function (Overall Calibration Gain 2) include an asset level of a specific audio asset 6. The asset level can be that of an already normalized audio asset (where the audio asset 6 may have already been normalized or managed, as required, for example, by the author or a client program).Alternatively, the asset level can be obtained from an asset normalization function (Asset Normalization 5) which is applied to redefine an unknown audio asset (relative to a known one).

[0018] If the audio asset 6 is held at an alternative normalization assumption by an external software tool, and consequently the audio asset 6 should not be renormalized using asset normalization 5, then such a "Do not renormalize" specification can be provided as one of the other parameters or gains 8 in the audio asset 6's metadata. This parameter generally indicates whether or how the audio asset 6 has been normalized, or what assumptions were made for normalization. In this case, the SDK, or the code it injects into the simulated reality application, determines the overall calibration gain based on a specified or implied level of the audio asset, without applying asset normalization to the audio asset. This can occur in response to such a specification received via the API, or in a tag or metadata associated with the audio asset.The statement indicates that the audio asset i) has already been normalized or ii) should not be normalized again.

[0019] Asset normalization 5 can be performed by applying a static normalization gain, i.e., a single gain value, to the entire audio asset to bring it to an expected level (while also preserving the dynamic range). Alternatively, asset normalization 5 can be dynamic normalization, which may employ a loudness model where the gain applied to the audio asset 6 can change from one sample to the next (and is applied across the entire audio asset). In some cases, dynamic normalization can occur in real time, each time the audio asset 6 is played back. The overall calibration gain is then determined based on the normalized level of the audio asset produced by either dynamic or static normalization.

[0020] Contributions from other enhancements include grouping and other mixed enhancements (not shown) that are determined by the author in the case where the audio asset in question is one of several audio assets assigned to the same group.

[0021] Another aspect of the present revelation, which is also in Fig. As illustrated in Figure 2, the input to the API—in addition to or as an alternative to the user-requested sound level parameter 9—can include other parameters or gains 8 that contribute to the processing of the audio asset 6. These other parameters or gains 8 can include a parameter or description of a desired behavior for the audio asset 6. For example, the author might want the value set in Block 9 and displayed in Block 3 of Fig. 1. The level of sound used at the reference distance X is [value missing], but the overall calibration gain 2, together with the downstream gains (which are in [value missing]), is not desired. Fig. (1 to the right of block 11) results in an overall level lower than Y. Ideally, this can include a contribution (or at least knowledge) of the gain based on relative distance and, from another perspective, be viewed as a minimum audibility parameter. As another example, the API includes one or more other parameters or gains 8 that specify a loudness range in which the sound of the audio asset 6 should be perceived, or a minimum acceptable loudness, e.g., how the author wants to "tamp down" the sound of this particular audio asset or keep the asset audible by specifying a minimum SPL. For example, the API can be automatically populated by a wrapping layer or management layer that manages the inputs to the API for the user.The audio asset 6 file can be registered with various codec containers. In this case, the author only needs to specify the file, and then, when the file is decoded or the asset is registered, the SDK automatically reads a tag (for the file) containing the desired sound level and provides the API with the desired sound level extracted from the tag. The desired sound level can also be provided directly by the author via the API to enhance or replace the one in the asset's tag.

[0022] The loudness correction 3 receives at least one parameter that can be set by the author for each audio asset, for example, via the API provided by the application development software (and where the audio asset 6 can be identified as another parameter, e.g., as a pointer to the audio asset 6 in a library of audio assets). As shown in the figure, the user-desired sound level parameter 9 is specified for the audio asset 6. It includes one or more values ​​that either directly or indirectly represent a sound level that an end user (listener) should hear when the virtual listening position is at a specific reference distance, e.g., 50 centimeters or 1 meter, from a virtual sound source to which the audio content of the audio asset 6 is assigned.The user-requested sound level parameter 9 can be a sound pressure level (SPL), a perceived loudness level, or another sound level directly specified by the author of the simulated reality application (e.g., if the author knows the actual SPL that has been specified). The desired sound level can be specified as a relative value (e.g., -6 dB from a nominal output level) or as an absolute value (e.g., 75 dB SPL, defined at a virtual reference distance from a virtual sound source). From one perspective, the one or more values ​​for the user-requested sound level parameter 9 are located in a tag or metadata associated with the digital audio asset, e.g., in the same file as the audio asset.

[0023] As an example of the user-requested sound level parameter 9, which indirectly represents a desired sound level, the tag or metadata may include or reference a codec, where the codec includes one or more values, such as a desired sound level, and these values ​​may actually be unknown to the author.

[0024] One or more output (return) parameters of the API include a formula or process that defines the loudness correction 3. This formula or process can optionally include the other gains shown in the figure, which are combined to produce the result generated by the overall calibration gain 2. The formula or process is embedded as software code into the code of the simulated reality application being assembled, using the SDK. The code determines the loudness correction 3 for the audio asset 6 based on the user-requested sound level parameter 9 and a parameter 10 of the playback hardware device. From one perspective, parameter 10 of the playback hardware device represents an output signal with the highest possible or fully driven sound level that can be generated by a playback hardware device on which the simulated reality application is running.Parameter 10 of the playback hardware device is used here to correct the sliding scale between a target sound pressure level (SPL) and a playback hardware device SPL for a selected asset normalization (or a digital reference audio level). More generally, parameter 10 of the playback hardware device can represent a known or predefined sound level that can be generated by a playback hardware device (on which the simulated reality application is running) for a selected digital reference audio level. The code can determine a difference between i) one or more values ​​of the user-desired sound level parameter and ii) the sound level parameter of the playback hardware device, and the loudness correction gain 3 is selected to compensate for the difference, e.g., to be equal to the difference.

[0025] From one perspective, the code embedded in the simulated reality application receives parameter 10 of the playback hardware device during runtime or initialization time of the simulated reality application (and not during compilation). For example, the code can query the operating system when the application is launched to retrieve the type of playback hardware device on which the operating system is running. It can then use this identification to perform a table lookup and retrieve the parameter previously specified for that type of hardware device. Alternatively, the code could retrieve the parameter directly from the operating system, for example, if the hardware device is an unknown device that provides its own parameter.The parameter can include one or more gain values ​​that depend on the acoustic sensitivity of the hardware device during playback, such as the combination of its audio amplifier's gain and its speaker sensitivity. Thus, headphones, an extra-aural loudspeaker driver on a head-worn device, and a loudspeaker driver in a room will each have different playback device gains. In one case, these different device gains can all be stored in a lookup table in the code that determines the calibration gain, or they can be individually provided to the query code directly by the operating system running on a particular playback hardware device.

[0026] The overall calibration gain includes a contribution from the loudness correction gain and is automatically applied in real time (during runtime) by the simulated reality app to each playback of the audio asset, without requiring any energy analysis of the audio asset. Referring again to Fig. The loudness correction gain, along with contributions from other gains, can be incorporated into the code containing the formula or process for the overall calibration gain, as shown in the figure. The overall calibration gain can be calculated, for example, in the linear domain by multiplying all contributing gains in the linear domain.

[0027] From one perspective, the loudness correction gain API is executed only once for each audio asset during the development or creation of the simulated reality app. The formula or process that calculates the loudness correction gain is then integrated into the application, where it remains immutable throughout the app's runtime. From another perspective, the code integrated into the application allows the formula or process for calculating the loudness correction gain (for a given audio asset) to be a runtime function that can be called multiple times during the app's runtime, resulting in the loudness correction gain changing over time.

[0028] From another perspective, the additional code provided by the SDK and integrated into the simulated reality app's code determines whether a real-world distance or physical relationship between a speaker driver (transducer) of the playback hardware device and a listener's ear differs from a known or predefined distance or relationship. Based on this determination, it adjusts the overall calibration gain for the digital audio asset to compensate for this difference. This gain adjustment might involve an increase, for example, if the real-world distance is greater than the known or predefined distance, such as when a listener is farther from their laptop or tablet computer than the known or predefined distance during playback via speaker drivers.It should be noted that such real-world attenuation (which the listener experiences because he is further away from the loudspeaker driver than the author expected) depends on the given type of playback hardware device (and is not necessarily a function of the desired sound level specified by the author).

[0029] In one case, a lookup table may have been predefined in the lab and then stored as part of the code embedded in the simulated reality app. This lookup table may contain gain matching values ​​that correspond to different distance variations for various types of playback hardware. These matching values ​​may have been determined in the lab based on measured real-world attenuation. Alternatively, the matching value could be estimated or determined during runtime (by additional code embedded in the simulated reality app, for example, which receives a real-time measurement from the operating system of the actual distance or relationship between a loudspeaker driver and the listener's ear).The subsequent code can track this real distance or relationship during the app's runtime and can use this tracked parameter to make real-time adjustments to the overall calibration gain.

[0030] From another perspective regarding overall calibration enhancement, additional code is inserted into the simulated reality application to contribute to overall calibration enhancement, regardless of whether the given playback hardware device has a volume control. This additional code determines whether a volume control is present, for example, by querying the operating system at runtime or by obtaining this information during the compilation of the simulated reality app. If a volume control is present, the code can further query the playback hardware device's operating system regarding the current manual volume setting of the volume control (e.g., within a range of 0-100%). The manual volume setting can be a data structure within the hardware device's operating system.The current manual volume setting can then be used to adjust the overall calibration gain in certain cases (e.g., to increase the overall calibration gain for certain audio assets that may contain speech but not others, in response to the listener having turned the volume control down). If it is determined that the hardware device does not have a volume control, then no additional contribution (adjustment) is made to the overall calibration gain.

[0031] More generally, there can be one or more downstream gains (e.g., master volume control, volume control) applied to the audio mix generated by the audio rendering engine—that is, "downstream" the audio rendering engine after the engine has combined other audio assets (and their respective gains) into an output audio mix that can be viewed as a signal for a loudspeaker driver. As an example, consider the situation where the downstream gains in the playback hardware device are reduced. The code implementing the master calibration gain 2 can detect and respond to such a condition. For example, the code might decide to increase the master calibration gain 2 for a particular audio asset if it determines that the downstream gain has decreased by more than a certain threshold, such as...This is especially relevant if the audio asset contains speech that would become inaudible if the downstream gain were reduced too much. Knowledge of the downstream gains can be useful in other ways when adjusting the overall calibration gain 2.

[0032] Furthermore, with reference to Fig. 1. The following should be considered regarding loudness normalization, which is performed by including the normalized asset level 5 in the overall calibration gain 2: In general, a goal of a normalization scheme can be to ensure that all normalized assets have the same "effective level" when considering the distance model and user-specified levels. From one perspective, this is achieved by the application development software, for example, in response to the author specifying the normalized asset level 5. From another perspective, the audio asset 6, as retrieved from the asset library, has already been loudness normalized to a known reference level, such as -6 dBFS, -12 dB LKFS, a K-weighted level, or a frequency-weighted loudness model.If audio asset 6 is a long sound, its loudness can also be dynamically normalized. In this case, the normalized asset level 5 is a current that can change from one audio frame to the next (and thus the overall calibration gain 2 becomes a current that can change, for example, from one audio frame to the next). A long sound can also be normalized with a single value in cases where the dynamic range should be preserved during loudness considerations, but the relevant / effective loudness of the sound should be adjusted to the system. If audio asset 6 is a short sound, then audio asset level 5 is a single fixed or static value.

[0033] Fig. Figure 3 is a flowchart of a more general computer-implemented procedure for creating a simulated reality application, which may use some of the specific aspects described above. In Step 31, the (computer-executed) procedure presents an API that includes a user-specified sound level parameter, and in Step 32, it receives one or more values ​​for the parameter via the API, which are assigned to the audio asset 6. In one example, the user-specified sound level parameter is the user-desired loudness level 9, based on the assumption that the asset is heard at a reference distance. In Step 33, the procedure injects code into a simulated reality application that is currently being assembled, the code determining the loudness correction function (Loudness Correction 3) for the audio asset based on the one or more values ​​of the user-specified sound level parameter.The loudness correction gain is then applied to audio asset 6 during the runtime of the simulated reality application. Variations of this generalized procedure can be as described above and may include one or more of the following features: The code can determine the loudness correction by determining a difference between i) the one or more values ​​for the user-specified sound level parameter and ii) a parameter of the playback hardware device that represents a known or predefined sound level that can be generated by a playback hardware device on which the simulated reality application is running, with the loudness correction gain compensating for the difference; The API configures a relative distance gain for the digital audio asset, and the code integrated into the simulated reality application implements the relative distance gain, which is then applied to the audio asset during the runtime of the simulated reality application; The API has one or more other input parameters or gains that specify a loudness range in which the sound of the audio asset should be perceived, or a minimum acceptable loudness at which the sound of the audio asset should be perceived; and The API has one or more other input parameters or gains that indicate how the digital audio asset has already been normalized or processed, managed by the author or by another digital audio processing software tool.

[0034] The formula or process of the overall calibration gain 2, including a contribution from the loudness correction 3 and optionally one or more additional gains, in Fig. The function shown in Figure 1 can be integrated into the software code of the simulated reality application and therefore result in a total calibration gain value that can be quickly calculated and applied during runtime or in real time with each playback of the audio asset 6 within the simulated reality application. In this way, the author can, for example, conveniently set the desired SPL for a given audio asset via the application development software's API and then expect the sound of the audio asset to be "calibrated" or at the desired sound level during runtime or playback of the simulated reality application. If the virtual object to which the audio asset is assigned moves beyond the virtual reference distance, further adjustments are made to this sound by means of a relative distance gain 11, which can also be applied in real time.The relative distance enhancement 11 is described below in conjunction with . Fig. 2 discussed. Relative distance amplification

[0035] In one aspect of the present disclosure, the API of the application development software exhibits a relative distance enhancement 11 (see Fig. 1) a function that, in addition to the overall calibration gain 2, determines another gain value that further adjusts the audio asset (when applied by the audio rendering engine during the runtime of the simulated reality app). The relative distance gain 11 is determined for each individual audio asset and can be independent of the gains that contribute to the overall calibration gain 2. The relative distance gain controls how loud an audio asset will sound depending on the distance between the virtual object (to which the audio asset is assigned) and the virtual listening position. The function returns or outputs a gain in response to an input distance.Accordingly, the relative distance gain is a variable that is applied (by the audio rendering engine) to the audio signal of the audio asset, for example, during each playback of the audio asset while the simulated reality app is running. The code implementing the relative distance gain can be included (via the application development software or SDK) in the compiled program code of the simulated reality application being created.

[0036] The relative distance gain 11 can be determined according to the model for which in Fig. Figure 2 illustrates how this can be determined. As shown in the figure, the gain output is plotted on the y-axis and is a function of the input distance along the x-axis, i.e., the distance between a sound source (located at the position of a virtual object) and a listening position. The listening position can be the position of another virtual object, such as one representing the listener's head or ears. If the virtual object to which the sound source or audio asset is assigned moves relative to the listening position, or vice versa (the listening position moves relative to the virtual object), the relative distance gain assigned to the sound source changes accordingly to achieve a realistic and desired listening experience (as intended by the app's author).

[0037] The relative distance gain 11 can be expressed linearly or in decibels (dB) and is generally a monotonically decreasing function of increasing distance from the sound source. A user or author assembling a simulated reality application can configure the function via the application development software's API by specifying one or more configuration parameters that define the behavior of one or more sections of the function.

[0038] With renewed reference to Fig. Section 2 defines a near section of the function (the model) that covers a distance range "near" the sound source, between zero and a proximity distance. In this section, the gain can, for example, be held at a constant maximum value, as shown, while the distance decreases towards and below the proximity distance. Both the maximum value (the held value) and the proximity distance could be API configuration parameters that the user can set during the creation of the simulated reality app.

[0039] The near section is followed by a mid section in which a gain is output that decreases with distance, for example linearly or according to a user-specified envelope or attenuation function, and includes the virtual reference distance, at which the gain is, by definition, 0 dB (or a linear value of one). For distances beyond the reference distance, the gain is in the negative dB range (or a linear value less than one). Again, the attenuation behavior and, in some cases, the reference distance could be API configuration parameters that can be set by the user during program development.

[0040] Even though certain aspects have been described and shown in the accompanying drawings, it should be understood that these are merely illustrative and not limiting for the broader invention, and that the invention is not limited to the specific constructions and arrangements shown and described, since a person skilled in the art may conceive of various other modifications. For example, the details of the [unclear] can be Fig. The distance model shown in point 2, e.g., the normalization scheme or the reference distance specified at 0 dB, may differ, as they could take on any value that defines an expected relationship. The description should therefore be considered illustrative rather than restrictive. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63285263

[0001]

Claims

[1] Computer-readable medium containing instructions that configure a computer to do the following: Presenting an application programming interface (API), where the API includes a user-specified sound level parameter; Receiving, via the API, one or more values ​​for the user-specified sound level parameter, which are assigned to a digital audio asset; and embedding code into a simulated reality application that is currently being assembled, wherein the code determines a loudness correction gain or loudness correction contribution for the digital audio asset based on the one or more values ​​of the user-specified sound level parameter, and then applying the loudness correction gain to the digital audio asset during the runtime of the simulated reality application. [2] Computer-readable medium according to claim 1, wherein the instructions configure the code that is incorporated into the simulated reality application to determine the loudness correction gain by: Determining a difference between i) the one or more values ​​for the user-specified sound level parameter and ii) a parameter of the playback hardware device that represents a known or predefined sound level that can be generated by a playback hardware device on which the simulated reality application is running, wherein the loudness correction gain compensates for the difference. [3] Computer-readable medium according to claim 2, wherein the known or predefined sound level is a sound level that can be generated by the playback hardware device at a maximum possible or fully driven audio signal value in the playback hardware device. [4] Computer-readable medium according to claim 2, wherein the code which is incorporated into the simulated reality application receives the parameter of the playback hardware device during the runtime of the simulated reality application and not during the compilation of the simulated reality application. [5] Computer-readable medium according to claim 4, wherein the code obtains the parameter of the playback hardware device by querying an operating system of the playback hardware device. [6] Computer-readable medium according to claim 1, wherein the user-specified sound level parameter is to be specified by an author of the simulated reality application. [7] Computer-readable medium according to claim 1, wherein the instructions configure the computer to insert further code into the simulated reality application being assembled, wherein the further code determines an overall calibration gain to be applied to the digital audio asset during the runtime of the simulated reality application, wherein the loudness correction gain contributes to the overall calibration gain as well as one or more other gains. [8] Computer-readable medium according to claim 7, wherein the further code determines a downstream gain in a playback hardware device on which the simulated reality application is run, and the further code adjusts the overall calibration gain for the digital audio asset based on the fact that the downstream gain is reduced. [9] Computer-readable medium according to claim 7, wherein the further code determines that a real distance or real physical relationship between a loudspeaker driver of a playback hardware device and an ear of the listener differs from a known or predefined distance or relationship, and adjusts the overall calibration gain for the digital audio asset based on this determination. [10] Computer-readable medium according to claim 9, wherein the further code receives updates for the real distance or the real physical relationship during runtime. [11] Computer-readable medium according to claim 2, wherein the known or predefined sound level that can be generated by the playback hardware device and is represented by the parameter of the playback hardware device is a sound level that is generated when a loudspeaker driver of the playback hardware device is located at a known or predefined distance or in a known or predefined relationship to an ear of a listener. [12] Computer-readable medium according to claim 1, wherein the API configures a relative distance gain for the digital audio asset and the instructions configure the computer to incorporate the relative distance gain into the simulated reality application, wherein the relative distance gain is then applied to the audio asset during the runtime of the simulated reality application. [13] Computer-implemented method for creating a simulated reality application, wherein the method comprises: Presenting an application programming interface (API) wherein the API includes a user-specified sound level parameter; Receiving, via the API, one or more values ​​for the user-specified sound level parameter, which are assigned to a digital audio asset; and embedding code into a simulated reality application that is currently being assembled, wherein the code determines a loudness correction gain for the digital audio asset based on the one or more values ​​of the user-specified sound level parameter, wherein the loudness correction gain is then to be applied to the digital audio asset during the runtime of the simulated reality application. [14] Method according to claim 13, wherein the code which is incorporated into the simulated reality application serves to determine the loudness correction gain by: Determining a difference between i) the one or more values ​​for the user-specified sound level parameter and ii) a parameter of the playback hardware device that represents a known or predefined sound level that can be generated by a playback hardware device on which the simulated reality application is running, wherein the loudness correction gain compensates for the difference. [15] Method according to claim 14, wherein the code which is incorporated into the simulated reality application receives the parameter of the playback hardware device during the runtime of the simulated reality application and not during the compilation of the simulated reality application. [16] Method according to claim 13, wherein the user-specified sound level parameter is a sound level specified by an author of the simulated reality application. [17] Method according to claim 13, wherein the code determines a current manual volume setting on a playback hardware device on which the simulated reality application is running. [18] Method according to claim 13, wherein the API configures a relative distance gain for the digital audio asset and the method further comprises: Integrate, into the simulated reality application, code that implements the relative distance amplification, which is then applied to the audio asset during the runtime of the simulated reality application. [19] Method according to claim 13, wherein the one or more values ​​for the user-specified sound level parameter are present in a tag or in metadata associated with the digital audio asset. [20] Method according to claim 19, wherein the tag or metadata includes or refers to a codec that includes the one or more values ​​for the user-specified sound level parameter. [21] Method according to claim 13, wherein the code, when integrated into the simulated reality application, performs one, but not both, of the following actions: Not applying asset normalization to the digital audio asset in response to a statement received via the API or in a tag or metadata associated with the digital audio asset indicating that the digital audio asset i) has already been normalized or ii) should not be renormalized, and determining an overall calibration gain based on a specified or implied level of the digital audio asset; or Applying dynamic or static normalization to the digital audio asset and determining the overall calibration gain based on a normalized level of the digital audio asset generated by the dynamic or static normalization. [22] Method according to claim 13, wherein the API comprises one or more other parameters or gains that specify a loudness range in which the sound of the audio asset should be perceived, or a minimum acceptable loudness at which the sound of the audio asset should be perceived. [23] Method according to claim 13, wherein the API comprises one or more other parameters or gains that indicate how the digital audio asset has already been normalized or processed, by management of the author or by another digital audio processing software tool. [24] Electronic hardware device comprising: a processor; and Memory comprising a simulated reality application which, when executed by the processor, determines a loudness correction gain or loudness correction contribution for a digital audio asset based on one or more values ​​of a user-specified sound level parameter and applies the loudness correction gain to the digital audio asset during the runtime of the simulated reality application. [25] Device according to claim 24, wherein the processor is configured by the simulated reality application to determine the loudness correction gain by: Determining a difference between i) the one or more values ​​for the user-specified sound level parameter and ii) a parameter of the playback hardware device representing a known or predefined sound level that can be generated by the electronic hardware device, wherein the loudness correction gain compensates for the difference. [26] Device according to claim 25, wherein the processor is configured by the simulated reality application to obtain the playback hardware device parameter during the runtime of the simulated reality application. [27] Completing computer system: a processor; and computer-readable medium containing instructions that configure the processor to do the following: Presenting an application programming interface (API), where the API includes a user-specified sound level parameter; Receiving, via the API, one or more values ​​for the user-specified sound level parameter, which are assigned to a digital audio asset; and embedding code into a simulated reality application that is currently being assembled, wherein the code determines a loudness correction gain or loudness correction contribution for the digital audio asset based on the one or more values ​​of the user-specified sound level parameter, and then applying the loudness correction gain to the digital audio asset during the runtime of the simulated reality application.

Images