Derivation of parameters of the reverberation processor

Abstract

The method (300) is performed by an audio renderer (151). The method includes obtaining (s302) metadata for an extended reality scene. The method also includes obtaining or deriving (s304) a first reverberation parameter from the metadata, the first reverberation parameter being a reverberation time parameter or a reverberation level parameter. The method also includes deriving (s306) a second reverberation parameter using the first reverberation parameter. If the first reverberation parameter is the reverberation time parameter, the second reverberation parameter is a reverberation level parameter, and if the first reverberation parameter is the reverberation level parameter, the second reverberation parameter is a reverberation time parameter.

Description

Technical Field

[0001] Embodiments are disclosed for deriving parameters of a reverberation processor.

Background Art

[0002] Extended Reality (XR) (e.g., Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), etc.) systems generally include an audio renderer for rendering audio to a user of the XR system. The audio renderer typically includes a reverberation processor that generates late reverberation and / or diffuse reverberation to be rendered to the user of the XR system to provide the auditory sensation of being in the rendered XR scene. The generated reverberation should provide the user with the auditory sensation of being in an acoustic environment (e.g., a church, a living room, a gym, an outdoor environment, etc.) corresponding to the XR scene.

[0003] Reverberation is one of the most important acoustic characteristics in an indoor environment. Sound generated indoors repeatedly bounces off reflective surfaces such as the floor, walls, ceiling, windows, tables, etc., while gradually losing energy. When these reflections mix with each other, a phenomenon known as "reverberation" occurs. Thus, reverberation is a collection of many reflections of sound.

[0004] Two of the most fundamental characteristics of reverberation in an acoustic environment are: 1) the reverberation time, and 2) the reverberation level, i.e., how strong or large the reverberation is (e.g., relative to the output of a sound source or the direct sound level within a space). Both of these are characteristics only of the acoustic environment, i.e., they do not depend on individual sound sources.

[0005] Reverberation time is a measure of the time required for the reflected sound to "fade out" within an enclosed space after the sound source has stopped. It is important to define how the interior responds acoustically. Reverberation time depends on the amount of acoustic absorption within the space and is lower in spaces with many absorptive surfaces such as curtains, padded chairs, and even people, and higher in spaces that mainly contain hard, reflective surfaces.

[0006] Conventionally, reverberation time has been defined as the time required for the sound pressure level to decrease by 60 dB after the sound source has been suddenly turned off. The abbreviated form of this time is "RT60" (or sometimes T60).

[0007] Typically, in the case of a reverb processor used in an audio renderer, these two (and other) characteristics of the generated reverb can be controlled individually and independently. For example, it is typically possible to configure the reverb processor to generate reverb with a specific desired reverberation time and a specific desired reverberation level.

[0008] In an XR system, the characteristics of the generated reverb are typically controlled by dedicated metadata included in an XR scene description, such as that specified by a scene creator, which describes many aspects of the XR scene, including control information such as the acoustic characteristics of the XR scene. The audio renderer receives this control information, for example, from a bitstream or a file, and uses this control information to configure the reverb processor to generate reverb with the desired characteristics. The exact way in which the reverb processor obtains the desired reverberation time and reverberation level in the generated reverb can vary depending on the type of reverb algorithm that the reverb processor uses to generate the reverb. Summary of the Invention Problems to be Solved by the Invention

[0009] Currently, there are several issues. For example, as described above, it is typically possible to control various characteristics of the generated reverberation (e.g., reverberation time and reverberation level) individually and independently of each other, providing great flexibility when generating reverberation while also bringing potential problems. In practice, the XR control information received by the audio renderer may not include control data for all characteristics of the generated reverberation that can be controlled. There can be many reasons for this. For example, the authoring software used to create the XR scene can generate only a limited set of acoustic characteristics for the acoustic environment. Or, the scene may only correspond to a real location (e.g., a specific famous church) where only a limited set of acoustic data is available. When the XR scene corresponds to the user's real physical space In the context of AR , the acoustic characteristics of that space usually need to be determined on-site using limited technical means available in the user's XR device.

[0010] As described above, the two most important characteristics of the generated reverberation are generally the reverberation time, expressed in terms of RT60, and the reverberation level, generally expressed as the ratio of reverberant sound to direct sound (RDR) energy. If neither the reverberation time nor the reverberation level is specified in the control information provided by the audio renderer, it is unclear how the reverberation processor should be configured.

[0011] From the perspective of the XR audio standard specification, even if the standard specification generally supports the specification of many reverberation parameters for the acoustic environment, only some of them may be essential for providing the XR scene, while others are optional. For example, in the currently developed ISO / IEC MPEG-I Immersive Audio standard specification, the RT60 value is the only essential reverberation-related parameter for the acoustic environment, and the reverberation level parameter (e.g., expressed as the RDR energy ratio) is optional.

[0012] Accordingly, what is needed is a solution for configuring the reverb processor of an XR audio renderer such that, when neither the reverberation time nor the reverberation level is specified for the rendered acoustic environment, a reverb signal with acoustically appropriate characteristics is generated for the XR scene.

Means for Solving the Problem

[0013] Accordingly, in one aspect, a method executed by an audio renderer is provided. In one embodiment, the method executed by the audio renderer includes the step of obtaining (e.g., receiving or retrieving) the metadata of the XR scene. The method also includes the step of obtaining or deriving a first reverb parameter from the metadata, where the first reverb parameter is a reverberation time parameter or a reverberation level parameter. The method also includes the step of deriving a second reverb parameter using the first reverb parameter. When the first reverb parameter is a reverberation time parameter, the second reverb parameter is a reverberation level parameter, and when the first reverb parameter is a reverberation level parameter, the second reverb parameter is a reverberation time parameter.

[0014] In one embodiment, the method executed by the audio renderer includes the step of obtaining a set of reverb parameters including at least a first reverb parameter and a second reverb parameter from the metadata of the extended reality scene. The method also includes the step of determining whether the first reverb parameter matches the second reverb parameter. The determining step includes the step of calculating a first value using the second reverb parameter and the step of comparing the difference between the first value and the first reverb parameter with a threshold value.

[0015] In another aspect, there is provided a computer program including instructions that, when executed by a processing circuit of an audio renderer, cause the audio renderer to execute any of the above-described methods. In one embodiment, a carrier including the computer program is provided, and the carrier is one of an electrical signal, an optical signal, a wireless signal, and a computer-readable storage medium. In another aspect, there is provided a rendering device configured to execute any of the above-described methods. The rendering device may include a memory and a processing circuit coupled to the memory.

[0016] The advantage of the embodiments disclosed herein is that the audio renderer enables both the reverberation time value and the reverberation level value to be provided to a reverberation processor (which may be part of the audio renderer itself or external thereto), thereby enabling the reverberation processor to generate an appropriate reverberation signal for the XR scene.

Brief Description of the Drawings

[0017] The accompanying drawings, which are incorporated herein and form a part of this specification, illustrate various embodiments.

[0018]

Figure 1A

[0019]

Figure 1B

[0020]

Figure 2

[0021]

Figure 3A

[0022]

Figure 3B

[0023]

Figure 4

[0024]

Figure 5

[0025]

Figure 6

DETAILED DESCRIPTION OF THE INVENTION

[0026] FIG. 1A shows an XR system 100 to which the embodiments disclosed herein may be applied. The XR system 100 includes speakers 104 and 105 (which may be the speakers of headphones worn by a user), and an XR device 110. The XR device 110 includes a display for displaying images to the user, and in some embodiments, may be configured to be worn by a listener. In the illustrated XR system 100, the XR device 110 has a display and is designed to be worn on the user's head, and is generally referred to as a head-mounted display (HMD).

[0027] As shown in FIG. 1B, the XR device 110 includes an orientation detection unit 101, a position detection unit 102, and a processing unit 103, and may be (directly or indirectly) coupled to an audio renderer 151 for generating output audio signals (e.g., a left audio signal 181 for the left speaker and a right audio signal 182 for the right speaker as shown).

[0028] The azimuth detection unit 101 is configured to detect changes in the azimuth of the listener and provide information regarding the detected changes to the processing unit 103. In some embodiments, the processing unit 103 determines the absolute azimuth (with respect to some coordinate systems) in consideration of the detected changes in the azimuth detected by the azimuth detection unit 101. Also, there may be different systems for determining azimuth and position, such as a system using a lighthouse tracker (rider), for example. In one embodiment, the azimuth detection unit 101 can determine the absolute azimuth (in relation to a certain coordinate system) when given the detected change in azimuth. In this case, the processing unit 103 can simply multiplex the absolute azimuth data from the azimuth detection unit 101 and the position data from the position detection unit 102. In some embodiments, the azimuth detection unit 101 may include one or more accelerometers and / or one or more gyroscopes.

[0029] The audio renderer 151 generates an audio output signal based on the input audio signal 161, the metadata 162 regarding the XR scene experienced by the listener, and the information 163 regarding the position and azimuth of the listener. The metadata 162 for the XR scene includes the metadata for each object and audio element included in the XR scene, and the metadata for the object may include information about the dimensions of the object and the occlusion factor for the object (for example, the metadata may specify a set of occlusion factors where each occlusion factor is applicable to a different frequency or frequency range). The metadata 162 may also include at least any one of control information such as reverberation time values, reverberation level values, absorption parameters, and the like.

[0030] The audio renderer 151 may be a component of the XR device 110 or may be remote from the XR device 110 (for example, the audio renderer 151 or its components may be implemented in the cloud).

[0031] FIG. 2 shows an exemplary implementation of an audio renderer 151 for generating sound for an XR scene. The audio renderer 151 includes a controller 201 and an audio signal generator 202 for generating an output audio signal (e.g., an audio signal of multi-channel audio elements) based on control information 210 from the controller 201 and input audio 161. In this embodiment, the audio signal generator 202 includes a reverberation processor 204 for generating a reverberation signal.

[0032] In some embodiments, the controller 201 may be configured to receive one or more parameters and trigger the audio signal generator 202 to perform modifications on the audio signal 161 based on the received parameters (e.g., increase or decrease the volume level). The received parameters include information 163 regarding the position and / or orientation of the listener (e.g., direction and distance to the audio element) and metadata 162 regarding the XR scene. For example, the metadata 162 may include metadata regarding the XR space in which the user is virtually located (e.g., dimensions of the space, information regarding objects in the space, and information regarding the acoustic characteristics of the space), as well as metadata regarding the audio element and metadata regarding objects that shield the audio element. In some embodiments, the controller 201 itself generates at least a portion of the metadata 162. For example, the controller 201 may receive metadata regarding the XR scene and derive additional metadata (e.g., control parameters) based on the received metadata. For example, the controller 201 can calculate one or more gain factors (g) for the audio elements in the XR scene using the metadata 162 and the position / orientation information 163.

[0033] Regarding the generation of the reverberation signal used by the signal generator 202 to generate the final output signal, the controller 201 provides reverberation parameters such as the reverberation time and the reverberation level to the reverberation processor 204 so that the reverberation processor 204 can be operable to generate the reverberation signal. The reverberation time of the generated reverberation is most commonly provided to the reverberation processor 204 as an RT60 value, but other reverberation time scales also exist and can be used similarly. In some embodiments, the metadata 162 includes some or all of the necessary reverberation parameters (e.g., RT60 value and reverberation level value). However, in embodiments where the metadata does not include the reverberation time parameter (i.e., the RT value such as the RT60 value) or the reverberation level parameter (i.e., the RL value such as the RDR energy ratio), the controller 201 is configured to generate these parameters. For example, as described herein, the controller 201 can generate the reverberation time parameter based on the reverberation level parameter, and vice versa.

[0034] The reverberation level can be expressed in various formats and provided to the reverberation processor 204. For example, it can be expressed as the energy ratio of the direct sound component to the reverberant sound component (DRR) at a certain distance from the sound source being rendered in the XR environment, or vice versa (i.e., the RDR energy ratio). Alternatively, the reverberation level may be expressed as the energy ratio of the reverberant sound of the sound source to the total radiated energy. In still other cases, the reverberation level may be directly expressed as the level / gain of the reverberation processor.

[0035] In this context, the term "reverberation" may typically refer only to the sound field component corresponding to the diffuse part of the acoustic room impulse response of the acoustic environment, but in some embodiments, it may also include the sound field component corresponding to the previous part of the room impulse response, for example, some late non-diffuse reflections, or even all of the reflected sounds.

[0036] Other metadata that can describe the reverberation-related characteristics of the acoustic environment included in the metadata 162 are parameters that describe the acoustic characteristics of the materials of the surfaces of the environment (for example, those that describe at least any one of the absorption, reflection, transmission, and diffusion characteristics of the materials), or specific time points of the room impulse response related to the acoustic environment, for example, the time after the sound source radiation when the room impulse response diffuses (sometimes called "pre-delay").

[0037] All of the above-mentioned reverberation-related characteristics typically depend on frequency, and therefore, their related metadata parameters are also typically provided and processed separately for several frequency bands.

[0038] Virtual reality When authoring a sound scene, in principle, it is possible to specify the reverberation time and the reverberation level individually and independently for a virtual acoustic environment. However, in an actual acoustic environment, the reverberation time and the reverberation level are not independent characteristics. There is not a one-to-one relationship between the two, and although it is not completely accurate in all cases, if only information about the reverberation time is available, it is possible to derive at least one reasonable estimate for the reverberation level, and vice versa, and it is possible to derive the relationship between them.

[0039] One derivation of such a relationship starts from the definition of the "critical distance (CD)", which is the distance (in meters) at which the sound pressure levels of the direct sound field and the reverberant sound field are equal. Assuming that the reverberant sound field is completely diffuse, the CD can be quantified as follows.

Number

[0040] Use the well-known statistical approximation formula of Sabine for RT60.

Number

Equation

[0041] Therefore, for a specific sound source directivity type (for example, an omnidirectional sound source with γ = 1), the critical distance CD is purely a characteristic of the acoustic environment.

[0042] The reverberation level of the acoustic environment can be represented by the ratio of the energy of the reverberant sound to the direct sound at a distance d from an omnidirectional point sound source (i.e., the RDR energy ratio). In that case, there is a simple relationship between the RDR energy ratio (denoted as RDR in the formula) and the critical distance (denoted as CD in the formula).

Equation

[0043] This relationship occurs because the energy of the direct sound of an omnidirectional point sound source changes with the square of the distance, and the RDR energy ratio should be equal to 1 at the critical distance.

[0044] Combining equations (3) and (4) gives an approximate relationship between the RDR energy ratio and RT60.

Equation

Equation

[0045] Equation (6) shows that the estimated value of the RDR energy ratio is obtained from RT60 and the volume V of the acoustic environment, and that the approximate relationship between the RDR energy ratio and RT60 is very simple and linear.

[0046] Similarly, Equation (6) also enables the estimation of RT60 from a known value of the RDR energy ratio.

[0047] Combining Equation (1) and Equation (4), the approximate equation for the RDR energy ratio regarding the acoustic absorption in the acoustic environment is obtained as follows.

Number

[0048] The equivalent absorption area A of the acoustic environment may be directly provided in the scene metadata, or may be derived from other parameters included in the scene metadata, for example, from the specifications of the materials or material properties (such as absorption coefficients) specified for the individual parts (such as individual walls, floors, ceilings, etc.) of the acoustic environment.

[0049] The derived equation above enables the controller 201 to configure the reverberation processor 204 when neither or both of the reverberation time or reverberation level are specified for the acoustic environment in which the reverberation signal having acoustically appropriate characteristics is generated for the scene.

[0050] As described above, the exact method for obtaining the desired reverberation time and reverberation level in the reverberation generated by the reverberation processor 204 may vary depending on the type of reverberation algorithm used by the reverberation processor to generate the reverberation. General examples of such algorithms include the feedback delay network (FDN) (which simulates reverberation processing using delay lines, filters, and feedback connections) and the convolution algorithm (which convolves the dry input signal with a measured, approximated, or simulated room impulse response (RIR)).

[0051] As an example, in the case of an FDN-based reverb processor, a desired reverberation time can be obtained by controlling the amount of feedback used. In the case of a convolution-based reverb processor, the desired reverberation time can be obtained either by loading a specific RIR having that reverberation time or by adapting the effective length of a general-purpose RIR (e.g., by filtering and time-windowing a general-purpose RIR).

[0052] For both FDN-based and convolution-based reverb processors, the reverberation level can be controlled by applying an appropriate gain either to the input signal entering the reverb processor, the output of the reverb processor, or internally within the reverb processor (e.g., by applying an overall gain to the FDN structure or RIR respectively).

[0053] Examples of how this gain can be set to obtain a desired reverberation level (e.g., a desired RDR energy ratio) expressed as the RDR energy ratio at 1 meter from an omnidirectional point source are described, for example, in U.S. Provisional Patent Application No. 63 / 217,076 filed on June 30, 2021 and International Patent Application PCT / EP2022 / 068015 filed on June 30, 2022 (both are incorporated herein by this reference). The renderer performs a calibration procedure to adjust the gain of the reverb processor such that the rendered direct sound and the reverberation component for the omnidirectional point source have the desired energy ratio at a distance of 1 meter from the source.

[0054] The renderer then generates an output signal for the user by combining (e.g., adding) the generated reverberation signal with other signal components for the source, such as the direct sound component and the early reflection component (both generated in other parts of the renderer).

[0055] As described above, the relationship between the RT60, room geometry, and the RDR energy ratio or its inverse, used above to derive the RDR energy ratio from the RT60, is an approximation that assumes a diffuse reverberant sound field. This assumption is not usually completely valid in an actual acoustic space, and the more the actual sound field deviates from a completely diffuse field, the less accurate the derived relationship becomes. However, although the assumption of a diffuse field is not usually completely valid, using the relationship derived when generating reverberation for a given virtual acoustic space usually results in perceptually reasonable reverberation for that space.

[0056] Typically, the deviation from the assumption of a diffuse field is greater in rooms with a smaller deviation from the assumption of a diffuse field and in rooms with a high absorption amount. Therefore, in smaller and highly absorptive rooms, the relationship derived above does not predict the actual relationship between the reverberation time and the reverberation level very accurately. This may not be a problem when rendering the acoustics of a virtual space. This is because, as described above, the results from using the relationship are typically still reasonable and there is no real-world reference for comparison. However, in an augmented reality (AR) use case where a virtual sound source is rendered to appear to be in the same physical space as the user, it is desirable to make the perceptual match between the reverberation of the real physical space and the generated reverberation as close as possible. In that case (and other cases where an optimal match between the actual reverberation and the generated reverberation is desired), it is possible to increase the accuracy of the derived relationship by adding the geometric shape of the room (e.g., the volume of the room, one or more room dimensions, the ratio between the maximum and minimum dimensions, etc.), the RT60, and / or the absorption characteristics of the acoustic environment (if available), and / or a frequency-dependent correction factor. For example, Equation (6) can be extended as follows.

Equation

[0057] Optionally, Equation (6) may be extended by expressing the RDR energy ratio as a power of the ratio of RT60 to V. That is, it is represented by the following equation. [Number] Here, C2 is a second correction factor having a value of 1 for a fully diffused room and may depend on any of the variables described above for the correction factor C.

[0058] As a further example, the RDR energy ratio is represented by the following equation. [Number] However, f1 is a first correction parameter and f2 is a second correction parameter. For example, f1 is equal to 3.1x10 2 , or ((3.1x10 2 )xd 2 ), or (cx(3.1x10 2 )) or (Cx(3.1x10 2 )xd 2 ), and f2 is equal to C2.

[0059] In a further embodiment, Equation (6) can be generalized as follows to express that the RDR energy ratio is a function of the ratio of RT60 to V. [Number] Here, f() represents a function.

[0060] In a further embodiment, to represent that the RDR energy ratio is a function of RT60 and V, i.e., with h() as a function, RDR = h(RT60, V) (Equation 10a), or that it is a function of RT60, i.e., with j() as a function, RDR = j(RT60) (Equation 10b), Equation (6) can be further generalized.

[0061] In addition to correcting the relationships between various reverberation parameters when the reverberant sound field is not fully diffused, the correction factors C and C2 in Equations 8 and 9 (as well as the correction parameters f1 and f2 in Equation 9a and the functional relationships in Equations (10), (10a), and (10b)) can correct the derived relationships for other factors as well.

[0062] One example is when the renderer (implicitly) uses a definition of the RDR energy ratio (or convention for measurement) that is different (in one or more respects) from the definitions assumed in the derivation of the above Equations (1) - (7).

[0063] Specifically, in the derivation of the above Equations (1) - (7) assuming a fully diffused reverberant field, in a theoretical diffused field, the room impulse response starts to diffuse from the start (i.e., immediately after the direct sound is emitted by the sound source), so it is implicitly assumed that the energy of the reverberant field used to calculate the RDR energy ratio is determined over the entire length of the room impulse response.

[0064] On the other hand, a particular renderer may instead (implicitly) use a slightly different definition of the RDR energy ratio, and the reverberant energy component of the RDR energy ratio includes only the energy contained in a portion of the room impulse response starting from a specific instant indicated by the value t1.

[0065] One reason for this design choice is that in the real world space, after the direct sound is emitted by the sound source, the reverberant field actually starts to spread for a certain amount of time. This amount of time can depend on various factors such as the geometric shape of the room, for example, its volume, one or more of its dimensions (e.g., the longest), or the ratio of its dimensions, as well as acoustic parameters such as the absorption and RT60. The definition of the RDR energy ratio that considers only the reverberant energy after the time identified by t1 can be used to reflect this physical reality. Another reason is that the output response of the reverberation processor, which is part of the renderer itself (or used by the renderer itself), starts to spread after a while after feeding the direct sound signal to the reverberation processor. Therefore, for any of these reasons or other reasons, the renderer can use the definition of the RDR energy ratio in which only the energy after a certain time is included in the reverberant energy component of the RDR energy ratio.

[0066] As a result of this choice, the value resulting as the RDR energy ratio is smaller than both the value predicted from the above equations (1)-(7) and the value obtained when the reverberant energy of the complete room response is included in the reverberant energy component of the RDR energy ratio (i.e., t1 = 0).

[0067] Another example is, for example, in the real world space, due to the fact that, as described above, the reverberant field starts to spread for a certain amount of time after the radiation by the sound source, the case where the renderer starts to render the reverberation for a certain time t1 after the radiation of the direct sound by the sound source. This has the same effect on the value of the RDR energy ratio explained in the above example.

[0068] Equation (6) can be modified to include the effect of including only the reverberant energy from a certain time identified after value t1 in the reverberant energy component of the RDR energy ratio. As an example of this, by looking at the energy decay curve of a fully diffuse field and including only the reverberant energy after the time identified by t1, the amount of "missed" energy can be determined. On a logarithmic (dB) scale, the energy decay curve of a fully diffuse field is a straight line with a slope of -60 / RT60 (dB / s) (see Figure 5). This means that if the part of the diffuse response before time t1 is excluded, this leads to a reduction in the calculated reverberant energy compared to using the full length of the following diffuse response.

Number

Number

[0069] In a use case where the received RDR value is determined using (or implicitly assuming) a specific start time t2 of the reverberant energy component that is different from the start time t1 implicitly used by the renderer itself, essentially the same correction method as above can be used to correct the RDR energy ratio value (or simply the "RDR value") received by the renderer. In this case, the RDR value according to the renderer's definition is derived by correcting the received RDR value by the correction factor of Equation (11). However, t1 is replaced by (t1 - t2) as follows (see Figure 6).

Number

Equation

[0070] When the time parameter t2 of the received RDR value is greater than the renderer's own time parameter t1, the result of the correction is that the received RDR value is increased, and when t2 is less than t1, it is decreased.

[0071] The start time t2 corresponding to the received RDR value can be received by the renderer as additional metadata for the XR scene, or it can be obtained implicitly in any other way, for example, from the fact that it is known that the received RDR value was determined according to a certain definition (e.g., because the XR scene is in a specific known, e.g., standardized format). As an example, in the MPEG-I Immersive Audio Encoder Input Format (ISO / IEC JTC1 / SC29 / WG6, Document Number N0083, "MPEG-I Immersive Audio CfP Supplemental Information, Recommendations and Clarifications, Version 1", July 2021), it is stipulated that t2 is equal to 4 times the acoustic flight time related to the longest dimension of the acoustic environment.

[0072] The reverberation time (e.g., RT60) and the reverberation level (e.g., the RDR value) typically depend on frequency, and thus are specified for various frequency bands. This means that it should be understood that all the above equations and processing steps may be evaluated and executed respectively for different frequency bands.

[0073] The above equation was derived for the RDR energy ratio expressed on a linear energy scale. However, the RDR energy ratio is equally well represented on a logarithmic (dB) scale, and an equivalent logarithmic version of the equation can be easily derived.

[0074] Specifically, the logarithmic version of Equation (6) is expressed by the following equation

Equation

Equation

Equation

[0075] In addition to providing a solution for configuring the reverberation processor when neither or both of the reverberation time or reverberation level are specified for the acoustic environment of the XR scene, the derived equations also make it possible to check whether the provided values are consistent with each other when at least two of the reverberation time, reverberation level, and absorption information are provided. Of course, as mentioned above, the derived relationships are only approximate, so strict consistency cannot be expected from using them, but at least provide a means to perform a "sanity check" on the provided data, i.e., to check whether the combination of their values is reasonable. (Note that "reasonableness" here means occurring in a real-world acoustic environment, but of course there is no reason why a virtual environment cannot have acoustic characteristics that do not exist in the real world).

[0076] The audio renderer can use such checks in many ways. In one embodiment, the renderer uses the derived formula to check the provided parameters for mutual consistency, and if the consistency is worse than a threshold, it can reject the value of at least one of the parameters and replace it with a value derived from the formula provided above. If two of the three parameters (reverberation time, reverberation level, and absorption information) are consistent and one is not, it is possible to derive from the inconsistent formula a formula that is the inconsistent one and whose value can be replaced. If only two parameters are provided, or if all three are provided and they are all mutually contradictory, hierarchical rules can be used to determine which parameter should be replaced. For example, there is a hierarchy where the reverberation time is the highest, the reverberation level is the second, and the absorption information is the third. As a result, for example, if the reverberation time and the reverberation level are provided and found to be inconsistent, the value of the reverberation level is rejected and replaced, while the value of the reverberation time is maintained.

[0077] Figure 3A is a flowchart showing a process 300 according to some embodiments. The process 300 can start from step s302. Step s302 includes obtaining the metadata of the extended reality scene. Step s304 includes obtaining from the metadata or deriving a first reverberation parameter from the metadata, where the first reverberation parameter is a reverberation time (RT) parameter (e.g., RT60) or a reverberation level (RL) parameter (e.g., RDR value). Step s306 includes deriving a second reverberation parameter using the first reverberation parameter. If the first reverberation parameter is a reverberation time parameter, the second reverberation parameter is a reverberation level parameter, and if the first reverberation parameter is a reverberation level parameter, the second reverberation parameter is a reverberation time parameter.

[0078] In some embodiments, the metadata includes an acoustic absorption parameter (denoted as "A") indicating the amount of acoustic absorption, and the first reverberation parameter is derived using the acoustic absorption parameter. In some embodiments, the first reverberation parameter is an RDR value, and deriving the RDR value includes calculating RDR = Y / A, where Y is a predetermined constant. In one embodiment, Y = 16×π.

[0079] In some embodiments, the first reverberation parameter is a reverberation time parameter (RT) (e.g., RT60), and deriving the second reverberation parameter includes calculating X×RT or RT / X, where X is a number. In some embodiments, the first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume, and deriving the second reverberation parameter includes calculating f1x(RT / V) f2 including calculating. In some embodiments, deriving the second reverberation parameter includes calculating f(RT / V) using a function f(). In some embodiments, deriving the second reverberation parameter includes calculating h(RT,V) using a function h(). In some embodiments, deriving the second reverberation parameter includes calculating j(RT) using a function j().

[0080] In some embodiments, the first reverberation parameter is a reverberation level parameter (RL) (e.g., an RDR value), and deriving the second reverberation parameter (i.e., the reverberation time parameter) includes calculating X×RL or RL / X when X is a number. In some embodiments, the first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume, and deriving the second reverberation parameter includes i) V×RL / f1, or ii) V×(RL / f1) 1 / f2 including calculating. In some embodiments, deriving the second reverberation parameter includes calculating V×g(RL) using a function g(). The function g() is the reciprocal of the function f(), i.e., g() = f -1It may be (the case). In some embodiments, deriving the second reverberation parameter includes calculating k(RL, V) using the function k(). The function k() may be the inverse of the function h(). In some embodiments, deriving the second reverberation parameter includes calculating l(RL) using the function l(). The function l() may be the inverse of the function j().

[0081] In some embodiments, the processing also includes generating a reverberation signal using the first reverberation parameter and the second reverberation parameter, and generating an output audio signal using the reverberation signal.

[0082] FIG. 3B is a flowchart showing a process 350 according to some embodiments. The process 350 can start from step s352. Step s352 includes obtaining a set of reverberation parameters including at least the first reverberation parameter and the second reverberation parameter from the metadata of the extended reality scene. Step s354 includes determining (s354) whether the first reverberation parameter matches the second reverberation parameter. The determination includes calculating a first value using the second reverberation parameter (step s356) and comparing the difference between the first value and the first reverberation parameter with a threshold (step s358).

[0083] In some embodiments, the processing also includes generating a reverberation signal using the first value instead of the first reverberation parameter as a result of determining that the difference exceeds the threshold.

[0084] In some embodiments, i) the first reverberation parameter is a reverberation level parameter, and the second reverberation parameter is either a reverberation time parameter or an absorption parameter A, ii) the first reverberation parameter is a reverberation time parameter, and the second reverberation parameter is either a reverberation level parameter or an absorption parameter A, or iii) the first reverberation parameter is an absorption parameter, and the second reverberation parameter is either a reverberation level parameter or a reverberation time parameter.

[0085] In some embodiments, the set of reverberation parameters further includes a third reverberation parameter, and the process further includes determining whether the first reverberation parameter matches the third reverberation parameter as a result of determining that the first reverberation parameter does not match the second reverberation parameter. Determining whether the first reverberation parameter matches the third reverberation parameter includes: i) calculating a second value using the third reverberation parameter; and ii) comparing the difference between the second value and the first reverberation parameter with a threshold. In some embodiments, the process further includes generating a reverberation signal using either a first value or a second value instead of the first reverberation parameter as a result of determining that the first reverberation parameter does not match either the second reverberation parameter or the third reverberation parameter.

[0086] Figure 4 is a block diagram of an audio rendering device 400 according to some embodiments for implementing the method disclosed herein (e.g., the audio renderer 151 can be implemented using the audio rendering device 400). As shown in Figure 4, the audio rendering device 400 includes a processing circuit (PC) 402. The processing circuit (PC) 402 can include one or more processors (P) 455 (e.g., one or more other processors such as a general-purpose microprocessor and / or an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc.). The plurality of processors (P) 455 may be co-located within a single housing or within a single data center, or may be geographically distributed (i.e., the device 400 can be a distributed computing device). The audio rendering device 400 includes a network interface 448. The network interface 448 includes a transmitter (Tx) 445 and a receiver 447 to enable the device 400 to send and receive data with other nodes connected to a network 110 (e.g., an Internet Protocol (IP) network) to which the network interface 448 is (directly or indirectly) connected (e.g., the network interface 448 is connected to an antenna device). The audio rendering device 400 includes a storage unit (also referred to as a "data storage system") 408 that can include at least either one or more non-volatile storage devices and one or more volatile storage devices. In embodiments where the PC 402 includes a programmable processor, a computer program product (CPP) 441 can be provided. The CPP 441 includes a computer-readable storage medium (CRM) 442 that stores a computer program (CP) 443 including computer-readable instructions (CRI) 444. The CRM 442 can be a non-transitory computer-readable storage medium such as a magnetic medium (e.g., a hard disk), an optical medium, a memory device (e.g., a random access memory, a flash memory), etc.In some embodiments, when the CRI 444 of the computer program 443 is executed by the PC 402, the CRI is configured to cause the audio rendering device 400 to execute the processes described herein (e.g., the processes described herein with reference to the flowcharts). In other embodiments, the audio rendering device 400 may be configured to execute the processes described herein without requiring code. That is, for example, the PC 402 may be composed of only one or more ASICs. Therefore, the features of the embodiments described herein may be implemented by at least one of hardware and software.

[0087] Overview of Various Embodiments

[0088] A1. A method (300) executed by an audio renderer (151), the method comprising: obtaining metadata of an extended reality scene (s302); obtaining or deriving a first reverberation parameter from the metadata (s304), where the first reverberation parameter is a reverberation time parameter or a reverberation level parameter; and deriving a second reverberation parameter using the first reverberation parameter (s306), wherein when the first reverberation parameter is the reverberation time parameter, the second reverberation parameter is the reverberation level parameter, and when the first reverberation parameter is the reverberation level parameter, the second reverberation parameter is the reverberation time parameter.

[0089] A2. The method according to embodiment A1, wherein the metadata includes an acoustic absorption parameter indicating an acoustic absorption amount (A), and the first reverberation parameter is derived using the acoustic absorption parameter.

[0090] A3. The method according to embodiment A2, wherein the first reverberation parameter is a reverberant sound to direct sound energy ratio (RDR) value, and deriving the RDR value includes calculating 16×(π / A).

[0091] A4. The first reverberation parameter is the reverberation time parameter (RT) (e.g., RT60 value), and deriving the second reverberation parameter includes calculating X×RT or RT / X, where X is a number. A method according to embodiment A1 or A2, characterized in that.

[0092] A5. The first reverberation parameter is the reverberation time parameter (RT) (e.g., RT60 value), the first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume, and deriving the second reverberation parameter includes calculating f1x(RT / V) or (f1x(RT / V) when f1 is a predetermined coefficient, f2 is a predetermined value (in some embodiments f2 = 1), and V is the volume value indicating the volume of the acoustic environment. f2 ) including. An apparatus according to embodiment A1 or A2, characterized in that. In one embodiment, f1 is a function of the distance d from an omnidirectional point sound source. For example, when c is a predetermined coefficient (e.g., c = 3.1×10 2 ) f1 is equal to c×d 2 . In another embodiment, f1 is equal to 3.1×10 2 . In another embodiment, when c is a predetermined coefficient (e.g., c = 3.1×10 2 ) and C is a predetermined coefficient, f1 = C×c.

[0093] A6. The first reverberation parameter is the reverberation level parameter (RL), and deriving the second reverberation parameter includes calculating X×RL or RL / X, where X is a number. A method according to any one of embodiments A1 - A3, characterized in that.

[0094] A7. The first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume, and deriving the second reverberation parameter includes calculating VxRL / f1 or (Vx(RL / f1) when f1 is a predetermined coefficient, V is the volume value indicating the volume of the acoustic environment, and f2 is a predetermined value. 1 / f2 ) including. An apparatus according to embodiment A6, characterized in that.

[0095] A8. The first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume, the first reverberation parameter is the reverberation time parameter (RT), and deriving the second reverberation parameter involves, when V is the volume of the acoustic environment and t1 is a time value,

Equation

[0096] A9. The second reverberation parameter is the reverberation level parameter, and the second reverberation parameter is derived using the first reverberation parameter and a predetermined time value t1, the method according to any one of Embodiments A1, A2, A4, A5.

[0097] A10. When C is a correction coefficient depending on the first reverberation parameter and the time value t1, and c is a predetermined value, f1 is equal to C × c, the method according to Embodiment A5.

[0098] A11. C is

Equation

[0099] A12. t1 is derived based on at least one dimension of the acoustic environment, the method according to any one of Embodiments A8 to A11.

[0100] A13. t1 is proportional to the acoustic flight time related to the dimension of the acoustic environment, the method according to any one of Embodiments A8 to A11.

[0101] A14. When L is the size of the longest dimension of the acoustic environment and s is the speed of sound, t1 = 4xL / s, the method according to Embodiment A13.

[0102] A15.t1 indicates a pre-delay time related to the acoustic environment, and the method according to any one of Embodiments A8 to A11, characterized in that.

[0103] A16.t1 is a time value indicating a part of the room impulse response related to the acoustic environment, and the method according to any one of Embodiments A8 to A11, characterized in that.

[0104] A17. The reverberation level parameter is represented by an energy ratio between the reverberant sound and the total radiated energy of the sound source, and the method according to any one of Embodiments A1 to A16, characterized in that.

[0105] A18. A method further comprising a step of generating a reverberation signal using the first reverberation parameter and the second reverberation parameter, and a step of generating an output audio signal using the reverberation signal, and the method according to any one of Embodiments A1 to A17, characterized in that.

[0106] B1. A method (350) executed by an audio renderer (151), comprising a step (s352) of obtaining a set of reverberation parameters including at least a first reverberation parameter and a second reverberation parameter from metadata of an extended reality scene, and a step (s354) of determining whether the first reverberation parameter matches the second reverberation parameter, and the determining step includes a step (s356) of calculating a first value using the second reverberation parameter, and a step (s358) of comparing a difference between the first value and the first reverberation parameter with a threshold value, and the method is characterized in that.

[0107] B2. As a result of determining that the difference exceeds the threshold value, the method according to Embodiment B1, further comprising a step of generating a reverberation signal using the first value instead of the first reverberation parameter, and the method is characterized in that.

[0108] B3. The first reverberation parameter is a reverberation level parameter, and the second reverberation parameter is either a reverberation time parameter or an absorption parameter (A), or the first reverberation parameter is a reverberation time parameter, and the second reverberation parameter is either the reverberation level parameter or the absorption parameter (A), or the first reverberation parameter is the absorption parameter, and the second reverberation parameter is either the reverberation level parameter or the reverberation time parameter, the method according to embodiment B1 or B2, characterized in that.

[0109] B4. The set of reverberation parameters further includes a third reverberation parameter, and the method further includes a step of determining whether the first reverberation parameter matches the third reverberation parameter as a result of determining that the first reverberation parameter does not match the second reverberation parameter, and the step of determining whether the first reverberation parameter matches the third reverberation parameter includes a step of calculating a second value using the third reverberation parameter, the method according to embodiment B1, characterized in that.

[0110] Comparing the difference between the second value and the first reverberation parameter with the threshold value.

[0111] B5. As a result of determining that the first reverberation parameter does not match either the second reverberation parameter or the third reverberation parameter, the method according to embodiment B4, further comprising a step of generating a reverberation signal using either the first value or the second value instead of the first reverberation parameter.

[0112] C1. A computer program, characterized in that it includes instructions for causing an audio renderer to execute any one of the methods of the above embodiments when executed by a processing circuit of the audio renderer.

[0113] A carrier containing the computer program according to Embodiment C1, wherein the carrier is one of an electrical signal, an optical signal, a wireless signal, and a computer-readable storage medium.

[0114] D1. An audio rendering apparatus configured to execute any one of the methods of the above embodiments.

[0115] D2. The audio rendering apparatus according to Embodiment D1, wherein the audio rendering apparatus includes a memory and a processing circuit coupled to the memory.

[0116] E1. A method executed by an audio renderer, the method including: a step (s302) of acquiring metadata of an extended reality scene; a step of acquiring or deriving a first reverberation parameter from the metadata; and a step of deriving a second reverberation level parameter using the first reverberation parameter.

[0117] E2. Further including a step of acquiring a reverberation time parameter (RT), RDR received as the first reverberation level parameter, Let t1 be the start time used by the audio renderer, Let t2 be the start time related to the first reverberation level parameter (for example, the start time included in the metadata), then the second reverberation level parameter is

Number

[0118] Although various embodiments are described in this specification, it should be understood that they are presented by way of example only and not by way of limitation. Accordingly, the breadth and scope of the present disclosure should not be limited by any of the above-described exemplary embodiments. Further, unless otherwise indicated herein or clearly contradicted by context, any combination of the above objects in all possible variations thereof is encompassed by the present disclosure.

[0119] In addition, the processes described above and shown in the drawings are shown as a series of steps, but this is done for illustration purposes only. Accordingly, some steps may be added, some steps may be omitted, the order of the steps may be rearranged, or some steps may be executed in parallel.

Claims

1. A method (300) executed by an audio renderer (151), comprising: obtaining metadata of an extended reality scene (s302); obtaining or deriving a first reverberation parameter from the metadata, where the first reverberation parameter is a reverberation time parameter RT; deriving a second reverberation parameter using the first reverberation parameter and a predetermined time value t1 (s306); wherein the second reverberation parameter is a reverberation level parameter. A method characterized by the above.

2. The metadata includes an acoustic absorption parameter indicating an acoustic absorption amount (A), and the first reverberation parameter is derived using the acoustic absorption parameter. The method according to claim 1, characterized by the above.

3. Deriving the second reverberation parameter includes calculating X×RT or RT / X, where X is a number. The method according to claim 1, characterized by the above.

4. The first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume. Deriving the second reverberation parameter includes When f1 is a predetermined coefficient, f2 is a predetermined value, and V is a volume value indicating the volume of the acoustic environment, it includes calculating f1x(RT / V) or (f1x(RT / V) ｆ２ ) The method according to claim 1, characterized by the above.

5. The first reverberation parameter and the second reverberation parameter are associated with an acoustic environment having a volume. Deriving the second reverberation parameter includes when V is the volume of the acoustic environment, 【Number 22】 including calculating The method according to claim 1, characterized by the above.

6. When C is a correction coefficient depending on the first reverberation parameter and the time value t1, and c is a predetermined value, f1 is equal to C×c. The method according to claim 4, characterized by the above.

7. C is 【Number 23】 equal to, The method according to claim 6, characterized by the above.

8. t1 is derived based on at least one dimension of the acoustic environment. The method according to claim 5, characterized by the above.

9. t1 is proportional to the acoustic flight time related to the dimension of the acoustic environment. The method according to claim 5, characterized by the above.

10. When L is the size of the longest dimension of the acoustic environment and s is the speed of sound, t1 = 4×L / s. The method according to claim 9, characterized by the above.

11. t1 indicates a pre-delay time related to the acoustic environment. The method according to claim 5, characterized by the above.

12. The method according to claim 5, characterized in that t1 is a time value indicating a part of the room impulse response related to the acoustic environment.

13. The method according to claim 1, characterized in that the reverberation level parameter is represented by an energy ratio between the reverberant sound and the total radiated energy of the sound source.

14. A step of generating a reverberation signal using the first reverberation parameter and the second reverberation parameter; A step of generating an output audio signal using the reverberation signal; The method according to claim 1, further comprising the above steps.

15. An audio rendering device (400), wherein the audio rendering device A step of acquiring metadata of an extended reality scene (s302); A step of acquiring or deriving a first reverberation parameter from the metadata (s304), wherein the first reverberation parameter is a reverberation time parameter RT; A step of deriving a second reverberation parameter using the first reverberation parameter and a predetermined time value t1 (s306); Is configured to execute a process including The second reverberation parameter is a reverberation level parameter. An audio rendering device characterized by the above.

16. The audio rendering device according to claim 15, further configured to execute the method according to any one of claims 2 to 14.

Images