An audio rendering method, apparatus, device and computer readable storage medium

Abstract

This invention discloses an audio rendering method, apparatus, device, and computer-readable storage medium, applied in the field of computer technology. The method includes: extracting sound bed components, audio object components, and metadata from a spatial audio file; mapping the audio object components to virtual speaker channels for sound image localization based on the metadata, obtaining a first format file; mapping the sound bed components to a second format file; the second format file is a spherical format file; merging the first and second format files to obtain a target rendering file; and performing audio rendering based on the actual playback scene and the target rendering file. Compared to current uniform rendering, which results in poor audio rendering quality, this invention distinguishes between the rendering processes of objects and sound beds, and uses a mapping algorithm for sound image localization to achieve channel gain allocation. This avoids the sound image blurring caused by phase interference in traditional solutions, thus better preserving the creator's intent and sound field consistency, thereby improving the rendering effect.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to an audio rendering method, apparatus, device, and computer-readable storage medium. Background Technology

[0002] In existing technologies, spatial audio files are directly used, and the Object (audio object) and Bed (sound bed) components are input into a renderer for integrated rendering. Then, they are downmixed to a predetermined standard multichannel format for distribution and playback. This method processes the phase of the audio, which causes the spherical sound image of the Bed to be stretched or deformed, affecting the sound field layout originally set by the producer.

[0003] It is evident that the current audio rendering effect is poor, and how to improve the audio rendering effect is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide an audio rendering method, apparatus, device and computer-readable storage medium, which solves the technical problem of poor audio rendering effect in the prior art.

[0005] To address the aforementioned technical problems, this invention provides an audio rendering method, comprising:

[0006] Extract sound bed components, audio object components, and metadata from spatial audio files;

[0007] Based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization to obtain a first format file;

[0008] The acoustic bed components are mapped to a second format file; wherein the second format file is a spherical format file;

[0009] The first format file and the second format file are merged to obtain the target rendering file;

[0010] Audio rendering is performed based on the actual playback scene and the target rendering file.

[0011] Optionally, before mapping the audio object components to virtual speaker channels for sound image localization based on the metadata to obtain the first format file, the method further includes:

[0012] The mapping algorithm is determined to be an algorithm that determines the projection coordinates corresponding to the audio object components and selects a preset number of virtual speaker channels from all virtual speaker channels that satisfy the first and second conditions.

[0013] Wherein, the first condition is that the quadrilateral formed by the preset number of virtual speaker channels as vertices contains the horizontal projection coordinates of the projection coordinates;

[0014] The second condition is that the sum of the distances between the preset number of virtual speaker channels and the projection coordinates is less than the sum of the distances between the preset number of other virtual speaker channels and the projection coordinates that satisfy the first condition;

[0015] Accordingly, based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization to obtain a first format file, including:

[0016] Based on the metadata, the audio object components are mapped to virtual speaker channels using the defined mapping algorithm to perform sound image localization, thereby obtaining the first format file.

[0017] Optionally, based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization using the defined mapping algorithm to obtain the first format file, including:

[0018] Based on the Z-axis coordinates of the audio object components, the audio object components are divided into the upper ear layer and the sky layer;

[0019] In the above-ear layer, a preset number of virtual speaker channels in the above-ear layer are selected using the set mapping algorithm, and the gain of each virtual speaker channel in the above-ear layer is calculated according to the spatial distance relationship between the virtual speaker channels in the above-ear layer and the projected coordinates of the audio object components, so as to obtain the first format file of the above-ear layer.

[0020] In the sky layer, a preset number of sky layer virtual speaker channels are selected using the set mapping algorithm, and the gain of each sky layer virtual speaker channel is calculated based on the spatial distance relationship between the sky layer virtual speaker channels and the projected coordinates of the audio object components, to obtain the first format file of the sky layer;

[0021] The first format file is determined based on the first format file of the ear layer and the first format file of the sky layer.

[0022] Optionally, in the ear-high layer, a preset number of ear-high layer virtual speaker channels are selected using the defined mapping algorithm, and the gain of each ear-high layer virtual speaker channel is calculated based on the spatial distance relationship between the ear-high layer virtual speaker channels and the projected coordinates of the audio object components, to obtain a first format file of the ear-high layer, including:

[0023] The gain of each virtual speaker channel at the upper ear level is obtained by performing acoustic image localization processing on each virtual speaker channel at the upper ear level using either a vector basis algorithm or a distance basis algorithm.

[0024] Optionally, the first format file and the second format file are merged to obtain the target rendering file, including:

[0025] The channels in the first format file and the second format file are superimposed to obtain the target rendering file.

[0026] Optionally, audio rendering is performed based on the actual playback scene and the target rendering file, including:

[0027] The target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters.

[0028] Optionally, the target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters, including:

[0029] When the actual playback scenario is a multi-speaker playback scenario, the user's set reference point in three-dimensional space is taken as the origin, and the total number of actual speakers used by the user and the three-dimensional coordinate information of each actual speaker relative to the origin are obtained.

[0030] Based on the three-dimensional coordinate information of the actual loudspeakers, a mappable space is determined around the origin; wherein, the mappable space is the region outside the mappable space such that all actual loudspeakers are located outside the mappable space.

[0031] Determine the virtual speaker channel corresponding to the target rendering file in the mappable space;

[0032] The virtual speaker channel is used as the virtual speaker to be mapped. Based on the virtual speaker to be mapped, the multi-channel audio signal in the target rendering file is mapped to the channel corresponding to the actual speaker using the set mapping algorithm. Playback is then performed based on the channel of the actual speaker.

[0033] Optionally, the target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters, including:

[0034] When the actual playback scenario is a binaural headphone playback scenario, the virtual speaker channel position is determined according to the target rendering file;

[0035] Based on the user's head and ear features, as well as the location of the virtual speaker channel, determine the matching binaural impulse response data for the user;

[0036] Using the aforementioned binaural impulse response data and convolution technology, the multi-channel audio signal in the target rendering file is rendered into a binaural spatial audio signal;

[0037] The spatial audio signals from both ears are transmitted to the left and right earphones for playback.

[0038] Optionally, the target rendering file is a 31-track multi-channel audio file in 20.2.8.1 format, including 20 tracks for the upper ear layer, 2 tracks for the middle layer, 8 tracks for the sky layer, and 1 track for the low-frequency effects channel.

[0039] The present invention also provides an audio rendering apparatus, comprising:

[0040] The extraction module is used to extract the soundbed component, audio object component, and metadata from spatial audio files;

[0041] The first format file determination module is used to map the audio object components to virtual speaker channels for sound image localization based on the metadata, thereby obtaining the first format file;

[0042] The second format file determination module is used to map the acoustic bed components to a second format file; wherein the second format file is a spherical format file;

[0043] The target rendering file determination module is used to merge the first format file and the second format file to obtain the target rendering file;

[0044] The audio rendering module is used to perform audio rendering based on the actual playback scene and the target rendering file.

[0045] The present invention also provides an audio rendering device, comprising:

[0046] Memory, used to store computer programs;

[0047] A processor for executing the computer program to implement the steps of the above-described audio rendering method.

[0048] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described audio rendering method.

[0049] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described audio rendering method.

[0050] As can be seen, this invention extracts the sound bed component, audio object component, and metadata from a spatial audio file; based on the metadata, it maps the audio object component to a virtual speaker channel for sound image localization, obtaining a first format file; it maps the sound bed component to a second format file, wherein the second format file is a spherical format file; it merges the first and second format files to obtain a target rendering file; and it performs audio rendering based on the actual playback scene and the target rendering file. Compared to the current uniform rendering, which results in poor audio rendering effects, this invention distinguishes between the rendering processing of objects and sound beds, and performs sound image localization only through a mapping algorithm, thereby better preserving the creator's intent and sound field consistency, thus improving the rendering effect. Furthermore, it adjusts the audio playback rendering parameters based on the actual playback scene, thereby improving the accuracy of the audio playback rendering parameters and further enhancing the rendering effect.

[0051] In addition, the present invention also provides an audio rendering apparatus, device, and computer-readable storage, which also have the above-mentioned beneficial effects. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0053] Figure 1 A flowchart of an audio rendering method provided in an embodiment of the present invention;

[0054] Figure 2 A schematic diagram of a mappable space provided for an embodiment of the present invention;

[0055] Figure 3 This is an overall structural framework diagram of an audio rendering method provided in an embodiment of the present invention;

[0056] Figure 4 A schematic diagram of an OBOB virtual speaker channel provided in an embodiment of the present invention;

[0057] Figure 5 A schematic diagram of a BDOB view provided in an embodiment of the present invention;

[0058] Figure 6 A schematic diagram of a 31-track multi-channel audio provided in an embodiment of the present invention;

[0059] Figure 7 This is a schematic diagram of the structure of an audio rendering device provided in an embodiment of the present invention;

[0060] Figure 8 This is a schematic diagram of the structure of an audio rendering device provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Some terms that appear in the description of the embodiments of this application are subject to the following interpretation:

[0063] Multi-channel: refers to a multi-channel audio file that fully contains spatial audio information. Different channels in this file correspond to audio signals with different spatial locations or functions.

[0064] Surround playback: refers to the arrangement of multiple speakers in a space, and through signal processing, mapping and playback of the distributed spatial audio multi-channel file to these speakers, thereby forming a surround sound or panoramic sound playback scene.

[0065] Binaural playback: refers to using headphones as the playback terminal and rendering the downloaded spatial audio multi-channel file into a playback form of binaural spatial audio signal through signal processing such as binaural impulse response and convolution.

[0066] Audio object: Represents a point sound source in space, that is, an audio signal unit with a definite three-dimensional position and time-varying characteristics.

[0067] Bed: refers to a sound source of plane wave or ambient sound field, usually used to carry background sound, ambient sound, etc., and has a relatively stable spatial distribution.

[0068] Virtual speaker channel: In this invention, the virtual speaker channel is used to carry multi-channel signals of the object or bed. It corresponds to a virtual speaker virtually placed at a certain position in space and is the target channel of the dual-balance mapping algorithm. The virtual speaker channel can correspond to the actual speaker channel in spatial position, and is used to realize the mapping from content format to actual playback layout.

[0069] Please refer to Figure 1 , Figure 1 A flowchart illustrating an audio rendering method provided in an embodiment of the present invention. The method may include:

[0070] S101 extracts the sound bed component, audio object component, and metadata from the spatial audio file.

[0071] The steps in this embodiment can be performed by a designated electronic device, which can be a server, a portable terminal, or other form. The spatial audio file in this embodiment can be an ADM BWF (Audio Defined Model Broadcast Waveform Format) file. ADM BWF is the industry standard "ultimate delivery file" for next-generation immersive audio (such as Dolby Atmos), packaging all audio data and three-dimensional spatial description information into a single, self-contained professional audio file. In this embodiment, the Bed component is a set of multi-channel audio streams identified by ADM as constituting the "sound bed." The AudioObject component is the extraction of the mono or stereo audio stream corresponding to each independently defined Object, and each stream is labeled with a unique ID. Metadata: Extracts a set of time-synchronized spatialized parameters strictly bound to each Object ID.

[0072] S102, based on the metadata, map the audio object components to the virtual speaker channel for sound image localization to obtain the first format file.

[0073] In this embodiment, the audio object components can be rendered as a first format file with a set format, for example, the object component can be rendered as a 24-channel Object-Object (OBOB) file in 16.8 format. The image localization in this embodiment is mainly to calculate the gain mapping of each channel to the virtual speaker channel. This embodiment can achieve smooth mapping and image interpolation of the object in both horizontal and vertical dimensions.

[0074] It should be further explained that, based on any of the above embodiments, before mapping the audio object components to virtual speaker channels for sound image localization according to metadata to obtain the first format file, it may further include: determining that the mapping algorithm is an algorithm for determining the projection coordinates corresponding to the audio object components and selecting a preset number of virtual speaker channels from all virtual speaker channels that satisfy the first and second conditions; wherein, the first condition is that the preset number of virtual speaker channels form a quadrilateral containing the projection coordinates; the second condition is that the sum of the distances between the preset number of virtual speaker channels and the projection coordinates is less than the sum of the distances between the preset number of virtual speaker channels and the projection coordinates of other preset number of virtual speaker channels that satisfy the first condition; correspondingly, mapping the audio object components to virtual speaker channels for sound image localization according to metadata to obtain the first format file may include: mapping the audio object components to virtual speaker channels for sound image localization according to metadata using the set mapping algorithm to obtain the first format file. In this embodiment, the set mapping algorithm is to select a group of 4 virtual speaker channels from all virtual speaker channels on a given horizontal plane according to the projection coordinates (x, y) of the mapped object, such that:

[0075] The quadrilateral formed by the four virtual speaker channels contains the horizontal projected coordinates of the mapped object; the sum of the distances between the four virtual speaker channels and the mapped object is smaller than the sum of the distances of any other combination of four channels satisfying condition a.). It should be noted that this embodiment can perform audio-visual interpolation when mapping based on the spatial coordinates of the mapped object, making the audio-visual positioning more accurate and continuous, thus allowing mapping processing to be performed based on the interpolated coordinates.

[0076] It should be further explained that, based on any of the above embodiments, according to metadata, using a set mapping algorithm to map audio object components to virtual speaker channels for sound image localization, to obtain a first format file, it may include:

[0077] S1021, based on the Z-axis coordinate of the audio object component, divide the audio object component into the upper ear layer and the sky layer.

[0078] For the height dimension Z, based on the Z-axis coordinate of the mapped object, the object is proportionally divided into the upper ear layer (when Z=0) or the sky layer (Z=1, because the lowest height is considered to be 0 and the highest height is considered to be 1, in a Cartesian coordinate system).

[0079] S1022, In the upper ear layer, a preset number of upper ear layer virtual speaker channels are selected using a set mapping algorithm, and the gain of each upper ear layer virtual speaker channel is calculated based on the spatial distance relationship between the upper ear layer virtual speaker channels and the projected coordinates of the audio object components, thus obtaining the first format file of the upper ear layer.

[0080] In this embodiment, for each object in the ear layer, four virtual speaker channels in the ear layer are selected according to its coordinates using the above-mentioned mapping algorithm, and the gain of each channel is calculated according to the spatial distance relationship to obtain the first format file of the ear layer, such as obtaining the 16-track object mapping result of the ear layer.

[0081] S1023, In the sky layer, a preset number of sky layer virtual speaker channels are selected using a set mapping algorithm, and the gain of each sky layer virtual speaker channel is calculated based on the spatial distance relationship between the sky layer virtual speaker channels and the projected coordinates of the audio object components, thus obtaining the first format file of the sky layer.

[0082] In this embodiment, for each object in the sky layer, based on its coordinates, four virtual speaker channels in the sky layer are selected using a predefined mapping algorithm to obtain a second format file for the sky layer, such as the mapping result of 8-track objects in the sky layer.

[0083] S1024, Determine the first format file based on the first format file of the ear layer and the first format file of the sky layer.

[0084] This embodiment renders the object component into a first format file, such as a 24-channel Object-Object (OBOB) file, based on the first format files of the ear layer and the sky layer. This embodiment provides a specific method for determining the first format file, improving the accuracy of the first format file determination.

[0085] It should be further explained that, based on any of the above embodiments, in the upper ear layer, a preset number of virtual speaker channels in the upper ear layer are selected using a set mapping algorithm, and the gain of each virtual speaker channel in the upper ear layer is calculated according to the spatial distance relationship between the virtual speaker channels in the upper ear layer and the projected coordinates of the audio object components, resulting in a first format file of the upper ear layer. This can include: performing sound image localization processing on each virtual speaker channel in the upper ear layer using a vector basis algorithm or a distance basis algorithm to obtain the gain of the virtual speaker channels in the upper ear layer. The gain is calculated based on the geometric distance weight between the virtual speaker channel and the projected coordinates, or sound image localization is achieved based on a vector basis / distance basis algorithm. By determining the gain, sound image localization can be achieved. This embodiment provides a specific method for sound image localization, improving the accuracy of sound image localization.

[0086] S103, map the sound bed components to a second format file; wherein, the second format file is a spherical format file.

[0087] The method for mapping the sound bed components to a second format file in this embodiment can be VBAP (Vector Base Amplitude Panning). Specifically, this embodiment can map the Bed components to a 10-channel spherical 7.1.2 format file based on the target distribution of the Bed components on a sphere, to represent the ambient sound field and background sound.

[0088] S104: Merge the first format file and the second format file to obtain the target rendering file.

[0089] This embodiment merges the first and second format files primarily by overlaying the channels. For example, the Object is mapped to a 24-channel file in 16.8 format; the Bed component is mapped to a 10-channel file in spherical 7.1.2 format; and the two are merged into a 31-track multi-channel audio file in 20.2.8.1 format. The target rendering file is a 31-track multi-channel audio file in 20.2.8.1 format, comprising 20 tracks for the upper layer, 2 tracks for the middle layer, 8 tracks for the sky layer, and 1 track for low-frequency effects (LFE). Through this processing, the music producer's intentions can be reproduced to the greatest extent possible, sound image shift can be avoided, and the accuracy and continuity of sound image movement can be guaranteed during downmixing and playback.

[0090] It should be further explained that, based on any of the above embodiments, the above-mentioned merging of the first format file and the second format file to obtain the target rendering file may include: superimposing the channels in the first format file and the second format file to obtain the target rendering file. In this embodiment, for example, the 16-track Object of the upper ear layer, the 8-track Object of the sky layer, and the 10-track Bed of the spherical 7.1.2 format are considered together, and a 31-track multi-channel audio in the 20.2.8.1 format is formed through channel merging and position matching. During the merging process, channels with equivalent spatial positions are merged or superimposed to reduce the number of redundant channels while maintaining spatial resolution.

[0091] S105 performs audio rendering based on the actual playback scene and the target rendering file.

[0092] The actual playback scenarios in this embodiment can include multi-speaker scenarios, binaural playback scenarios, etc. When rendering audio based on the target rendering file, for multi-speaker scenarios, the 31-track multi-channel signal can be mapped to the user's speaker layout using the aforementioned mapping algorithm, based on the actual number and three-dimensional position of the speakers used by the user. For headphone scenarios, the 31-track multi-channel signal is rendered into binaural spatial audio signals through binaural impulse response and convolution. The mapping and binaural response are dynamically updated based on the real-time tracked user position and / or head posture, ensuring that the user is always in an area as close as possible to the optimal listening position when moving or turning their head, thus obtaining a stable and natural spatial sound image experience. The technical solution of this invention can be applied to the rendering, distribution, and playback processes of spatial audio. Through the multi-channel storage and distribution format and corresponding client mapping processing provided by this invention: when using multi-speaker systems in home theaters, multimedia living rooms, multi-speaker studios, etc., users can obtain reasonable spatial audio playback under any number and placement of speakers; when using headphones (including wired headphones, Bluetooth headphones, over-ear / in-ear headphones, etc.), users can obtain a binaural spatial audio experience with stable sound images and head movement tracking capabilities; the same content can be uniformly stored and distributed using the 20.2.8.1 format, and personalized adaptation can be achieved through client mapping under different terminals and different playback scenarios, thereby significantly enhancing the end-user's immersion and consistent experience.

[0093] It should be further explained that, based on any of the above embodiments, the process of merging the first format file and the second format file to obtain the target rendering file, and then performing audio rendering based on the target rendering file, can include: processing the target rendering file based on the actual playback scene to obtain audio playback rendering parameters, and then rendering and playing back the audio based on the audio playback rendering parameters. In this embodiment, the target rendering file is mapped based on the actual playback scene and adapted to the current scene to obtain the audio playback rendering parameters. This embodiment can adjust the audio playback rendering parameters based on the actual playback scene, thereby improving the accuracy of the audio playback rendering parameters.

[0094] It should be further explained that, based on any of the above embodiments, the above-described processing of the target rendering file based on the actual playback scenario to obtain audio playback rendering parameters, and rendering and playing back the audio based on the audio playback rendering parameters, may include:

[0095] Step 1: When the actual playback scenario is a multi-speaker playback scenario, take the user's set reference point in three-dimensional space as the origin, obtain the total number of actual speakers used by the user, and the three-dimensional coordinate information of each actual speaker relative to the origin.

[0096] This step is primarily for obtaining the speaker layout. In this embodiment, a reference point in three-dimensional space (e.g., a typical listening position) can be used as the origin to obtain the total number of speakers actually used by the user, as well as the three-dimensional coordinate information of each speaker relative to that origin.

[0097] Step 2: Based on the three-dimensional coordinate information of the actual loudspeakers, determine the mappable space around the origin; wherein, the mappable space is the area outside the mappable space such that all actual loudspeakers are located outside the mappable space.

[0098] This embodiment can determine a mappable space around the user based on the layout of the user's speakers.

[0099] Step 3: Determine the virtual speaker channel corresponding to the target rendering file in the mappable space.

[0100] This embodiment allows for the arrangement of virtual speaker channels within this space, each corresponding one-to-one with the 31-track multi-channel signal in the target rendering file, ensuring a one-to-one correspondence with the issued 20.2.8.1 multi-channel format. Specifically, this includes, but is not limited to: a. Constructing an axis-aligned cuboid region with a length-width-height ratio of 2:2:1 in a three-dimensional coordinate system, using the current user position as the origin. This cuboid is defined as the current mappable space. b. The dimensions of the mappable space along the three coordinate axes are determined by the user position and the relative three-dimensional coordinates of each speaker: In the positive and negative directions of each coordinate axis, the nearest speaker to the user is located, and the maximum extension distance of the cuboid in that direction is calculated under the constraint that "the speaker does not fall inside the cuboid." For example, while ensuring all speakers are located outside the mappable space (allowing the nearest speaker to fall on its boundary), the extension distance of the cuboid in each direction is maximized, making the boundary of the mappable space tangent or approximately tangent to one or more speakers closest to the user, thereby maximizing the effective mapping area while ensuring a safety margin.

[0101] Step 4: Use the virtual speaker channel as the virtual speaker to be mapped. Based on the virtual speaker to be mapped, use the set mapping algorithm to map the multi-channel audio signal in the target rendering file to the channel corresponding to the actual speaker, and play back the audio based on the channel of the actual speaker.

[0102] This embodiment, after determining the mappable space, maps the 31 virtual speaker channels corresponding to the multi-channel format in section 20.2.8.1 to the interior of the cuboid according to predefined relative orientations, obtaining the three-dimensional coordinates of each virtual speaker channel in the mappable space. For example, using virtual speaker channels as the objects to be mapped and actual speaker channels as the mapping set, the aforementioned mapping algorithm is used again to map the 31-track multi-channel audio signal to the speaker channels actually used by the user, thereby achieving spatial audio playback in scenarios with any number and placement of speakers. When a change in the user's position or the placement of some speakers is detected, the mappable space and mapping relationship are recalculated, and the mapping gain is updated in real time, ensuring that the user remains within the optimal listening area as much as possible during movement, obtaining a stable spatial sound field experience. For easier understanding, please refer to... Figure 2 , Figure 2 This is a schematic diagram of a mappable space provided in an embodiment of the present invention.

[0103] It should be further explained that, based on any of the above embodiments, processing the target rendering file according to the actual playback scenario to obtain audio playback rendering parameters, and then rendering and playing back the audio based on the audio playback rendering parameters, may include:

[0104] S1: When the actual playback scenario is a binaural headphone playback scenario, determine the virtual speaker channel position based on the target rendering file;

[0105] S2: Based on the user's head and ear features, as well as the location of the virtual speaker channel, determine the matching binaural impulse response data for the user;

[0106] S3: Using binaural impulse response data and convolution technology, the multi-channel audio signal in the target rendering file is rendered as a binaural spatial audio signal;

[0107] S4: Transmits the spatial audio signals from both ears to the left and right earbuds for playback.

[0108] The application scenario of this embodiment is a binaural playback scenario for headphones. In the headphone playback scenario, the client processing flow of this invention includes, but is not limited to, the following steps: Virtual speaker channel placement: In the virtual three-dimensional space, according to the issued multi-channel format 20.2.8.1, 31 virtual speaker channel positions are placed for each of the 31 channels. Binaural impulse response selection: Based on the user's head size, auricular features, and other information, suitable binaural impulse response (HRTF) data is selected, or the most matching response is selected from a set of standard HRTFs. Binaural rendering: Using the selected binaural impulse response and convolution technology, the 31-track multi-channel audio signal is rendered into a binaural spatial audio signal and output to the left and right headphone units. Head tracking and dynamic update: When a change in the user's head rotation angle or posture is detected, the azimuth and elevation angles of the 31 virtual speaker channels relative to the user's head coordinate system are calculated in real time, and binaural impulse response data in the corresponding direction is selected or interpolated accordingly. The binaural rendering results are updated in real time, so that the user can still perceive the relative position of the sound image in the external space as stable when turning their head, thereby improving immersion and realism.

[0109] It should be further explained that, based on any of the above embodiments, after merging the first format file and the second format file to obtain the target rendering file, the process may further include: storing the target rendering file, which is a multi-channel file containing audio information but not ADM BWF metadata. In this embodiment, on the distribution and storage side, only the multi-channel file containing audio information needs to be distributed and stored; ADM metadata is no longer carried (because the signal in the distributed multi-channel file is already the effect of the signal plus ADM metadata). That is, the role of metadata has been "internalized" into the rendering result of the 31 channels, so it is not needed during transmission, thereby reducing transmission and storage pressure.

[0110] An audio rendering method provided by this invention may include: S101, extracting sound bed components, audio object components, and metadata from a spatial audio file; S102, mapping the audio object components to virtual speaker channels for sound image localization based on the metadata, obtaining a first format file; S103, mapping the sound bed components to a second format file; wherein the second format file is a spherical format file; S104, merging the first format file and the second format file to obtain a target rendering file; S105, performing audio rendering based on the actual playback scene and the target rendering file. Compared with the current method of uniformly rendering sound bed components and audio object components, which results in poor audio rendering effects, this invention distinguishes between the rendering processing of objects and sound beds, and achieves channel gain only through mapping algorithms for sound image localization. This avoids the sound image blurring caused by phase interference in traditional solutions, thus better preserving the creator's intent and sound field consistency, thereby improving the rendering effect. Furthermore, it adjusts the audio playback rendering parameters based on the actual playback scene, thereby improving the accuracy of the audio playback rendering parameters and further enhancing the rendering effect.

[0111] In the existing technology, there are mainly two typical solutions:

[0112] The direct integration rendering scheme based on ADM BWF files: directly use ADM BWF files, input the Object and Bed components into the renderer for integrated rendering, and then downmix to a predetermined standard multi-channel format (such as 5.1, 7.1.4, etc.) for distribution and playback.

[0113] A direct binaural rendering scheme based on ADM BWF files: without going through intermediate multi-channel formats, the Object and Bed components in the ADM BWF file are directly processed by a binaural renderer to output binaural spatial audio for playback.

[0114] The above-mentioned existing technical solutions have the following shortcomings:

[0115] Issues with pre-rendering and delivery: Objects and Beds are not modeled and processed independently. They are usually rendered together, which can easily cause the spherical sound image of the Bed to be stretched or deformed, affecting the sound field layout originally set by the producer.

[0116] Issues with rendering and playback in multi-speaker scenarios: Multi-speaker playback typically only supports predefined standard speaker layouts and does not support scenarios where users can place any number of speakers arbitrarily; it is generally only optimized for a single "default listening position," and when the user deviates from this position, the sound image positioning accuracy and immersion significantly decrease; when the user's position changes, it is impossible to dynamically adjust the rendering parameters according to the listener's real-time position to keep the user constantly near the optimal listening position.

[0117] Problems in binaural playback scenarios: The movement of sound images in space is often not continuous or smooth enough. Users may experience "jumps" or instability in sound images when turning their heads or moving, which affects the sense of immersion.

[0118] This invention aims to solve the above problems by processing Object and Bed separately on the pre-rendering side and introducing a dual-balanced mapping algorithm (setting the mapping algorithm) and a unified 20.2.8.1 multi-channel format. This enables a relatively consistent, accurate, and smooth spatial sound image representation in both multi-speaker and binaural playback scenarios, while balancing file size, transmission pressure, and sound image positioning accuracy. This invention proposes a spatial audio storage and distribution format based on a multi-channel format and a dual-balanced mapping algorithm. On the server side, this format pre-renders ADM BWF (Audio Definition Model Broadcast Waveform Format) files to generate multi-channel audio files containing Object-Object (OBOB, audio object-audio object) and Bed-Object (BDOB, audio object-sound bed) components. On the client side, signal post-processing and playback are performed according to the actual usage scenario (including multi-speaker scenarios with any number and placement, and binaural headphone scenarios).

[0119] For a clearer understanding of this invention, please refer to the following details. Figure 3 , Figure 3 This is a structural framework diagram of an audio rendering method provided by an embodiment of the present invention. The pre-rendering, distribution, signal post-processing, and playback process of the present invention may specifically include:

[0120] (1) The renderer extracts the Bed sound bed component, the Object audio object component, and the corresponding metadata from the ADM BWF file.

[0121] (2) The renderer renders the Object component into a 24-channel Object-Object (OBOB) file in 16.8 format using the set mapping algorithm based on the metadata.

[0122] For easier understanding, please refer to Figure 4 , Figure 4This is a schematic diagram of an OBOB virtual speaker channel provided in an embodiment of the present invention. In the left diagram, the X-axis represents left and right, the Y-axis represents front and back, and the Z-axis represents height. In the middle and right diagrams, the X-axis represents left and right, and the Y-axis represents depth. The mapping algorithm in this embodiment is used to map the object to be rendered to a suitable number of virtual speaker channels given a virtual speaker channel layout. The inputs to the double-balanced mapping algorithm include: the spatial coordinates of the rendered object to be mapped; the rendered object being mapped is an Object; and the spatial coordinates of all possible virtual speaker channels involved in the mapping. The core idea of ​​the double-balanced mapping algorithm includes: on a given horizontal plane, based on the projection coordinates of the object being mapped, selecting a group of four virtual speaker channels from all virtual speaker channels, such that: the quadrilateral formed by the four virtual speaker channels contains the horizontal projection coordinates of the object being mapped; and the sum of the distances between the four virtual speaker channels and the object being mapped is smaller than the sum of the distances of any other combination of four channels satisfying condition a.). For the height dimension z, based on the z-axis coordinates of the mapped object, the object is proportionally divided into the upper ear layer or the sky layer: In the upper ear layer, for each object, based on its coordinates, four virtual speaker channels are selected using the mapping algorithm described above, and the gain of each channel is calculated based on the spatial distance relationship, resulting in a 16-track object mapping result for the upper ear layer. In the sky layer, for each object, based on its coordinates, four virtual speaker channels are selected using the same double-balanced mapping algorithm, resulting in an 8-track object mapping result for the sky layer.

[0123] (3) The renderer maps the Bed component to a 10-channel Bed-Object (BDOB) file in spherical 7.1.2 format.

[0124] For easier understanding, please refer to Figure 5 , Figure 5 This is a schematic diagram of a BDOB view provided in an embodiment of the present invention.

[0125] (4) The renderer integrates the OBOB and BDOB lines and merges channels with consistent or equivalent spatial positions to obtain a 31-track multi-channel audio file in the 20.2.8.1 format. These 31 tracks can be divided into: 20 tracks for the upper layer, 2 tracks for the middle layer, 8 tracks for the top layer, and 1 track for low-frequency effects (LFE).

[0126] For easier understanding, please refer to Figure 6 , Figure 6 This is a schematic diagram of a 31-track multi-channel audio provided in an embodiment of the present invention.

[0127] (5) Send the above 31-track multi-channel audio files to the client as the download and storage format.

[0128] In addition to using the 20.2.8.1 format and setting the mapping method, this embodiment can also use multi-channel layouts with other channel numbers, different object interpolation algorithms, or layered sound bed schemes to achieve similar spatial audio playback effects.

[0129] (6) The client performs signal post-processing and playback on the multi-channel audio based on the user's actual playback scenario (multiple speakers or headphones), as well as information such as the user's position and head posture.

[0130] This invention employs a unified 20.2.8.1 multi-channel format combined with a dual-balance mapping algorithm to improve sound image localization accuracy, ensure compatibility with various speaker layouts and binaural playback, and reduce ADM data transmission and storage overhead. By differentiating between object and soundbed pre-rendering processing, it better preserves the creator's intent and sound field consistency. The client dynamically maps based on the user's real-time position and posture, enabling users to obtain a stable and natural immersive experience even in non-ideal listening positions. This invention achieves a balance between file size, sound image localization accuracy, and the reproduction of the music producer's creative intent.

[0131] The audio rendering apparatus provided in the embodiments of the present invention will be described below. The audio rendering apparatus described below and the audio rendering method described above can be referred to each other.

[0132] Please refer to the details. Figure 7 , Figure 7 A schematic diagram of an audio rendering device provided in an embodiment of the present invention may include:

[0133] Extraction module 100 is used to extract sound bed components, audio object components, and metadata from spatial audio files;

[0134] The first format file determination module 200 is used to map the audio object components to virtual speaker channels for sound image localization based on the metadata, thereby obtaining a first format file;

[0135] The second format file determination module 300 is used to map the acoustic bed components to a second format file; wherein the second format file is a spherical format file;

[0136] The target rendering file determination module 400 is used to merge the first format file and the second format file to obtain the target rendering file;

[0137] The audio rendering module 500 is used to perform audio rendering based on the actual playback scene and the target rendering file.

[0138] Furthermore, based on any of the above embodiments, the audio rendering apparatus may further include:

[0139] The mapping algorithm determination module is used to determine the mapping algorithm as an algorithm for determining the projection coordinates corresponding to the audio object components and selecting a preset number of virtual speaker channels from all virtual speaker channels that satisfy the first and second conditions.

[0140] Wherein, the first condition is that the quadrilateral formed by the preset number of virtual speaker channels as vertices contains the horizontal projection coordinates of the projection coordinates;

[0141] The second condition is that the sum of the distances between the preset number of virtual speaker channels and the projection coordinates is less than the sum of the distances between the preset number of other virtual speaker channels and the projection coordinates that satisfy the first condition;

[0142] Accordingly, the first format file determination module 200 may include:

[0143] The first format file determination unit is used to map the audio object components to a virtual speaker channel for sound image localization based on the metadata and the set mapping algorithm, thereby obtaining the first format file.

[0144] Furthermore, based on any of the above embodiments, the first format file determining unit may include:

[0145] The sub-unit is used to divide the audio object component into an upper ear layer and a sky layer based on the Z-axis coordinate of the audio object component.

[0146] The first format file determination subunit of the ear layer is used to select a preset number of ear layer virtual speaker channels in the ear layer using the set mapping algorithm, and calculate the gain of each ear layer virtual speaker channel according to the spatial distance relationship between the ear layer virtual speaker channel and the projection coordinates of the audio object component to obtain the first format file of the ear layer.

[0147] The first format file determination subunit of the sky layer is used to select a preset number of sky layer virtual speaker channels in the sky layer using the set mapping algorithm, and calculate the gain of each sky layer virtual speaker channel according to the spatial distance relationship between the sky layer virtual speaker channel and the projection coordinates of the audio object component to obtain the first format file of the sky layer.

[0148] The first format file determination subunit is used to determine the first format file based on the first format file of the ear layer and the first format file of the sky layer.

[0149] Furthermore, based on any of the above embodiments, the first format file determination subunit of the ear layer is specifically used to perform acoustic image localization processing on each ear layer virtual speaker channel using a vector basis algorithm or a distance basis algorithm to obtain the gain of the ear layer virtual speaker channel.

[0150] Furthermore, based on any of the above embodiments, the audio rendering module 400 may include:

[0151] The target rendering file determination unit is used to superimpose the channels in the first format file and the second format file to obtain the target rendering file.

[0152] Furthermore, based on any of the above embodiments, the audio rendering module 400 may include:

[0153] The actual playback scene rendering unit is used to process the target rendering file based on the actual playback scene to obtain audio playback rendering parameters, and then render and play back the audio based on the audio playback rendering parameters.

[0154] Furthermore, based on any of the above embodiments, the actual playback scene rendering unit may include:

[0155] The three-dimensional coordinate information determination subunit is used to obtain the number of all actual speakers actually used by the user and the three-dimensional coordinate information of each actual speaker relative to the origin, with the user's set reference point in three-dimensional space as the origin, when the actual playback scenario is a multi-speaker playback scenario.

[0156] A mappable space determination subunit is used to determine a mappable space around the origin based on the three-dimensional coordinate information of the actual loudspeakers; wherein, the mappable space is the region outside the mappable space such that all actual loudspeakers are located outside the mappable space;

[0157] A virtual speaker channel determination subunit is used to determine the virtual speaker channel corresponding to the target rendering file in the mappable space;

[0158] The first playback subunit is used to use the virtual speaker channel as a virtual speaker to be mapped, and based on the virtual speaker to be mapped, to use the set mapping algorithm to map the multi-channel audio signal in the target rendering file to the channel corresponding to the actual speaker, and to play back based on the channel of the actual speaker.

[0159] Furthermore, based on any of the above embodiments, the actual playback scene rendering unit may include:

[0160] The virtual speaker channel position determination subunit is used to determine the virtual speaker channel position based on the target rendering file when the actual playback scenario is a binaural headphone playback scenario.

[0161] The binaural impulse response data determination subunit is used to determine the binaural impulse response data matching the user based on the user's head features and auricular features, as well as the position of the virtual speaker channel;

[0162] The binaural spatial audio signal determination subunit is used to render the multi-channel audio signal in the target rendering file into a binaural spatial audio signal using the binaural impulse response data and convolution technology.

[0163] The second playback subunit is used to transmit the binaural spatial audio signal to the left and right earphones for playback.

[0164] Furthermore, based on any of the above embodiments, the audio rendering apparatus may further include:

[0165] The storage module is used to store the target rendering file, which is a multi-channel file containing audio information but not ADM BWF metadata.

[0166] It should be noted that the order of the modules and units in the aforementioned audio rendering device can be changed without affecting the logic.

[0167] An audio rendering apparatus provided in this invention may include: an extraction module 100, used to extract sound bed components, audio object components, and metadata from a spatial audio file; a first format file determination module 200, used to map the audio object components to virtual speaker channels for sound image localization based on the metadata, to obtain a first format file; a second format file determination module 300, used to map the sound bed components to a second format file; wherein the second format file is a spherical format file; a target rendering file determination module 400, used to merge the first format file and the second format file to obtain a target rendering file; and an audio rendering module 500, used to perform audio rendering based on the target rendering file. Compared with the current uniform rendering, which results in poor audio rendering effects, this invention distinguishes between the rendering processing of objects and sound beds, and performs sound image localization only through a mapping algorithm, thereby better preserving the creator's intent and sound field consistency, thus improving the rendering effect. Furthermore, it adjusts the audio playback rendering parameters based on the actual playback scene, thereby improving the accuracy of the audio playback rendering parameters and further enhancing the rendering effect.

[0168] The following describes an audio rendering device provided by an embodiment of the present invention. The audio rendering device described below can be referred to in correspondence with the audio rendering method described above.

[0169] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of an audio rendering device provided in an embodiment of the present invention may include:

[0170] Memory 10 is used to store computer programs;

[0171] Processor 20 is used to execute computer programs to implement the audio rendering method described above.

[0172] The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.

[0173] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions:

[0174] Extract sound bed components, audio object components, and metadata from spatial audio files;

[0175] Based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization, resulting in the first format file;

[0176] The sound bed components are mapped to a second format file; wherein, the second format file is a spherical format file;

[0177] The first format file and the second format file are merged to obtain the target rendering file;

[0178] Audio rendering is performed based on the actual playback scene and the target rendering file.

[0179] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.

[0180] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.

[0181] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.

[0182] The communication interface 30 can be an interface for the communication module, used to connect with other devices or systems.

[0183] Of course, it should be noted that, Figure 8 The structure shown does not constitute a limitation on the audio rendering device in the embodiments of the present invention. In practical applications, the audio rendering device may include devices such as audio rendering devices with different structures. Figure 8 More or fewer components as shown, or combinations of certain components.

[0184] The following describes the computer-readable storage medium provided in the embodiments of the present invention. The computer-readable storage medium described below can be referred to in correspondence with the audio rendering method described above.

[0185] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described audio rendering method.

[0186] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0188] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0189] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0190] The above provides a detailed description of an audio rendering method, apparatus, device, and computer-readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An audio rendering method, characterized in that, include: Extract sound bed components, audio object components, and metadata from spatial audio files; Based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization to obtain a first format file; The acoustic bed components are mapped to a second format file; wherein the second format file is a spherical format file; The first format file and the second format file are merged to obtain the target rendering file; Audio rendering is performed based on the actual playback scene and the target rendering file.

2. The audio rendering method according to claim 1, characterized in that, Before mapping the audio object components to virtual speaker channels for sound image localization based on the metadata to obtain the first format file, the method further includes: The mapping algorithm is determined to be an algorithm that determines the projection coordinates corresponding to the audio object components and selects a preset number of virtual speaker channels from all virtual speaker channels that satisfy the first and second conditions. Wherein, the first condition is that the quadrilateral formed by the preset number of virtual speaker channels as vertices contains the horizontal projection coordinates of the projection coordinates; The second condition is that the sum of the distances between the preset number of virtual speaker channels and the projection coordinates is less than the sum of the distances between the preset number of other virtual speaker channels and the projection coordinates that satisfy the first condition; Accordingly, based on the metadata, the audio object components are mapped to virtual speaker channels for sound image localization to obtain a first format file, including: Based on the metadata, the audio object components are mapped to virtual speaker channels using the defined mapping algorithm to perform sound image localization, thereby obtaining the first format file.

3. The audio rendering method according to claim 2, characterized in that, Based on the metadata, and using the defined mapping algorithm, the audio object components are mapped to virtual speaker channels for sound image localization to obtain the first format file, including: Based on the Z-axis coordinates of the audio object components, the audio object components are divided into the upper ear layer and the sky layer; In the above-ear layer, a preset number of virtual speaker channels in the above-ear layer are selected using the set mapping algorithm, and the gain of each virtual speaker channel in the above-ear layer is calculated according to the spatial distance relationship between the virtual speaker channels in the above-ear layer and the projected coordinates of the audio object components, so as to obtain the first format file of the above-ear layer. In the sky layer, a preset number of sky layer virtual speaker channels are selected using the set mapping algorithm, and the gain of each sky layer virtual speaker channel is calculated based on the spatial distance relationship between the sky layer virtual speaker channels and the projected coordinates of the audio object components, to obtain the first format file of the sky layer; The first format file is determined based on the first format file of the ear layer and the first format file of the sky layer.

4. The audio rendering method according to claim 3, characterized in that, In the aforementioned upper ear layer, a preset number of upper ear layer virtual speaker channels are selected using the defined mapping algorithm. Based on the spatial distance relationship between the upper ear layer virtual speaker channels and the projected coordinates of the audio object components, the gain of each upper ear layer virtual speaker channel is calculated to obtain a first format file for the upper ear layer, including: The gain of each virtual speaker channel at the upper ear level is obtained by performing acoustic image localization processing on each virtual speaker channel at the upper ear level using either a vector basis algorithm or a distance basis algorithm.

5. The audio rendering method according to claim 1, characterized in that, The first format file and the second format file are merged to obtain the target rendering file, which includes: The channels in the first format file and the second format file are superimposed to obtain the target rendering file.

6. The audio rendering method according to any one of claims 1 to 5, characterized in that, Audio rendering is performed based on the actual playback scene and the target rendering file, including: The target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters.

7. The audio rendering method according to claim 6, characterized in that, The target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters, including: When the actual playback scenario is a multi-speaker playback scenario, the user's set reference point in three-dimensional space is taken as the origin, and the total number of actual speakers used by the user and the three-dimensional coordinate information of each actual speaker relative to the origin are obtained. Based on the three-dimensional coordinate information of the actual loudspeakers, a mappable space is determined around the origin; wherein, the mappable space is the region outside the mappable space such that all actual loudspeakers are located outside the mappable space. Determine the virtual speaker channel corresponding to the target rendering file in the mappable space; The virtual speaker channel is used as the virtual speaker to be mapped. Based on the virtual speaker to be mapped, the multi-channel audio signal in the target rendering file is mapped to the channel corresponding to the actual speaker using the set mapping algorithm. Playback is then performed based on the channel of the actual speaker.

8. The audio rendering method according to claim 6, characterized in that, The target rendering file is processed based on the actual playback scenario to obtain audio playback rendering parameters, and the audio is rendered and played back based on the audio playback rendering parameters, including: When the actual playback scenario is a binaural headphone playback scenario, the virtual speaker channel position is determined according to the target rendering file; Based on the user's head and ear features, as well as the location of the virtual speaker channel, determine the matching binaural impulse response data for the user; Using the aforementioned binaural impulse response data and convolution technology, the multi-channel audio signal in the target rendering file is rendered into a binaural spatial audio signal; The spatial audio signals from both ears are transmitted to the left and right earphones for playback.

9. The audio rendering method according to claim 1, characterized in that, The target rendering file is a 31-track multi-channel audio file in 20.2.8.1 format. The 31 tracks include 20 tracks for the upper ear layer, 2 tracks for the middle layer, 8 tracks for the sky layer, and 1 track for the low-frequency effects channel.

10. An audio rendering apparatus, characterized in that, include: The extraction module is used to extract the soundbed component, audio object component, and metadata from spatial audio files; The first format file determination module is used to map the audio object components to virtual speaker channels for sound image localization based on the metadata, thereby obtaining the first format file; The second format file determination module is used to map the acoustic bed components to a second format file; wherein the second format file is a spherical format file; The target rendering file determination module is used to merge the first format file and the second format file to obtain the target rendering file, and to perform audio rendering based on the target rendering file; The audio rendering module is used to perform audio rendering based on the actual playback scene and the target rendering file.

11. An audio rendering device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the audio rendering method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the audio rendering method as described in any one of claims 1 to 9.

Images