Method for generating audio object metadata and electronic equipment

Abstract

The invention relates to a method for generating audio object metadata and electronic equipment, and belongs to the technical field of audio processing, and the method comprises the following steps: inputting an audio file into a renderer; identifying an audio track of the audio file through the renderer, and generating an audio track unique identifier corresponding to the audio track; determining an audio track group of the audio track and identifying the position of the audio track in the audio file; and generating audio object metadata according to the audio track data of the same audio track group. The audio file can be analyzed to generate the audio object metadata, and accurate audio object metadata can be provided for a renderer during audio playing.

Description

Technical Field

[0001] The present application relates to the technical field of audio processing, and in particular to a method and electronic device for generating audio object metadata. Background Art

[0002] As technology advances, audio becomes increasingly complex. From early mono audio to stereo, the focus has been on correctly processing the left and right channels. However, with the advent of surround sound, processing has become increasingly complex. Surround 5.1 speaker systems prioritize the sequencing of multiple channels, and subsequent 6.1 and 7.1 speaker systems have further complicated audio processing, ensuring the correct signal is delivered to the appropriate speakers to create a coherent effect. Consequently, as sound becomes more immersive and interactive, the complexity of audio processing has increased significantly.

[0003] Audio channels (or channels) refer to independent audio signals collected or played back at different spatial locations during recording or playback. The number of channels refers to the number of sound sources during recording or the number of corresponding speakers during playback. For example, a 5.1 surround speaker system includes six audio signals at different spatial locations, and each independent audio signal is used to drive a speaker at the corresponding spatial location; a 7.1 surround speaker system includes eight audio signals at different spatial locations, and each independent audio signal is used to drive a speaker at the corresponding spatial location.

[0004] However, the effects achieved by current speaker systems depend on the number and spatial placement of speakers. For example, a two-channel speaker system cannot achieve the effects of a 5.1 surround speaker system. Existing speaker systems adapt fixed output channels to fixed input formats, making them inflexible to the complex audio track grouping requirements of different surround speaker systems.

[0005] The present application provides an audio program metadata and a generation method thereof, so as to provide metadata that can solve the above-mentioned technical problems. Summary of the Invention

[0006] The purpose of this application is to propose a method and electronic device for generating audio object metadata to solve the above technical problems.

[0007] To achieve the above objectives, the present application provides, in a first aspect, a method for generating audio object metadata, comprising:

[0008] Feed the audio file into the renderer;

[0009] Identifying the audio track of the audio file by the renderer and generating a unique audio track identifier corresponding to the audio track;

[0010] determining a track grouping of the audio track and identifying a location of the audio track in the audio file;

[0011] Generates audio object metadata based on track data of the same track group.

[0012] Optionally, determining the track grouping of the audio track and identifying the position of the audio track in the audio file includes:

[0013] According to the surround speaker system version of the audio file, the track grouping and the track position of the audio track are determined.

[0014] Optionally, generate audio object metadata based on the track data of the same track group, including:

[0015] Generate audio track unique identification reference information in audio object metadata according to audio track data of the same audio track group;

[0016] An audio object identifier and an audio object name of the audio object are generated, wherein the audio object identifier includes information indicating a relationship between multiple audio objects.

[0017] Optionally, audio object metadata is generated based on the track data of the same track group, further comprising:

[0018] The start time information and duration information of the audio object are determined based on the track data of the same track group.

[0019] Optionally, before identifying the audio track of the audio file by the renderer and generating a unique audio track identifier corresponding to the audio track, the method further includes:

[0020] The renderer identifies the type of audio in the audio file.

[0021] Optionally, the method for generating audio object metadata further includes:

[0022] The renderer identifies the audio file and determines at least one of audio object importance information, user-object interaction information, and automatic dodging information in the audio object metadata.

[0023] Optionally, the method for generating audio object metadata further includes:

[0024] The renderer identifies the audio file and determines at least one of audio package format identification reference information, audio object identification reference information, audio complementary object identification reference information, and audio object interaction information in the audio object metadata.

[0025] A second aspect of the present application provides an electronic device, comprising: a memory and one or more processors;

[0026] The memory is used to store one or more programs;

[0027] When the one or more programs are executed by the one or more processors, the one or more processors execute the method for generating audio object metadata as provided in any embodiment of the present application.

[0028] Grouping of audio tracks, identifying positions in audio files, and generating audio object metadata based on the same grouped audio data.

[0029] Generate audio track unique identification metadata based on the audio data corresponding to each group of audio objects.

[0030] The renderer processes the audio track data and generates audio content metadata based on the information of each audio object, which describes the parameters that constitute the content audio object.

[0031] According to the generated audio content composition, the composed program information is extracted and the audio program metadata AudioProgramme is generated, which describes the composed audio content attributes.

[0032] The method and electronic device for generating audio object metadata provided in this application dynamically analyze the surround system version of the audio file (such as 5.1, 7.1 speaker systems, etc.), combine it with multi-dimensional audio track feature analysis, automatically generate audio track grouping and spatial position information that matches the target speaker system, and construct an adaptive metadata framework containing interactive parameters, thereby enabling the renderer to dynamically optimize complex audio scenes and provide the renderer with accurate audio object metadata during audio playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a method for generating audio object metadata is provided in an embodiment of the present application;

[0034] Figure 2 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The following examples are used to illustrate the present application but are not used to limit the scope of the present application.

[0036] Metadata is information that describes the structural characteristics of data. Metadata supports functions such as indicating storage location, historical data, resource search, or file records. Each stage of the multidimensional audio model produces metadata that describes the characteristics of that stage.

[0037] Example

[0038] This application provides a method for generating audio object metadata in a multi-dimensional audio model and provides a detailed description. Figure 1 The flowchart of the method for generating audio object metadata shown in FIG. 1 includes:

[0039] Step 110: Input the audio file into the renderer;

[0040] Step 120: Identify the audio track of the audio file by the renderer, and generate a unique audio track identifier corresponding to the audio track;

[0041] Step 130: determining the track grouping of the audio track and identifying the position of the audio track in the audio file;

[0042] Extract predefined surround system identifiers from the audio file header and directly match preset grouping templates; and be compatible with non-standardized audio files through speaker layout configuration or custom configuration.

[0043] Step 140: Generate audio object metadata based on the audio track data of the same audio track group.

[0044] Optionally, determining the track grouping of the audio track and identifying the position of the audio track in the audio file includes:

[0045] According to the surround speaker system version of the audio file, the track grouping and the track position of the audio track are determined.

[0046] Specifically, the audio tracks are grouped by L / R / C / LFE / Ls / Rs and associated with preset polar coordinate positions (such as the default azimuth angle of the left surround channel Ls is 110°). For the audio object to be processed, the mapping and connection of audio content, format and other related information are established through the unique identification number of the audio track to generate an independent audio object. The normalized coordinates (X, Y, Z) of the audio track are calculated using a sound field simulation algorithm (such as VBAP, Vector Base Amplitude Panning) and written into the position (spatial position) field of the metadata to ensure that the renderer accurately calculates the speaker drive gain. Optionally, audio object metadata is generated based on the audio track data of the same audio track grouping, including:

[0047] Generate audio track unique identification reference information in audio object metadata according to audio track data of the same audio track group;

[0048] An audio object identifier and an audio object name of the audio object are generated, wherein the audio object identifier includes information indicating a relationship between multiple audio objects.

[0049] Optionally, audio object metadata is generated based on the track data of the same track group, further comprising:

[0050] The start time information and duration information of the audio object are determined based on the track data of the same track group.

[0051] The start time information of the audio object is related to the start time of playing the audio program in the audio object, and the duration information of the audio object is related to the start time and end time of playing the audio program in the audio object.

[0052] Optionally, before identifying the audio track of the audio file by the renderer and generating a unique audio track identifier corresponding to the audio track, the method further includes:

[0053] The renderer identifies the type of audio in the audio file.

[0054] Optionally, the method for generating audio object metadata further includes:

[0055] The renderer identifies the audio file and determines at least one of audio object importance information, user-object interaction information, and automatic dodging information in the audio object metadata.

[0056] The audio object importance information is used to describe the importance index of the audio object. The user-object interaction information is used to indicate whether the user is allowed to interact with the object. The automatic dodging information is used to indicate whether the object is allowed to automatically dodge.

[0057] Specifically, in the process of whether to allow objects to automatically dodge, priority parameters are defined (such as dialogue track priority = 10, background music = 5). When a high-priority object is activated, the gain of other objects is automatically reduced. The Boolean value (allow_gain_adjust) and gain (gain_range, -12dB ~ +6dB) fields are used to control whether the user can adjust the volume and the adjustment range. Position offset thresholds (such as azimuth angle ±30°, distance 0.5 to 1.5 times) are preset in the metadata to prevent users from dragging objects to invalid sound field areas. Through the above parameters, metadata not only describes the static properties of audio objects, but also defines dynamic behavior rules, so that the renderer can adjust the output strategy in real time according to user operations or environmental changes (such as adding or removing speakers).

[0058] Optionally, the method for generating audio object metadata further includes:

[0059] The renderer identifies the audio file and determines at least one of audio package format identification reference information, audio object identification reference information, audio complementary object identification reference information, and audio object interaction information in the audio object metadata.

[0060] The audio package format identifier reference information is used to reference the audio package format identifier for format description. The audio object identifier reference information is used to describe other referenced audio objects. The audio complementary object identifier reference information is used to describe the audio objects that complement the audio object. The audio object interaction information is used to characterize the specifications for user interaction with the object.

[0061] Optionally, the audio object interaction information includes:

[0062] Audio object switch information, where the audio object switch information is used to indicate whether the audio object is allowed to be turned on or off;

[0063] Audio object gain information, where the audio object gain information is used to indicate whether the gain of the audio object is allowed to be changed;

[0064] Audio object position information, where the audio object position information is used to indicate whether the user is allowed to change the position of the audio object.

[0065] Audio object gain information includes:

[0066] Maximum value information of the audio gain, where the maximum value information of the audio gain is used to represent a maximum gain factor allowing user gain interaction;

[0067] Minimum value information of the audio gain, where the minimum value information of the audio gain is used to represent a minimum gain factor allowing user gain interaction.

[0068] Optionally, the audio object position information includes:

[0069] Polar coordinate parameters and Cartesian coordinate parameters, the polar coordinate parameters are used to represent the position information of the audio object in a polar coordinate manner, and the Cartesian coordinate parameters are used to represent the position information of the audio object in a Cartesian coordinate manner.

[0070] Optionally, the polar coordinate parameters include:

[0071] Maximum azimuth information, used to represent the maximum azimuth offset value that allows user location interactivity;

[0072] Minimum azimuth information, used to characterize the minimum azimuth offset value that allows user location interactivity;

[0073] Maximum elevation angle information, used to represent the maximum elevation offset value that allows user position interactivity;

[0074] Minimum elevation angle information, used to represent the minimum elevation offset value that allows user position interactivity;

[0075] Maximum distance information, used to represent the maximum standardized distance value that allows user location interaction;

[0076] Minimum distance information is used to represent the minimum standardized distance value that allows user location interaction.

[0077] Optionally, the Cartesian coordinate parameters include:

[0078] Maximum X-axis coordinate information, used to represent the maximum X-axis offset value of the normalized unit that allows user position interaction;

[0079] Minimum X-axis coordinate information, used to represent the minimum X-axis offset value that allows the user position interaction normalization unit;

[0080] Maximum Y-axis coordinate information, used to represent the standardized unit of the maximum Y-axis offset value that allows user position interaction;

[0081] Minimum Y-axis coordinate information, used to represent the standardized unit of the minimum Y-axis offset value that allows user position interaction;

[0082] Maximum Z-axis coordinate information, used to represent the standardized unit of the maximum Z-axis offset value that allows user position interaction;

[0083] Minimum Z-axis coordinate information, used to represent the standardized unit of the minimum Z-axis offset value that allows user position interaction.

[0084] An audio object is the connection between an actual audio track and its format. The track's unique identifier establishes a mapping and connection between the audio content, format, and other related information. Audio objects can nest within themselves to group other audio objects together. As can be seen, an audio object can be a combination of one or more. The track's unique identifier element, also known as the track's unique identifier, generates metadata describing the structural characteristics of the track's unique identifier.

[0085] The audio content describes the content of a component of the audio content (such as background music) and refers to one or more audio objects to link the content with its format. The audio content element is to produce audio content and generate metadata for the audio content to describe the structural characteristics of the audio content.

[0086] An audio program includes narration, sound effects, and background music. It references one or more audio content, which are combined to form a complete audio object. Audio program elements, also known as audio objects, generate metadata for audio programs, which describe the structural characteristics of the audio program.

[0087] Group audio tracks, identify their locations in audio files, and generate audio object metadata (AudioObject) based on the same grouped audio data.

[0088] In one implementation, after grouping and identifying the positions of the audio tracks, the process also includes generating unique identification metadata for the audio tracks:

[0089] Generate audio track unique identification metadata (audioTrackUID) based on the audio data corresponding to each group of audio objects.

[0090] The process generates a track unique identifier and track description information corresponding to the track. It also generates an audio material exchange format lookup sub-element, audio track format reference information, and audio package format reference information. The track description information includes the track sampling rate and track bit depth. The track unique identifier metadata includes an attribute area containing the track unique identifier information and preset track description information; a sub-element area containing the audio material exchange format lookup sub-element, audio track format reference information, and audio package format reference information. Relevant information from the track unique identifier metadata attribute area and sub-element area is obtained to generate the track unique identifier metadata. The obtained information is then set in the corresponding area of ​​the track unique identifier metadata. The audio package format reference information is the audio package identifier information of the audio package format referenced by the track unique identifier metadata. The audio track format reference information is the audio track identifier information of the audio track format referenced by the track unique identifier metadata. The audio material exchange format lookup sub-element includes MXF package track identifier reference information, MXF track reference information, and channel track reference information.

[0091] In one implementation, after grouping and identifying the positions of the audio tracks, the method further includes processing the audio track data through a renderer;

[0092] generating audio content metadata based on audio track data;

[0093] Audio content metadata (AudioContent) is generated based on information about each audio object, describing the parameters that make up the content audio object. The audio content metadata includes an audio content identifier (AID) representing identification information about the audio content, and an audio content name representing name information about the audio content. The audio content metadata also includes reference audio object information, which is used to reference one or more audio object elements and associate the content of the audio object elements with their format. The audio content metadata may include: an attribute area, including an audio content identifier (AID) and an audio content name (AID), wherein the AID represents identification information about the audio content, and the AID represents name information about the audio content; and a sub-element area, including reference audio object information, which is used to reference one or more audio object elements and associate the content of the audio object elements with their format. Relevant information from the audio content metadata attribute area and sub-element area is obtained to generate the audio content metadata. The obtained information is then placed in the corresponding area of ​​the audio content metadata. The attribute area may also include language information for the audio content, which describes the selected language of the audio content. The sub-element area may further include: content measured loudness information of the audio content, the content measured loudness information being used to describe the measured loudness of the audio content; and / or dialogue element information, the dialogue element information being used to indicate whether the audio program is a dialogue, and its corresponding attribute information.

[0094] In one implementation, after generating the audio content metadata, the process further includes generating playable audio:

[0095] According to the generated audio content composition, the composed program information is extracted to generate audio program metadata (AudioProgramme), which describes the composed audio content attributes.

[0096] The audio program metadata includes an audio program identifier and an audio program name, and the audio program identifier includes audio program information created by referencing one or more audio contents;

[0097] The audio program metadata also includes: audio content reference information, which includes audio content information referenced when the audio program is played. The audio content reference information includes one or more referenced audio content information.

[0098] Audio program metadata may include: an attribute area including an audio program identifier and audio program name of the audio program, wherein the audio program identifier includes audio program information created by referencing one or more audio content; and a sub-element area including audio content reference information, wherein the audio content reference information includes audio content information referenced during audio program playback, wherein the audio content reference information includes one or more audio content information referenced. Relevant information in the audio program metadata attribute area and sub-element area is obtained to generate audio program metadata. The obtained information is set in the corresponding area of ​​the audio program metadata. The attribute area may also include audio language information indicating the audio program displayed on the display screen during audio program playback. The attribute area may also include start time information indicating the audio program playback, wherein the start time information has at least five digits in seconds. The attribute area may also include end time information indicating the audio program playback, wherein the end time information has at least five digits in seconds. The attribute area may also include maximum allowable ducking information indicating the allowable ducking of the audio program playback. The sub-element area may also include program loudness information of the audio program playback, wherein the program loudness information is the decibel value of the audio program playback. The sub-element area may further include screen information of the audio program playing, where the screen information is used to represent the screen aspect ratio, center point position, and screen width information of the display screen when the audio program is playing.

[0099] The input audio file is rendered and played as a whole through the renderer, and a BW64 (Broadcast WAVE) file of "metadata + audio data" is exported. The objects in the ADM XML file are bound to the BW64 channel through a unique identifier (such as UUID) to build an audio production model that complies with the ADM (Audio Definition Model) standard.

[0100] According to an embodiment of the present invention, audio tracks with similar functions (such as "background sounds" or "dialogues") can be automatically grouped together, and batch adjustment of spatial parameters (such as uniformly setting surround sound diffusion) is supported. By identifying the position of the audio track in the original audio (such as the proportion of left and right channels), accurate spatial coordinate data is provided to the ADM to achieve immersive sound field construction. For audio tracks in the same group, dynamic processing (such as compression and limiters) can be uniformly applied to ensure the loudness consistency of audio objects in the group. For example: in game audio, a unified volume attenuation curve can be set for the "ambient sound effects" group to avoid sudden volume jumps.

[0101] Furthermore, tracks in the same group share basic metadata (such as copyright information and sampling rate), reducing the need for re-entry. The generated metadata complies with the ETSI TS103 190 specification and supports fast switching between multi-language and multi-version ADM files.

[0102] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 2 As shown, the electronic device includes: a processor 30, a memory 31, an input device 32 and an output device 33. The number of processors 30 in the electronic device can be one or more. Figure 2 In the example, a processor 30 is used. The number of memories 31 in the electronic device can be one or more. Figure 2 In the example, a memory 31 is used. The processor 30, memory 31, input device 32 and output device 33 of the electronic device can be connected through a bus or other means. Figure 2 The example of the bus connection is shown in FIG. The electronic device may be a computer or a server. The embodiment of the present application is described in detail with the electronic device being a server, which may be an independent server or a cluster server.

[0103] Memory 31, as a computer-readable storage medium, can be used to store software programs, computer executable programs, and modules, such as program instructions / modules for generating audio program metadata as described in any embodiment of the present application. Memory 31 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data generated based on device usage. Furthermore, memory 31 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state memory device. In some instances, memory 31 may further include memory remotely located relative to processor 30, and such remote memory may be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0104] The input device 32 can be used to receive input digital or character information and generate key signals related to the user settings and function control of the electronic device. It can also be a camera for capturing images and a sound pickup device for capturing audio data. The output device 33 can include audio equipment such as speakers. It should be noted that the specific composition of the input device 32 and output device 33 can be set according to actual circumstances.

[0105] The processor 30 executes the software programs, instructions and modules stored in the memory 31 to perform various functional applications and data processing of the device, that is, to generate audio metadata.

[0106] In the description of this specification, reference to the terms "in one embodiment," "in another embodiment," "exemplary," or "in a specific embodiment" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples.

[0107] Although the present application has been described in detail above using general explanations, specific embodiments, and experiments, it will be apparent to those skilled in the art that modifications or improvements may be made to the present application. Therefore, such modifications or improvements, without departing from the spirit of the present application, are within the scope of protection claimed in the present application.

Claims

1. A method for generating audio object metadata, characterized in that: include: Feed the audio file into the renderer; Identifying the audio track of the audio file by the renderer and generating a unique audio track identifier corresponding to the audio track; determining a track grouping of the audio track and identifying a location of the audio track in the audio file; Generates audio object metadata based on track data of the same track group.

2. The method for generating audio object metadata according to claim 1, characterized in that: Determining a track grouping of the audio track and identifying a position of the audio track in the audio file comprises: According to the surround speaker system version of the audio file, the track grouping and the track position of the audio track are determined.

3. The method for generating audio object metadata according to claim 2, characterized in that: Generates audio object metadata based on the track data of the same track group, including: Generate audio track unique identification reference information in audio object metadata according to audio track data of the same audio track group; An audio object identifier and an audio object name of the audio object are generated, wherein the audio object identifier includes information indicating a relationship between multiple audio objects.

4. The method for generating audio object metadata according to claim 3, characterized in that: Generates audio object metadata based on track data of the same track group, including: The start time information and duration information of the audio object are determined based on the track data of the same track group.

5. The method for generating audio object metadata according to claim 4, characterized in that: Before identifying the audio track of the audio file by the renderer and generating a unique audio track identifier corresponding to the audio track, the method further includes: The renderer identifies the type of audio in the audio file.

6. The method for generating audio object metadata according to claim 5, characterized in that: Also includes: The renderer identifies the audio file and determines at least one of audio object importance information, user-object interaction information, and automatic dodging information in the audio object metadata.

7. The method for generating audio object metadata according to claim 6, characterized in that: Also includes: The renderer identifies the audio file and determines at least one of audio package format identification reference information, audio object identification reference information, audio complementary object identification reference information, and audio object interaction information in the audio object metadata.

8. An electronic device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors execute the method for generating audio object metadata according to any one of claims 1 to 7.

Images