Method for generating audio content metadata and electronic equipment

Abstract

The invention relates to a method for generating audio content metadata and electronic equipment, and belongs to the technical field of audio processing, and the method comprises the following steps: identifying audio tracks of an audio file through a renderer, determining audio track marshalling of the audio tracks, and identifying the positions of the audio tracks in the audio file; and processing the audio track data through a renderer, and generating audio content metadata according to the audio track data.

Description

[0001] This application is a divisional application of the patent application filed on April 9, 2025, with application number 202510444067.2 and invention title "A method and electronic device for generating audio object metadata". Technical Field

[0002] This application relates to the field of audio processing technology, and more particularly to a method and electronic device for generating audio content metadata. Background Technology

[0003] With the advancement of technology, audio has become increasingly complex. Early mono audio evolved into stereo, with a focus on the correct processing of the left and right channels. However, the advent of surround sound made the processing even more complex. Surround 5.1 speaker systems, by ordering and constraining multiple channels, and further, surround 6.1 speaker systems, surround 7.1 speaker systems, and so on, have created countless variations in audio processing, delivering the correct signals to the appropriate speakers to create interconnected effects. Therefore, as sound becomes more immersive and interactive, the complexity of audio processing has increased significantly.

[0004] An audio channel (or channel) refers to an independent audio signal that is captured or played back from different spatial locations during recording or playback. The number of channels corresponds to the number of audio sources during recording or the number of speakers during playback. For example, a 5.1 surround sound system includes six audio signals from different spatial locations, each driving a speaker at its corresponding location; a 7.1 surround sound system includes eight audio signals from different spatial locations, each driving a speaker at its corresponding location.

[0005] However, the effects achieved by current speaker systems depend on the number and spatial placement of the speakers. For example, a two-channel speaker system cannot achieve the effect of a 5.1 surround sound system. Existing speaker systems adapt to fixed input formats with fixed output channels, and cannot flexibly adapt to the complex track grouping requirements of different surround sound systems. Summary of the Invention

[0006] The purpose of this application is to propose a method and electronic device for generating audio content metadata in order to solve the above-mentioned technical problems.

[0007] To achieve the above objectives, the first aspect of this application provides a method for generating audio content metadata, comprising:

[0008] The renderer identifies the audio tracks of the audio file, determines the track grouping of the audio tracks, and identifies the position of the audio tracks in the audio file.

[0009] The renderer processes the audio track data and generates audio content metadata based on the audio track data.

[0010] A second aspect of this application provides an electronic device, including: a memory and one or more processors;

[0011] The memory is used to store one or more programs;

[0012] When the one or more programs are executed by the one or more processors, the one or more processors perform a method for generating audio content metadata as provided in any embodiment of this application.

[0013] Track grouping and position identification within audio files; generating audio object metadata based on audio data from the same group.

[0014] Generate unique metadata for each audio track based on the audio data corresponding to each audio object.

[0015] The renderer processes audio track data and generates audio content metadata based on the information of each audio object, describing the parameters that constitute the content audio object.

[0016] Based on the composition of the generated audio content, the program information of the composition is extracted, and audio program metadata (AudioProgramme) is generated, which describes the attributes of the composed audio content. Attached Figure Description

[0017] Figure 1 This application provides a flowchart of a method for generating audio object metadata.

[0018] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0020] Metadata is information that describes the structural characteristics of data, and its supported functions include indicating storage location, historical data, resource lookup, or file records. Metadata describing the characteristics of each stage of the multidimensional audio-visual model is generated.

[0021] Example

[0022] This application provides a method for generating audio object metadata in a multidimensional audio model, and details it. For example... Figure 1 The flowchart shown illustrates a method for generating audio object metadata, which includes:

[0023] Step 110: Input the audio file into the renderer;

[0024] Step 120: Identify the audio track of the audio file using the renderer and generate a unique identifier for the audio track corresponding to the audio track;

[0025] Step 130: Determine the track grouping of the audio track and identify the position of the audio track in the audio file;

[0026] Extract predefined surround system identifiers from the header of the audio file and directly match them with preset grouping templates; and make the audio files compatible with non-standard audio files through speaker layout configuration or custom configuration.

[0027] Step 140: Generate audio object metadata based on the audio track data of the same audio track group.

[0028] Optionally, determining the track grouping of the audio track and identifying the position of the audio track in the audio file includes:

[0029] Based on the surround speaker system version of the audio file, determine the track grouping and track position of the audio track.

[0030] Specifically, audio tracks are grouped by L / R / C / LFE / Ls / Rs and associated with preset polar coordinate positions (e.g., the default azimuth angle for the left surround channel Ls is 110°). For each audio object to be processed, a mapping and relationship between audio content, format, and other relevant information is established using the unique identifier of the audio track, generating an independent audio object. Sound field simulation algorithms (such as VBAP, Vector Base Amplitude Panning) are used to calculate the normalized coordinates (X, Y, Z) of the audio track and written to the position (spatial location) field of the metadata to ensure the renderer accurately calculates the speaker drive gain. Optionally, audio object metadata is generated based on the audio track data of the same track group, including:

[0031] Generate unique track identifier reference information in the audio object metadata based on the track data of the same track group;

[0032] Generate an audio object identifier and an audio object name for the audio object, wherein the audio object identifier includes information indicating the relationship between multiple audio objects.

[0033] Optionally, generating audio object metadata based on track data from the same track group also includes:

[0034] The start time and duration information of an audio object are determined based on the track data of the same track group.

[0035] The start time information of an audio object is related to the start time of playback of the audio program within the audio object. The duration information of an audio object is related to the start and end times of playback of the audio program within the audio object.

[0036] Optionally, before identifying the audio track of the audio file through the renderer and generating a unique track identifier corresponding to the audio track, the method further includes:

[0037] The renderer identifies the type of audio in the audio file.

[0038] Optionally, methods for generating audio object metadata also include:

[0039] The renderer identifies the audio file and determines at least one of the following in the audio object metadata: audio object importance information, user-object interaction information, and automatic dodging information.

[0040] Among them, audio object importance information describes the importance index of the audio object. User interaction information indicates whether user interaction with the object is allowed. Automatic dodging information indicates whether automatic dodging of the object is allowed.

[0041] Specifically, in determining whether to allow objects to automatically dodge, priority parameters are defined (e.g., dialogue audio track priority = 10, background music = 5). When a high-priority object is activated, the gain of other objects is automatically reduced. Boolean values ​​(allow_gain_adjust) and gain range (-12dB to +6dB) control whether the user can adjust the volume and the adjustment range. Preset position offset thresholds in the metadata (e.g., azimuth ±30°, distance 0.5 to 1.5 times) prevent users from dragging objects to invalid sound field areas. Through these parameters, the metadata not only describes the static attributes of audio objects but also defines dynamic behavior rules, allowing the renderer to adjust its output strategy in real time based on user actions or environmental changes (e.g., adding or removing speakers).

[0042] Optionally, methods for generating audio object metadata also include:

[0043] The renderer identifies the audio file and determines at least one of the following in the audio object metadata: audio package format identifier reference information, audio object identifier reference information, audio complementary object identifier reference information, and audio object interaction information.

[0044] The audio package format identifier reference information is used to describe the format of the audio package format. The audio object identifier reference information is used to describe other referenced audio objects. The audio complementary object identifier reference information is used to describe audio objects that are complementary to the aforementioned audio object. The audio object interaction information is used to characterize the specifications of user interaction with the object.

[0045] Optionally, the audio object interaction information includes:

[0046] Audio object switch information, which is used to indicate whether the audio object is allowed to be turned on or off;

[0047] Audio object gain information, which is used to characterize whether changing the gain of the audio object is allowed;

[0048] Audio object location information, which is used to indicate whether the user is allowed to change the location of the audio object.

[0049] Audio object gain information includes:

[0050] Maximum audio gain information, which is used to characterize the maximum gain factor that allows user gain interaction;

[0051] Minimum audio gain information, which is used to characterize the minimum gain factor that allows user gain interaction.

[0052] Optionally, the audio object location information includes:

[0053] Polar coordinate parameters and Cartesian coordinate parameters, wherein the polar coordinate parameters are used to represent the position information of the audio object in a polar coordinate manner, and the Cartesian coordinate parameters are used to represent the position information of the audio object in a Cartesian coordinate manner.

[0054] Optionally, the polar coordinate parameters include:

[0055] Maximum azimuth information is used to characterize the maximum azimuth offset value that allows user location interactivity;

[0056] Minimum azimuth information is used to characterize the minimum azimuth offset value that allows user positional interactivity;

[0057] Maximum elevation angle information is used to characterize the maximum elevation offset value that allows users to interact with their location.

[0058] Minimum elevation angle information is used to characterize the minimum elevation offset value that allows users to interact with each other's positions.

[0059] Maximum distance information is used to characterize the maximum standardized distance value that allows users to interact at different locations.

[0060] Minimum distance information is used to characterize the minimum standardized distance value that allows users to interact at different locations.

[0061] Optionally, the Cartesian coordinate parameters include:

[0062] Maximum X-axis coordinate information is used to characterize the maximum X-axis offset value that allows users to interact with their positions in normalized units;

[0063] Minimum X-axis coordinate information, used to characterize the minimum X-axis offset value that allows users to interact with the position in normalized units;

[0064] Maximum Y-axis coordinate information is used to characterize the standardized unit of the maximum Y-axis offset value that allows users to interact with their location.

[0065] Minimum Y-axis coordinate information, used to characterize the minimum Y-axis offset value in normalized units that allows user positional interaction;

[0066] Maximum Z-axis coordinate information is used to characterize the maximum Z-axis offset value that allows user positional interaction, in standardized units.

[0067] Minimum Z-axis coordinate information, used to characterize the minimum Z-axis offset value in a standardized unit that allows user positional interaction.

[0068] An audio object is a connection between an actual audio track and its format. A unique track identifier establishes a mapping and relationship between audio content, format, and other relevant information. Audio objects can be nested to combine other audio objects. Therefore, an audio object can be a combination of one or more. The unique track identifier element is the metadata used to create the unique track identifier, describing its structural characteristics.

[0069] The audio content describes the content of a component of the audio content (such as background music) and references one or more audio objects to associate the content with its format. The audio content element is essentially the metadata used to create the audio content, describing its structural characteristics.

[0070] The audio program includes narration, sound effects, and background music. Each audio program references one or more audio contents, which are combined to form a complete audio object. The audio program elements are the metadata that creates the audio object, used to describe the structural characteristics of the audio program.

[0071] Track grouping and position identification in audio files; generating audio object metadata (AudioObject) based on audio data in the same group.

[0072] In one implementation, after grouping and identifying the positions of the audio tracks, the method also includes generating unique metadata for each track.

[0073] Based on the audio data corresponding to each group of audio objects, generate unique metadata for the audio track (audioTrackUID).

[0074] This process generates a unique identifier for each audio track, along with track description information. It also generates sub-elements for finding audio material exchange formats, audio track format references, and audio package format references. The track description information includes the track sampling rate and bit depth. The unique track identifier metadata includes: an attribute area containing the unique track identifier information and preset track description information; and a sub-element area containing sub-elements for finding audio material exchange formats, audio track format references, and audio package format references. The process involves acquiring relevant information from the attribute area and sub-element area of ​​the unique track identifier metadata to generate the unique track identifier metadata. The acquired information is then set in the corresponding area of ​​the unique track identifier metadata. The audio package format reference information is the audio package identifier information of the audio package format referenced by the unique track identifier metadata. The audio track format reference information is the audio track identifier information of the audio track format referenced by the unique track identifier metadata. The audio material exchange format lookup sub-elements include MXF package audio track identifier reference information, MXF audio track reference information, and channel audio track reference information.

[0075] In one implementation, after grouping and identifying the positions of the audio tracks, the audio track data is also processed by a renderer.

[0076] Generate audio content metadata based on audio track data;

[0077] The process involves generating audio content metadata (AudioContent) based on information from each audio object. This metadata describes the parameters that constitute the audio content object. The audio content metadata includes an audio content identifier representing identification information about the audio content, and an audio content name representing the name information about the audio content. The audio content metadata also includes reference audio object information, which references one or more audio object elements and associates the content of the audio object element with its format. The audio content metadata may include: an attribute area, containing an audio content identifier and an audio content name; and a sub-element area, containing reference audio object information, which references one or more audio object elements and associates the content of the audio object element with its format. The relevant information from the attribute area and sub-element area of ​​the audio content metadata is obtained to generate the audio content metadata. The obtained information is then set in the corresponding area of ​​the audio content metadata. The attribute area may also include language information for the audio content, which describes the language selected for the audio content. The sub-element area may further include: content measurement loudness information of the audio content, which is used to describe the measured loudness of the audio content; and / or, dialogue element information, which is used to characterize whether the audio program is a dialogue and its corresponding attribute information.

[0078] In one implementation, after generating the audio content metadata, the process also includes generating playable audio:

[0079] Based on the composition of the generated audio content, the program information of the composition is extracted to generate audio program metadata, which describes the attributes of the composed audio content.

[0080] The audio program metadata includes an audio program identifier and an audio program name. The audio program identifier includes audio program information created by referencing one or more audio contents.

[0081] Audio program metadata also includes: audio content reference information, which contains audio content information referenced when the audio program is played. The audio content reference information includes one or more referenced audio content information.

[0082] Audio program metadata may include: an attribute area, including the audio program identifier and audio program name, wherein the audio program identifier includes audio program information created by referencing one or more audio contents; and a sub-element area, including: audio content reference information, wherein the audio content reference information includes audio content information referenced when the audio program is played, wherein the audio content reference information includes one or more referenced audio content information. The relevant information in the attribute area and sub-element area of ​​the audio program metadata is obtained to generate the audio program metadata. The obtained information is then set in the corresponding area of ​​the audio program metadata. The attribute area may also include audio language information indicating the audio program displayed on the display screen during audio program playback. The attribute area may also include start time information indicating the audio program playback, wherein the start time information has at least five digits in seconds. The attribute area also includes end time information indicating the audio program playback, wherein the end time information has at least five digits in seconds. The attribute area may also include maximum allowable dodging information indicating the maximum allowable dodging during audio program playback. The sub-element area may also include program loudness information, wherein the program loudness information is the decibel value of the audio program playback. The sub-element area may also include screen information for audio program playback, which is used to characterize the screen aspect ratio, center point position, and screen width information of the screen displayed when the audio program is played.

[0083] The renderer renders the input audio file as a whole and exports a BW64 (Broadcast WAVE) file containing metadata and audio data. Objects in the ADM XML file are bound to BW64 channels using unique identifiers (such as UUIDs), thus realizing the construction of an audio production model based on the ADM (Audio Definition Model) standard.

[0084] According to embodiments of the present invention, audio tracks with similar functions (such as "background sounds" or "dialogue") can be automatically grouped into the same group, supporting batch adjustment of spatial parameters (such as uniformly setting surround sound diffusion). By identifying the position of the audio track in the original audio (such as the proportion of left and right channels), accurate spatial coordinate data is provided to the ADM to achieve immersive sound field construction. For audio tracks in the same group, dynamic processing (such as compression and limiters) can be uniformly applied to ensure the loudness consistency of audio objects within the group. For example, in game audio, the "ambient sound effects" group can be set with a uniform volume decay curve to avoid sudden volume jumps.

[0085] On the other hand, tracks within the same group share basic metadata (such as copyright information and sampling rate), reducing repetitive input. The generated metadata conforms to the ETSI TS 103 190 specification and supports quick switching between multiple languages ​​and versions of ADM files.

[0086] Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 2 As shown, the electronic device includes a processor 30, a memory 31, an input device 32, and an output device 33. The electronic device may have one or more processors 30. Figure 2 Taking a processor 30 as an example. The electronic device may contain one or more memory units 31. Figure 2 Taking a memory 31 as an example, the processor 30, memory 31, input device 32, and output device 33 of this electronic device can be connected via a bus or other means. Figure 2 Taking a bus connection as an example, the electronic device can be a computer or a server. This application uses an electronic device as a server for detailed description; the server can be a standalone server or a cluster server.

[0087] Memory 31, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as program instructions / modules for generating audio program metadata as described in any embodiment of this application. Memory 31 may primarily include a program storage area and a data storage area, wherein the program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device, etc. In addition, memory 31 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, memory 31 may further include memory remotely located relative to processor 30, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0088] Input device 32 can be used to receive input digital or character information, and to generate key signal inputs related to viewer user settings and function control of electronic devices. It can also be a camera for acquiring images and a sound pickup device for acquiring audio data. Output device 33 may include audio devices such as speakers. It should be noted that the specific composition of input device 32 and output device 33 can be set according to actual conditions.

[0089] The processor 30 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 31, namely generating audio metadata.

[0090] In the description of this specification, references to terms such as "in one embodiment," "in yet another embodiment," "exemplary," or "in a particular embodiment," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0091] Although this application has been described in detail above with general descriptions, specific embodiments, and experiments, some modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of this application fall within the scope of protection claimed in this application.

Claims

1. A method for generating audio content metadata, characterized in that, include: The renderer identifies the audio tracks of the audio file, determines the track grouping of the audio tracks, and identifies the position of the audio tracks in the audio file. The renderer processes the audio track data and generates audio content metadata based on the audio track data.

2. The method for generating audio content metadata according to claim 1, characterized in that, The renderer processes the audio track data and generates audio content metadata based on it, including: Generate audio object metadata based on the track data of the same track group; Audio content metadata is generated based on the information of each audio object.

3. The method for generating audio content metadata according to claim 2, characterized in that, Audio content metadata is generated based on the information of each audio object, including: The audio content metadata is generated based on the information of each audio object. The audio object information is used to reference one or more audio object elements and to associate the content of the audio object element with its format. The audio content identifier that generates the audio content metadata represents the identification information of the audio content, and the audio content name that generates the audio content metadata represents the name information of the audio content.

4. The method for generating audio content metadata according to claim 3, characterized in that, The audio content metadata is set to include an attribute area and a child element area; The attribute area includes the audio content identifier and audio content name; the child element area includes information referencing the audio object.

5. The method for generating audio content metadata according to claim 4, characterized in that, Generate audio content metadata, including: Retrieve relevant information from the audio content metadata attribute area and child element area, and set the retrieved information in the corresponding area of ​​the audio content metadata.

6. The method for generating audio content metadata according to claim 4 or 5, characterized in that, The attribute area also includes: language information of the audio content, which is used to describe the language selected for the audio content.

7. The method for generating audio content metadata according to claim 6, characterized in that, The sub-element area also includes: content measurement loudness information of the audio content, which is used to describe the measured loudness of the audio content; and / or, dialogue element information, which is used to characterize whether the audio program is a dialogue and its corresponding attribute information.

8. An electronic device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method for generating audio content metadata as described in any one of claims 1-7.

Images