Method for generating audio track unique identifier metadata and electronic equipment

Abstract

The invention relates to a method for generating audio track unique identifier metadata and electronic equipment, and belongs to the technical field of audio processing, the method comprises the following steps: identifying an audio track of an audio file through a renderer, and generating an audio track unique identifier corresponding to the audio track; determining an audio track group of the audio track and identifying the position of the audio track in the audio file; and generating audio track unique identification metadata according to the audio data corresponding to each group of audio objects.

Description

[0001] This application is a divisional application of the patent application with application number 202510444067.2, application date 2025-04-09, and invention title "A method for generating audio object metadata and an electronic device". TECHNICAL FIELD

[0002] The present application relates to the technical field of audio processing, and in particular to a method for generating track unique identification metadata and an electronic device. BACKGROUND

[0003] With the development of technology, audio is becoming more and more complex. From the early monaural audio to the evolution of stereo sound, the focus of work is on the correct processing method of left and right channels. But after the emergence of surround sound, the processing process began to become complex. And the surround 5.1 speaker system sorts and constrains multiple channels, and then the surround 6.1 speaker system, the surround 7.1 speaker system, etc. make the audio processing changeable, and correctly deliver the signal to the appropriate speaker to form a mutually related effect. Therefore, as sound becomes more immersive and interactive, the complexity of audio processing has also increased greatly.

[0004] Audio channels (or channels), refers to the independent audio signals collected or played back at different spatial positions during recording or playback. And the number of channels is the number of sound sources during recording or the number of corresponding speakers during playback. For example, in the surround 5.1 speaker system, it includes 6 audio signals at different spatial positions, and each independent audio signal is used to drive the speaker at the corresponding spatial position; in the surround 7.1 speaker system, it includes 8 audio signals at different spatial positions, and each independent audio signal is used to drive the speaker at the corresponding spatial position.

[0005] However, the effect of the current speaker system depends on the number and spatial position of the speakers. For example, a two-channel speaker system cannot achieve the effect of a surround 5.1 speaker system. The speaker system of the prior art adapts to a fixed input format by fixing the output channel, and cannot flexibly adapt to the complex track grouping requirements of different surround speaker systems. SUMMARY

[0006] The purpose of the present application is to provide a method for generating track unique identification metadata and an electronic device to solve the above technical problems.

[0007] To achieve the above purpose, the first aspect of the present application provides a method for generating track unique identification metadata, comprising:

[0008] identifying the track of the audio file by the renderer, and generating the track unique identification corresponding to the track;

[0009] determining a track grouping of the audio track and identifying a location of the audio track in the audio file;

[0010] generating track unique identification metadata according to audio data corresponding to each group of audio objects.

[0011] The second aspect of the present application provides an electronic device, comprising: a memory and one or more processors;

[0012] The memory is configured to store one or more programs.

[0013] When the one or more programs are executed by the one or more processors, the one or more processors perform the method for generating track unique identification metadata provided by any embodiment of the present application.

[0014] Track grouping, location identification in the audio file, and generating audio object metadata according to audio data of the same grouping.

[0015] Generating track unique identification metadata according to audio data corresponding to each group of audio objects.

[0016] The renderer processes track data, generates audio content metadata according to information of each audio object, and describes parameters of the constituent content audio objects.

[0017] According to the generated audio content composition, extracting program information of the composition, generating audio program metadata AudioProgramme, and describing attributes of the constituent audio content. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 A flowchart of a method for generating audio object metadata is provided in the embodiments of the present application.

[0019] Figure 2 A structural schematic diagram of an electronic device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0020] The following embodiments are used to illustrate the present application, but not to limit the scope of the present application.

[0021] Metadata is information describing the structural characteristics of data, and the functions supported by metadata include indicating storage location, historical data, resource search, or file record. Metadata is made for each stage of the multi-dimensional sound model to describe the characteristics of the stage.

[0022] EMBODIMENTS

[0023] The present application provides a method for generating audio object metadata in a multi-dimensional sound model, and is described in detail. As shown in FIG. 1, the multi-dimensional sound model comprises a plurality of audio objects, and each audio object is composed of a plurality of audio tracks. Figure 1A flowchart of a method for generating audio object metadata is shown, the method comprising:

[0024] Step 110, inputting an audio file into a renderer;

[0025] Step 120, identifying audio tracks of the audio file by the renderer, and generating track unique identifiers corresponding to the audio tracks;

[0026] Step 130, determining audio track grouping of the audio tracks and identifying positions of the audio tracks in the audio file;

[0027] Extracting a predefined surround system identifier from a header of the audio file, directly matching a preset grouping template, and being compatible with non-standardized audio files through speaker layout configuration or custom configuration.

[0028] Step 140, generating audio object metadata according to audio track data of the same audio track grouping.

[0029] Optionally, determining the audio track grouping of the audio tracks and identifying the positions of the audio tracks in the audio file comprises:

[0030] Determining the audio track grouping of the audio tracks and the positions of the audio tracks according to a surround speaker system version of the audio file.

[0031] Specifically, the audio tracks are grouped according to L / R / C / LFE / Ls / Rs, and are associated with preset polar coordinate positions (such as a default azimuth angle of 110° for a left surround channel Ls). For an audio object to be processed, an independent audio object is generated by establishing a mapping and a connection of audio content, format and other related information through a track unique identifier. A sound field simulation algorithm (such as VBAP, Vector Base Amplitude Panning) is used to calculate normalized coordinates (X, Y, Z) of the audio tracks, and the normalized coordinates are written into a position (spatial position) field of the metadata, so as to ensure that a renderer accurately calculates speaker driving gain. Optionally, generating the audio object metadata according to the audio track data of the same audio track grouping comprises:

[0032] Generating track unique identifier reference information in the audio object metadata according to the audio track data of the same audio track grouping;

[0033] Generating an audio object identifier and an audio object name of the audio object, wherein the audio object identifier comprises information indicating a relationship between a plurality of audio objects.

[0034] Optionally, generating the audio object metadata according to the audio track data of the same audio track grouping further comprises:

[0035] Determining start time information and duration information of the audio object according to the audio track data of the same audio track grouping.

[0036] The start time information of the audio object is related to a play start time of the audio program in the audio object. The duration information of the audio object is related to a play start time and a play end time of the audio program in the audio object.

[0037] Optionally, before the track of the audio file is identified by the renderer and the unique track identifier corresponding to the track is generated, the method further comprises:

[0038] The type of the audio in the audio file is identified by the renderer.

[0039] Optionally, the method for generating the audio object metadata further comprises:

[0040] The audio object importance information, the user-object interaction information, and the automatic dodging information in the audio object metadata are determined by identifying the audio file by the renderer.

[0041] The audio object importance information is used to describe an importance index of the audio object. The user-object interaction information is used to represent whether the user is allowed to interact with the object. The automatic dodging information is used to represent whether the object is allowed to automatically dodge.

[0042] Specifically, in the process of whether the object is allowed to automatically dodge, a priority parameter (for example, a dialogue track priority = 10 and background music = 5) is defined, and when a high-priority object is activated, the gain of other objects is automatically reduced. By using a Boolean value (allow_gain_adjust) and a gain (gain_range, -12 dB ~ +6 dB) field, it is controlled whether the user can adjust the volume and the adjustment range. A preset position offset threshold (for example, an azimuth angle ± 30° and a distance of 0.5~1.5 times) is provided in the metadata to prevent the user from dragging the object to an invalid sound field area. By using the above parameters, the metadata not only describes the static properties of the audio object, but also defines dynamic behavior rules, so that the renderer can adjust the output strategy in real time according to the user operation or environmental changes (for example, the increase or decrease of the loudspeaker).

[0043] Optionally, the method for generating the audio object metadata further comprises:

[0044] The audio packet format identification reference information, the audio object identification reference information, the audio complementary object identification reference information, and the audio object interaction information in the audio object metadata are determined by identifying the audio file by the renderer.

[0045] The audio packet format identification reference information is used for referencing the audio packet format identification for format description. The audio object identification reference information is used for describing the referenced other audio object. The audio complementary object identification reference information is used for describing the audio object complementary to the audio object. The audio object interaction information is used for representing the specification of the user-object interaction.

[0046] Optionally, the audio object interaction information comprises:

[0047] Audio object switch information, which is used for representing whether the audio object is allowed to be opened or closed;

[0048] Audio object gain information, which is used for representing whether the gain of the audio object is allowed to be changed;

[0049] Audio object position information, which is used for representing whether the position of the audio object is allowed to be changed by the user.

[0050] The audio object gain information comprises:

[0051] Maximum audio gain information, which is used for representing the maximum gain factor allowed for the user gain interaction;

[0052] Minimum audio gain information, which is used for representing the minimum gain factor allowed for the user gain interaction.

[0053] Optionally, the audio object position information comprises:

[0054] Polar coordinate parameter and Cartesian coordinate parameter, the polar coordinate parameter is used for representing the position information of the audio object in the polar coordinate mode, and the Cartesian coordinate parameter is used for representing the position information of the audio object in the Cartesian coordinate mode.

[0055] Optionally, the polar coordinate parameter comprises:

[0056] Maximum azimuth information, which is used for representing the maximum azimuth offset value allowed for the user position interaction;

[0057] Minimum azimuth information, which is used for representing the minimum azimuth offset value allowed for the user position interaction;

[0058] Maximum elevation information, which is used for representing the maximum elevation offset value allowed for the user position interaction;

[0059] Minimum elevation information, which is used for representing the minimum elevation offset value allowed for the user position interaction;

[0060] Maximum distance information, which is used for representing the maximum normalized distance value allowed for the user position interaction;

[0061] Minimum distance information for characterizing a minimum normalized distance value allowing user position interaction.

[0062] Optionally, the Cartesian coordinate parameter comprises:

[0063] Maximum X-axis coordinate information for characterizing a maximum X-axis offset value allowing user position interaction in normalized units.

[0064] Minimum X-axis coordinate information for characterizing a minimum X-axis offset value allowing user position interaction in normalized units.

[0065] Maximum Y-axis coordinate information for characterizing a maximum Y-axis offset value allowing user position interaction in normalized units.

[0066] Minimum Y-axis coordinate information for characterizing a minimum Y-axis offset value allowing user position interaction in normalized units.

[0067] Maximum Z-axis coordinate information for characterizing a maximum Z-axis offset value allowing user position interaction in normalized units.

[0068] Minimum Z-axis coordinate information for characterizing a minimum Z-axis offset value allowing user position interaction in normalized units.

[0069] An audio object is a connection between an actual audio track and its format, the mapping and linking of audio content, format and other related information is established by a track unique identifier, and an audio object can be self-nested to combine other audio objects together. Therefore, an audio object can be one or more combinations. The track unique identifier element is also the production track unique identifier, and the metadata for generating the track unique identifier is used to describe the structural characteristics of the track unique identifier.

[0070] The audio content describes the content of a component of the audio content (such as background music), and references one or more audio objects to link the content with its format. The audio content element is also the production audio content, and the metadata for generating the audio content is used to describe the structural characteristics of the audio content.

[0071] The audio programme (audioProgramme) includes narration, sound effects and background music, and the audio programme references one or more audio contents, which are combined together to form a complete audio object. The audio programme element is also the production audio object, and the metadata for generating the audio programme is used to describe the structural characteristics of the audio programme.

[0072] Track grouping, position identification in an audio file, and audio object metadata (AudioObject) generated according to the same grouping audio data.

[0073] In an implementation, after grouping and identifying the audio tracks, generating audio track unique identification metadata is further included.

[0074] According to the audio data corresponding to each group of audio objects, generating audio track unique identification metadata (audioTrackUID).

[0075] The audio track unique identification corresponding to the audio track is generated, and audio track description information is also generated. An audio material exchange format lookup sub-element, audio track format reference information, and audio packet format reference information are also generated. The audio track description information includes audio track sampling rate and audio track bit depth. The audio track unique identification metadata includes an attribute area including audio track unique identification information and preset audio track description information, and a sub-element area including the audio material exchange format lookup sub-element, the audio track format reference information, and the audio packet format reference information. The relevant information of the attribute area and the sub-element area of the audio track unique identification metadata is obtained to generate the audio track unique identification metadata. That is, the obtained information is set in the corresponding area of the audio track unique identification metadata. The audio packet format reference information is audio packet identification information of an audio packet format referenced by the audio track unique identification metadata. The audio track format reference information is audio track identification information of an audio track format referenced by the audio track unique identification metadata. The audio material exchange format lookup sub-element includes MXF packet track identification reference information, MXF track reference information, and channel track reference information.

[0076] In an implementation, after grouping and identifying the audio tracks, processing the audio track data through a renderer is further included.

[0077] Generating audio content metadata according to the audio track data;

[0078] The audio content metadata (AudioContent) is generated according to the information of each audio object, which describes the parameters of the content audio object. The audio content identifier of the audio content metadata represents the identification information of the audio content, and the audio content name represents the name information of the audio content; the audio content metadata further includes reference audio object information, which is used to reference one or more audio object elements and link the content of the audio object elements with their formats. The audio content metadata can include: an attribute area including the audio content identifier and the audio content name of the audio content, the audio content identifier representing the identification information of the audio content, and the audio content name representing the name information of the audio content; a sub-element area including reference audio object information, which is used to reference one or more audio object elements and link the content of the audio object elements with their formats. The relevant information of the attribute area and the sub-element area of the audio content metadata is obtained to generate the audio content metadata. That is, the obtained information is set in the corresponding area of the audio content metadata, wherein the attribute area can further include: language information of the audio content, which is used to describe the selected language of the audio content. The sub-element area can further include: content measurement loudness information of the audio content, which is used to describe the measurement loudness of the audio content; and / or dialogue element information, which is used to represent whether the audio program is a dialogue and the corresponding attribute information.

[0079] In an implementation manner, after the audio content metadata is generated, the playable audio is further generated:

[0080] According to the generated audio content composition, the program information of the composition is extracted to generate audio program metadata (AudioProgramme), which describes the attributes of the composed audio content.

[0081] The audio program metadata includes an audio program identifier and an audio program name, and the audio program identifier includes audio program information created by referencing one or more audio contents.

[0082] The audio program metadata further includes audio content reference information, which includes audio content information referenced when the audio program is played, and the audio content reference information includes the information of one or more referenced audio contents.

[0083] The audio program metadata can include: an attribute area including an audio program identification and an audio program name of the audio program, the audio program identification including audio program information created by referencing one or more audio contents; and a sub-element area including: audio content reference information including audio content information referenced during audio program playback, the audio content reference information including one or more pieces of audio content information referenced. Related information in the attribute area and the sub-element area of the audio program metadata is acquired to generate the audio program metadata. The acquired information is set in the corresponding area of the audio program metadata, wherein the attribute area can further include audio language information of the audio program displayed on a display screen during audio program playback. The attribute area can further include start time information of the audio program playback, the start time information having a number of seconds of no less than five digits. The attribute area can further include end time information of the audio program playback, the end time information having a number of seconds of no less than five digits. The attribute area can further include maximum allowed avoidance information indicating maximum allowed avoidance during audio program playback. The sub-element area can further include program loudness information of the audio program playback, the program loudness information being a decibel value of the audio program playback. The sub-element area can further include screen information of the audio program playback, the screen information being used to represent a screen aspect ratio, a center point position, and screen width information of the display screen during audio program playback.

[0084] The renderer forms an overall rendering broadcast for the input audio file, and exports a BW64 (Broadcast WAVE) file of "metadata + audio data", and binds the object in the ADM XML file to the BW64 channel through a unique identifier (such as a UUID), to realize construction of an ADM (Audio Definition Model) standard audio production model.

[0085] According to embodiments of the present application, similar-function audio tracks (such as "background sound" or "dialogue") can be automatically grouped into the same group, and batch adjustment of spatial parameters (such as uniform setting of surround sound diffusion) is supported. By identifying the position of the audio track in the original audio (such as the proportion of left and right channels), precise spatial coordinate data is provided for the ADM, to realize immersive sound field construction. For audio tracks in the same group, dynamic processing (such as compression, limiter) can be uniformly applied, to ensure the loudness consistency of audio objects in the group. For example, in game audio, the "environmental sound effect" group can be set with a uniform volume decay curve, to avoid sudden volume jumps.

[0086] On the other hand, audio tracks in the same group share basic metadata (such as copyright information, sampling rate), to reduce the workload of repeated input. The generated metadata conforms to the ETSI TS 103 190 specification, and supports fast switching of multi-language and multi-version ADM files.

[0087] Figure 2 A structural schematic diagram of an electronic device is provided in the embodiments of the present application. As shown in the figure, the electronic device comprises a processor 30, a memory 31, an input device 32 and an output device 33. The number of processors 30 in the electronic device can be one or more, Figure 2 The number of memories 31 in the electronic device can be one or more, Figure 2 The processor 30 in the electronic device is taken as an example. The number of memories 31 in the electronic device can be one or more, Figure 2 The processor 30, the memory 31, the input device 32 and the output device 33 of the electronic device can be connected through a bus or other means, Figure 2 The electronic device can be a computer and a server, etc. The embodiments of the present application take the electronic device as a server for detailed description, which can be a stand-alone server or a cluster server.

[0088] The memory 31 as a computer readable storage medium can be used to store software programs, computer executable programs and modules, such as program instructions / modules for generating audio program metadata according to any embodiments of the present application. The memory 31 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required by at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 31 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some examples, the memory 31 can further include a memory remotely arranged with respect to the processor 30, which can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0089] The input device 32 can be used to receive input digital or character information, and to generate key signal input related to the audience user settings and function control of the electronic device, and can also be a camera for acquiring images and a pickup device for acquiring audio data. The output device 33 can include an audio device such as a loudspeaker. It should be noted that the specific composition of the input device 32 and the output device 33 can be set according to the actual situation.

[0090] The processor 30 executes various function applications and data processing of the device by running the software programs, instructions and modules stored in the memory 31, i.e. generates metadata of audio.

[0091] In the description of the application, reference to a term "in an embodiment", "in a further embodiment", "exemplary" or "in a specific embodiment" or similar means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included within at least one embodiment or example of the application. The illustrative descriptions of the terms do not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0092] While the application has been described in detail and with reference to specific embodiments thereof, it will be apparent to one skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the application. Thus, it is intended that the application cover the modifications and variations of this application provided they come within the scope of the appended claims and their equivalents.

Claims

1. A method for generating unique identifier metadata for audio tracks, characterized in that, include: The renderer identifies the audio tracks of the audio file and generates a unique identifier for each track. Determine the audio track grouping and identify the position of the audio track in the audio file; Generate unique metadata for each audio track based on the audio data corresponding to each audio object.

2. The method for generating unique identifier metadata for audio tracks according to claim 1, characterized in that, Generate unique metadata for each audio track, including: It generates a unique identifier for each audio track, as well as audio track description information; it also generates sub-elements for finding audio material exchange formats, audio track format reference information, and audio package format reference information.

3. The method for generating unique identifier metadata for audio tracks according to claim 2, characterized in that, The audio track description information includes: audio track sampling rate and audio track bit depth.

4. The method for generating unique identifier metadata for audio tracks according to claim 1, characterized in that, The unique identifier metadata for the audio track is set to include both the attribute area and the child element area; The attribute area includes unique audio track identifiers and preset audio track descriptions; the sub-element area includes sub-elements for finding audio material exchange formats, audio track format references, and audio package format references.

5. The method for generating unique identifier metadata for audio tracks according to claim 4, characterized in that, Based on the audio data corresponding to each group of audio objects, generate unique metadata for each track, including: Retrieve relevant information from the unique identifier metadata attribute area and child element area of ​​the audio track, set the retrieved information in the corresponding area of ​​the unique identifier metadata of the audio track, and generate the unique identifier metadata of the audio track.

6. The method for generating unique identifier metadata for audio tracks according to claim 4 or 5, characterized in that, The audio package format reference information is the audio package identifier information of the audio package format referenced by the unique identifier metadata of the audio track; the audio track format reference information is the audio track identifier information of the audio track format referenced by the unique identifier metadata of the audio track; the audio material exchange format lookup sub-element includes MXF package audio track identifier reference information, MXF audio track reference information, and channel audio track reference information.

7. An electronic device, characterized in that, include: Memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the method for generating unique audio track identifier content metadata as described in any one of claims 1-6.

Images