Audio metadata processing device and program

Abstract

To convert acoustic metadata in which dynamic metadata is described in time series for each time in an object base audio and delete the data amount.SOLUTION: An acoustic metadata processor 1 includes: an overall determination unit 11 for comparing acoustic metadata of a specific time with acoustic metadata before the specific time, and performing an overall determination as to whether the acoustic metadata of the time is to be integrated with the acoustic metadata before the time; and an acoustic metadata combining unit 12 for combining the acoustic metadata by integrating the acoustic metadata of the time with the acoustic metadata before the time on the basis of the overall determination or by adding the acoustic metadata of the time to the acoustic metadata before the time instead of integrating the acoustic metadata of the time with the acoustic metadata before the time.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an audio metadata processing apparatus and a program for converting audio metadata in object-based audio. [Background technology]

[0002] Object-based audio is an audio method that records and transmits audio objects and audio metadata, and plays (renders) content in a format appropriate for the playback environment. It has become increasingly popular in the film industry as a method for playing back 3D audio, but in recent years, there have been moves to use object-based audio in next-generation broadcasting in various countries around the world. For example, the United States and South Korea, which have adopted the ATSC3.0 broadcasting format, have begun introducing it into their broadcasting systems using AC-4 or MPEG-H 3DA as the audio encoding format.

[0003] ITU-R has defined the Audio Definition Model (ADM) as an international standard for audio metadata for program production (see, for example, Non-Patent Document 1). In the ADM, the combination of audio objects that make up the program audio, speaker arrangement, playback position and gain of the audio objects, etc. are described in XML (Extensible Markup Language). Metadata whose values ​​change over time, such as the playback position and gain of objects, is called dynamic metadata, and is sometimes handled separately from static metadata that does not change throughout the program, such as the combination of audio objects.

[0004] When object-based audio is used in broadcasting, streaming services, etc., audio metadata must be repeatedly transmitted so that playback can be started from any time when reception began. For this reason, a time-series representation format of ADM (Serial ADM: S-ADM) (see, for example, Non-Patent Document 2) and a transmission method that synchronizes S-ADM with audio signals and outputs them using existing audio digital interfaces (see, for example, Non-Patent Document 3) have been internationally standardized. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] ITU-R, Rec. ITU-R BS.2076-2, “Audio definition model”, 2019 [Non-Patent Document 2] ITU-R, Rec. ITU-R BS.2125-0, “A serial representation of the Audio Definition Model”, 2019 [Non-Patent Document 3] SMPTE, SMPTE ST 2116:2019, “Format for Non-PCM Audio and Data in AES3 - Carriage of Metadata of Serial ADM (Audio Definition Model)”, 2020 Summary of the Invention [Problem to be solved by the invention]

[0006] In live production using object-based audio, it is expected that production equipment such as audio consoles will automatically generate S-ADMs sequentially based on the operations of the mixing engineer. In the S-ADM generated here, the static metadata is always written to the same content, whereas the dynamic metadata is periodically obtained from values ​​such as the playback position and gain adjusted by the audio console, and new values ​​are written for each frame (each S-ADM transmission cycle). The dynamic metadata is written as a value corresponding to the time of the relevant frame in the S-ADM, and the dynamic metadata for each frame of the S-ADM is treated as different metadata.

[0007] When recording and saving object-based audio sources, it is necessary to record audio metadata along with the audio signal. When a recorder compatible with object-based audio saves the audio signal as a file, it is expected that it will reconstruct and record an ADM that describes all the information about the entire program from the S-ADM that was input along with the audio signal. When recording audio metadata that is automatically generated by production equipment such as the audio console mentioned above, static metadata is recorded once and does not need to be recorded again until the program changes, whereas dynamic metadata requires new descriptions to be added for each frame, which can result in a huge amount of data to be recorded.

[0008] In consideration of the above circumstances, an object of the present invention is to provide an audio metadata processing device and program that can reduce the amount of audio metadata data in which dynamic metadata for each time is described in chronological order in object-based audio. [Means for solving the problem]

[0009] In order to solve the above problems, the acoustic metadata processing device of the present invention is an acoustic metadata processing device that combines acoustic metadata in which playback conditions for audio objects for each time are described in chronological order in object-based audio, and is equipped with an integration determination unit that compares the acoustic metadata at the time with the acoustic metadata up to the time prior to that time and makes an integration determination as to whether or not to integrate the acoustic metadata at the time with the acoustic metadata up to the time prior to that time, and an acoustic metadata combination unit that combines the acoustic metadata by integrating the acoustic metadata at the time with the acoustic metadata up to the time prior to that time based on the integration determination, or by adding the acoustic metadata at the time to the acoustic metadata up to the time prior to that time without integrating it.

[0010] Furthermore, in the acoustic metadata processing device of the present invention, the integration judgment unit determines whether to integrate by comparing the rate or amount of change in the playback position of the same audio object in the acoustic metadata at the time and the acoustic metadata immediately before the time with a threshold value.

[0011] Furthermore, in the acoustic metadata processing device of the present invention, the integration determination unit determines whether to integrate by comparing the relative error of the rate of change or the absolute error of the amount of change in the acoustic metadata at the time and the acoustic metadata immediately before the time with a threshold value.

[0012] Furthermore, in the acoustic metadata processing device of the present invention, when the acoustic metadata at a given time is integrated with the acoustic metadata up to the time before that time, and the change in the playback position of an audio object described in the acoustic metadata exceeds a change threshold, the acoustic metadata combination unit divides the audio object within a range not exceeding the change threshold.

[0013] Furthermore, in the acoustic metadata processing device of the present invention, the acoustic metadata combination unit integrates acoustic metadata where there is a change in the playback position of an audio object just before that time, but there is no change in the playback position of the same audio object at that time, by using a complementary time of a change in playback position, which can be described so that the change is continuous when describing a change in the playback position of an audio object.

[0014] In order to achieve the above object, a program according to the present invention causes a computer to function as the above audio metadata processing device. Effect of the Invention

[0015] According to the present invention, it is possible to reduce the amount of audio metadata data in which dynamic metadata for each time is described in chronological order in object-based audio. [Brief description of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the configuration of an acoustic metadata processing device according to a first embodiment. [Diagram 2] 4 is a flowchart showing an example of a processing procedure in the acoustic metadata processing device according to the first embodiment. [Diagram 3]5 is a flowchart showing an example of a process for deriving a gradient of a change in a playback position at the end of a previous block in the acoustic metadata processing device according to the first embodiment. [Figure 4] 6 is a flowchart showing an example of a process for deriving a gradient of a change in a playback position at the start point of a block in the acoustic metadata processing device according to the first embodiment. [Diagram 5] 4 is a flowchart showing an example of a block combining method in the acoustic metadata processing device according to the first embodiment. [Figure 6] 13 is a table showing an example of setting main parameters at the time of block integration. [Figure 7] 4 is a diagram showing an example of dividing an integrated block in the acoustic metadata processing device according to the first embodiment. FIG. [Figure 8] FIG. 11 is a block diagram showing an example of the configuration of an acoustic metadata processing device according to a second embodiment. [Figure 9] 10 is a flowchart showing an example of a processing procedure in an acoustic metadata processing device according to a second embodiment. [Figure 10] 10 is a flowchart showing an example of a block combining method in the acoustic metadata processing device according to the second embodiment. [Figure 11] A diagram showing a first example of an S-ADM. [Figure 12] FIG. 11 is a diagram showing an example of an ADM generated from the S-ADM shown in FIG. 10 by a conventional method. [Figure 13] 11 is a diagram showing an example of an ADM generated from the S-ADM shown in FIG. 10 by the acoustic metadata processing device according to the first embodiment. [Figure 14] A figure showing a second example of S-ADM. [Figure 15] 15 is a diagram showing an example of an ADM generated from the S-ADM shown in FIG. 14 by the acoustic metadata processing device according to the first embodiment. FIG. [Figure 16] A figure showing a third example of S-ADM. [Figure 17]17 is a diagram showing a first example of an ADM generated from the S-ADM shown in FIG. 16 by the acoustic metadata processing device according to the first embodiment. FIG. [Figure 18] 17 is a diagram showing a second example of an ADM generated from the S-ADM shown in FIG. 16 by the acoustic metadata processing device according to the first embodiment. FIG. [Figure 19] 15 is a diagram showing an example of an ADM generated from the S-ADM shown in FIG. 14 by an acoustic metadata processing device according to the second embodiment. FIG. [Figure 20] 17 is a diagram showing an example of an ADM generated from the S-ADM shown in FIG. 16 by an acoustic metadata processing device according to the second embodiment. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] The audio metadata processing device described below as an embodiment inputs audio metadata in S-ADM format and stores it as audio metadata in ADM format. First, an outline of the ADM and the description method of S-ADM related to the present invention will be given. However, the format of the input audio metadata is not limited to this. For example, if an ADM is reconstructed from an S-ADM without reducing the amount of metadata data by some object-based audio recording device, the input may also be an ADM. In addition, by sequentially converting the audio metadata integrated by the algorithm of the present invention into the S-ADM format and outputting it, it is possible to insert it into a system of production equipment as an S-ADM converter.

[0018] In ADM and S-ADM, each audio channel, such as an audio object whose playback position changes, is described using a descriptor called audioChannelFormat. Specific parameter changes such as playback position and gain are expressed by further dividing audioChannelFormat into multiple audio blocks called audioBlockFormat (hereinafter simply referred to as "blocks") in the time direction. Information about the start time and duration of the block, as well as the playback position at that time (azimuth, elevation, distance, etc.) and parameters such as gain are described in audioBlockFormat, and a new audioBlockFormat is added every time a parameter changes. However, with regard to the playback position, the parameters specified in each block are the parameters at the end of each block, and if there is a change in the playback position compared to the previous block, it is basically interpreted as a linear change in the playback position from the parameters of the previous block to the parameters of the block over the time length of the block.

[0019] If you want the time it takes for the change to be shorter than the block length, write jumpPosition, which is an option for audioBlockFormat. jumpPosition can also have its optional interpolationLength written in it. If the value of jumpPosition is 1 and interpolationLength is also written, the change will take place over the length of time of interpolationLength instead of the length of time of the block. If the value of jumpPosition is 1 and there is no description of interpolationLength, the change will take place instantly at the start of the block.

[0020] The duration of an audioBlockFormat does not need to be constant, and if the parameters do not change, the audioBlockFormat will not be split. However, in a situation where the production facility automatically generates S-ADMs in sequence, the dynamic metadata for each frame is treated as different metadata as described above, so it is expected that a new audioBlockFormat with the same duration as the frame length will be generated for each frame. Conversely, if the parameters change frequently in a time shorter than the frame length, multiple audioBlockFormats may be included in one frame.

[0021] Generally, a change in the playback position cannot be known without comparing it with the parameters of the previous block, so in S-ADM, the block corresponding to the time of the S-ADM is sent together with the block immediately before it or an initialization block so that rendering can be performed regardless of the frame from which reception begins. An initialization block is a special block that does not have a duration and is only available in S-ADM and describes the parameters at the start time of the block in question.

[0022] Furthermore, in S-ADM, the time reference point of audioBlockFormat can be roughly divided into two ways of describing: either using the start time of the program (audioProgramme) as the starting point, or using the start time of the S-ADM frame that contains the block as the starting point. The former is a description method common to ADM, and since it is easy to understand the relationship between metadata and time at each time, it is expected to be used preferentially in situations where ADM can be created in advance. On the other hand, in situations such as live production where production equipment such as an audio console automatically generates S-ADMs one after another, the latter description method is expected to be used preferentially, since using the start time of the frame as the starting point simplifies time calculations when processing each frame of S-ADM.

[0023] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0024] <First embodiment> Fig. 1 is a block diagram showing an example of the configuration of an acoustic metadata processing device 1 according to a first embodiment. The acoustic metadata processing device 1 shown in Fig. 1 includes a starting point conversion unit 10, an integrated judgment unit 11, an acoustic metadata combination unit 12, and an acoustic metadata storage unit 13. The acoustic metadata processing device 1 may also include an acoustic metadata output unit that outputs metadata stored in the acoustic metadata storage unit 13 as a processed ADM at any point in time.

[0025] The audio metadata processing device 1 is a device that reduces the amount of audio metadata data by combining (integrating or adding) audio metadata in which playback conditions (dynamic metadata) of audio objects for each time are described in chronological order in object-based audio. In this specification, "combining" is a concept that includes both "integrating" and "adding," which will be explained in detail below.

[0026] The origin conversion unit 10 checks the time description method of the S-ADM input to the audio metadata processing device 1, and if the time origin of the audioBlockFormat is the start time of the S-ADM frame, converts it to a description method that uses the start time of the audioProgramme as the origin to enable combination with audio metadata received in other frames. The origin conversion unit 10 then outputs the audio metadata with the converted time notation to the integration determination unit 11 and the audio metadata combination unit 12.

[0027] The integration judgment unit 11 compares the acoustic metadata at a certain time (a newly input S-ADM) with the acoustic metadata up to that time (ADM that has been combined up to the previous frame and stored in the acoustic metadata storage unit 13). Then, based on the comparison, the integration judgment unit 11 makes an integration judgment as to whether or not to combine the acoustic metadata at that time with the acoustic metadata up to that time, and outputs judgment information indicating the judgment result to the acoustic metadata combination unit 12.

[0028] For example, the integration judgment unit 11 performs integration judgment by comparing parameters such as gain and playback position of the same audio object in the audio metadata at the time and the audio metadata immediately before the time, particularly the rate or amount of change of the playback position. The integration judgment unit 11 may determine to integrate when the rate of change of the playback position is equal to or less than a threshold, and may determine not to integrate when the rate of change of the playback position exceeds the threshold. Here, the threshold may be a predetermined absolute error, or a relative error determined by the ratio of the rate of change at the time to the rate of change immediately before.

[0029] The acoustic metadata combining unit 12 combines the acoustic metadata at the time with the acoustic metadata up to the time before the time based on the judgment information input from the integration judgment unit 11, or adds the acoustic metadata at the time to the acoustic metadata up to the time without integrating it, thereby combining the acoustic metadata into one. That is, when the judgment information input from the integration judgment unit 11 indicates that the integration is possible, the acoustic metadata combining unit 12 combines the newly received acoustic metadata with the acoustic metadata already stored in the acoustic metadata storage unit 13 to generate new acoustic metadata. When the judgment information input from the integration judgment unit 11 indicates that the integration is not possible, the acoustic metadata combining unit 12 adds the newly received acoustic metadata to the acoustic metadata already stored in the acoustic metadata storage unit 13 to generate new acoustic metadata. Then, the acoustic metadata combining unit 12 outputs the combined acoustic metadata to the acoustic metadata storage unit 13.

[0030] The acoustic metadata combining unit 12 may include a dividing unit 121. When the change in the playback position of the audio object described in the acoustic metadata exceeds a change threshold due to integrating the acoustic metadata at the time with the acoustic metadata up to the time before the time, the dividing unit 121 divides the audio object within a range not exceeding the change threshold. This makes it possible to prevent an object-based audio playback device that has read the converted acoustic metadata from generating a playback signal that differs from the production intention. A specific example will be described later.

[0031] FIG. 2 is a flowchart showing an example of a processing procedure of the acoustic metadata processing device 1. As shown in FIG.

[0032] In step S11, if the time origin of audioBlockFormat is the start time of an S-ADM frame, origin conversion unit 10 converts it into a description method that uses the start time of audioProgramme as the origin.

[0033] In step S12, the integrated judgment unit 11 starts a comparison process from the block with the smallest number (the block with the earliest time) among the blocks that have not yet been compared, excluding the initialization block.

[0034] In step S13, the integration determination unit 11 determines whether the block is the first block of the audioChannelFormat. If the block is the first block of the audioChannelFormat (step S13-Yes), the block before integration does not exist, so it is determined that integration is not possible and the process proceeds to step S21. If the block is other than the first block of the audioChannelFormat (step S13-No), the process proceeds to step S14.

[0035] Here, the audio metadata processing device 1 may generate the first block based on the received block so that audio metadata can be converted to the extent possible even if the S-ADM is received halfway through, or may treat the first block at the start of S-ADM reception as the same as the first block, and start combining blocks received thereafter. In the latter case, the same processing can be applied by reading "first block" as "first block at the start of reception" in the subsequent processing. As described above, in S-ADM, the block corresponding to the time of the S-ADM is transmitted together with the block immediately before it or the initialization block. When the first S-ADM reception starts, if the immediately previous block is also received, that block is processed as the first block, and if the initialization block is also received, a block with duration=0 using the parameters of the initialization block is generated and used as the first block.

[0036] In step S14, the integration judgment unit 11 compares the parameters of the block and the previous block to judge whether there are any different parameters other than the playback position. If there are any different parameters other than the playback position, such as gain (step S14-Yes), it is determined that the blocks are not combineable because multiple values ​​cannot be described in a single audioBlockFormat, and the process proceeds to step S21. If there are no different parameters other than the playback position, that is, if the parameter that differs between the block and the previous block is only the playback position, or if all parameters are the same (step S14-No), the process proceeds to step S15. Here, even if there are different parameters other than the playback position, such as gain, if the change is small compared to the previous block (for example, if the gain changes from 0.700 to 0.705), the data amount reduction of the audio metadata may be prioritized, and the process may proceed to step S15 as if there was no change in the gain. As with video and audio compression, depending on the need to reduce the data amount, audio metadata may also be lossy compressed.

[0037] In step S15, the rate of change (slope of change) of the playback position at the end of the previous block is calculated by the integrated judgment unit 11. The process of step S15 will be described later in detail.

[0038] In step S16, the integrated judgment unit 11 judges whether or not the slope of the change calculated in step S15 is 0. If the slope of the change in the playback position at the end of the previous block is 0 (step S16-Yes), the process proceeds to step S17, and if the slope is not 0 (step S16-No), the process proceeds to step S18.

[0039] In step S17, the playback position of the block in question and the previous block are compared by the integration judgment unit 11. If the playback positions are the same, that is, if the parameters of both blocks are the same (step S17-Yes), it is determined that integration is possible and the process proceeds to step S20. If the parameters of both blocks are not the same (step S17-No), it is determined that integration is not possible and the process proceeds to step S21.

[0040] In step S18, the gradient of change in the playback position at the start point of the block is calculated by the integrated judgment section 11. The process of step S18 will be described later in detail.

[0041] In step S19, the integration determination unit 11 compares the slope at the end of the previous block with the slope at the start of the current block. If the slopes match (step S19-Yes), it is determined that integration is possible and the process proceeds to step S20. If the slopes differ (step S19-No), it is determined that integration is not possible and the process proceeds to step S21.

[0042] In step S19, the criteria for determining whether the slopes match should be set to a certain degree, taking into consideration calculation errors and the accuracy of reading parameters of the production equipment. For example, if the slope ω n-1 end and the slope of the playback position change at the start of the block ω n startIf the relative error R [%] between the two is equal to or less than a threshold α as shown in the following formula, it may be determined that the slopes match.

[0043]

number

[0044] When the threshold value α is set to a small value, only blocks with exactly the same slope are integrated, and an ADM is generated in which the change in playback position is more consistent with the S-ADM generated at the time of production, but the efficiency of data reduction is expected to decrease. On the other hand, when the threshold value α is set to a large value, blocks with somewhat different slopes are also integrated, and an ADM in which the change in playback position differs from the S-ADM generated at the time of generation in the strict sense may be generated, but this may contribute to reducing the data amount. In this way, there is a trade-off between the strictness of the change in playback position and the efficiency of data reduction, and the range of the slope for determining a match can be adjusted depending on the situation in which the audio metadata processing device 1 is used.

[0045] If the relative error R[%] is used as the criterion for determining whether the slopes match, then ω n-1 end =0 or ω n start At the point where ω = 0, the left side of formula (1) diverges or becomes 100%, and it is easy to judge that integration is impossible, so it is expected to have the effect of maintaining the time when the change in playback position starts or ends from before the conversion of the audio metadata. n-1 end =0 or ω n start = 0, it may be determined that integration is possible, so the time at which the change in the playback position starts or ends may shift from before the conversion of the acoustic metadata. The relative error R [%] is based on the rate of change of the playback position, and the absolute error A [degrees / second] is based on the amount of change. In this way, the integration determination unit 11 may determine whether or not to integrate by comparing the relative error R [%] of the rate of change or the absolute error A [degrees / second] of the amount of change in the acoustic metadata at the time and the acoustic metadata immediately before the time with the threshold value α.

[0046] In step S20, the audio metadata combining unit 12 combines the block with the previous block based on the gradient of the change in the playback position, using a method that differs depending on whether the block contains a description of jumpPosition. The process of step S20 will be described in detail later.

[0047] In step S21, when the audio metadata combining unit 12 determines that the block cannot be integrated, the block is added as a new block to the block following the previous block. Specifically, the audioBlockFormatID is set to a value obtained by incrementing the audioBlockFormatID of the previous block, and the other parameters of the block are adopted as they are.

[0048] In step S22, it is determined whether the block integration judgment process and the combination process have been performed for all received blocks, and the processes from step S12 to step S21 are repeated until the processing of all blocks is completed. The acoustic metadata combination unit 12 may output the converted ADM to the acoustic metadata storage unit 13 after each block is processed, or may output each frame of the S-ADM, or may output the ADM when the reception of the S-ADM is completed.

[0049] FIG. 3 is a flowchart showing the process of deriving the gradient of the change in the playback position at the end of the previous block by the integral judgment section 11 in step S15 of FIG.

[0050] In step S151, it is determined whether the previous block is the first block of the audioChannelFormat. If the previous block is the first block of the audioChannelFormat (step S151-Yes), the process proceeds to step S155. In this case, rendering is performed at the playback position described in the previous block throughout the previous block, so the slope of the change is 0 even at the end of the previous block. If the previous block is not the first block of the audioChannelFormat (step S151-No), the process proceeds to step S152. Note that the determination of the first block in step S151 is performed on the previous block, whereas the determination of the first block in S13 of FIG. 2 is performed on the block in question, and the objects being determined are different.

[0051] In step S152, the playback position of the block is compared with that of the block immediately before the previous block (hereinafter referred to as the "block before the previous block.") If there is no change in the playback position, that is, if the parameters of both blocks are the same (step S152-Yes), the process proceeds to step S155, and if there is a change in the playback position (step S152-No), the process proceeds to step S153.

[0052] In step S153, it is determined whether the value of jumpPosition is 1. If the previous block contains a description of jumpPosition and the value is 1 (step S153-Yes), the process proceeds to step S155. If the previous block does not contain a description of jumpPosition or the value of jumpPosition is 0 (step S153-No), the process proceeds to step S154.

[0053] Here, the standard specifies that interpolationLength should be set shorter than the frame length, so there is no need to check the value of interpolationLength to calculate the slope at the end of the previous block. However, if the interpolationLength described in the received S-ADM (ADM) is mistakenly set longer than the frame length, blocks that should not be integrated may be integrated. For safety reasons, a step may be inserted to check that interpolationLength is shorter than the frame length. If interpolationLength is set longer than the frame length, the value of jumpPosition is treated as 0.

[0054] In step S154, since the playback position changes over a period of time from the end of the block before the previous one to the end of the previous block, the gradient ω n-1 end [degrees / second] is calculated using the following formula. Here, P n-1 [degrees] is the playback position of the previous block, and P n-2 [degrees] is the playback position of the block before the previous one, and T n-1 dur is the duration [seconds] of the previous block.

[0055]

number

[0056] In step S155, the slope of the playback position change at the end of the previous block is set to zero.

[0057] FIG. 4 is a flowchart showing an example of the process of deriving the gradient of the playback position change at the start point of the block by the integral judgment section 11 in step S18 in FIG.

[0058] In step S181, the playback positions of the current block and the previous block are compared. If there is no change in the playback position, i.e., if the parameters of both blocks are the same (step S181-Yes), the process proceeds to step S186. If there is a change in the playback position (step S181-No), the process proceeds to step S182.

[0059] In step S182, it is determined whether the value of jumpPosition is 1. If the block contains a description of jumpPosition and the value is 1 (step S182-Yes), the process proceeds to step S183. If there is no description of jumpPosition or the value of jumpPosition is 0 (step S182-No), the process proceeds to step S185.

[0060] In step S183, the value of interpolationLength is confirmed. If there is no description of interpolationLength in the block (step S183-No), this means that the playback position of the block changes instantly at the same time as the start of the block, so the process proceeds to step S186. If interpolationLength has a value other than 0 (step S183-Yes), the process proceeds to step S184.

[0061] In step S184, the playback position described in the previous block is changed to the playback position described in the current block by multiplying the length of the interpolationLength. Therefore, the gradient ω n start [degrees / second] is calculated using the following formula. Here, P n [degrees] is the playback position of the block, T n int is the interpolationLength [seconds] of the block.

[0062]

number

[0063] In step S185, the playback position changes from the playback position described in the previous block to the playback position described in the current block over the time period of the block length, so that the gradient of the playback position change at the start point of the current block is ω n start [degrees / second] is calculated using the following formula. Here, T n dur is the duration [seconds] of the block.

[0064]

number

[0065] In step S186, the gradient of the playback position change at the start point of the block is set to zero.

[0066] FIG. 5 is a flowchart showing an example of a block integration method performed by the audio metadata combination unit 12 in step S20 of FIG.

[0067] In step S201, it is determined whether or not the value of jumpPosition is 1. If the block has a description of jumpPosition and the value is 1 (step S201-Yes), the process proceeds to step S203. If there is no description of jumpPosition or the value of jumpPosition is 0 (step S201-No), the process proceeds to step S202. Note that here, in the prior determination of whether or not the blocks can be integrated, blocks that have a description of jumpPosition and are determined to be able to be integrated are limited to blocks with a jumpPosition value of 1 and an interpolationLength setting.

[0068] FIG. 6 is a table showing an example of settings of main parameters (audioBlockFormatID, rtime, duration, position, jumpPosition, and interpolationLength) when integrating blocks.

[0069] In step S202, the block is integrated with the previous block using the settings shown in Fig. 6*1. That is, the values ​​of the previous block are set for audioBlockFormatID and rtime. The duration is set to a value obtained by adding the duration of the previous block and the duration of the block in question. The position is set to the position value of the block in question. The jumpPosition and interpolationLength are set to the values ​​of the previous block if a previous block exists.

[0070] In step S203, the block is first divided into two blocks, one with a time length of interpolationLength and the other with a time length of the remaining length.

[0071] In step S204, the divided first half block is integrated with the previous block using the settings shown in Fig. 6*2. That is, the values ​​of the previous block are set for audioBlockFormatID and rtime. The value obtained by adding the duration of the previous block and the interpolationLength of the block in question is set for duration. The value of the position of the block in question is set for position. JumpPosition and interpolationLength are not set.

[0072] In step S205, it is confirmed whether the absolute value of the difference in the change in the azimuth angle (change in the playback position) due to the integration exceeds the change threshold N [degrees]. If it exceeds the change threshold N (step S205-Yes), the process proceeds to step S206, and if it is equal to or less than the change threshold N (step S205-No), the process proceeds to step S208.

[0073] Fig. 7(a) is a diagram showing an example of the operation of audio objects before integration. Fig. 7(b) is a diagram showing an example of an unintended playback signal that a renderer may generate when audio metadata is integrated. Fig. 7(c) is a diagram showing an example of a playback signal that a renderer generates when audio metadata is divided to achieve an intended operation. If the change in the playback position after integration exceeds 180 degrees as shown in Fig. 7, the renderer that reads the block may be divided in its judgment as to whether to move the playback position clockwise or counterclockwise when viewed from above.

[0074] Therefore, in step S206, the newly integrated block is divided so that the change in the playback position does not exceed 180 degrees. If it is desired to minimize the final number of divisions, N may be set to the largest possible value not exceeding 180 degrees, such as 179 degrees, or if it is desired to improve the readability of the audio metadata, N may be set to a rounded value, such as 160 degrees or 120 degrees. The time width is set for the blocks before and after the division so that the slopes are equal before and after the division. The first half of the divided blocks inherits the audioBlockFormatID of the block before division as is, and the second half of the divided blocks is set to a value obtained by incrementing the audioBlockFormatID of the block before division.

[0075] In step S207, the latter half of the blocks divided in step S206 is added to the block as a new block.

[0076] In step S208, it is determined whether or not the latter block divided by the interpolationLength in step S203 exists. If the latter block exists (step S208-Yes), the process proceeds to step S209, and if the latter block does not exist (step S208-No), the process ends.

[0077] In step S209, the latter block divided in step S203 is added as a new block to the block next to the block combined in step S202 or step S204, or to the block next to the latter block divided in step S206, with the audioBlockFormatID incremented.

[0078] <Second embodiment> In addition to the above-mentioned integration based on the gradient of the change in the playback position, it is also possible to integrate the current block and the previous block using jumpPosition and interpolationLength, and further compress the amount of data. An example of compressing the amount of data using jumpPosition and interpolationLength will be described below as a second embodiment.

[0079] Fig. 8 is a block diagram showing an example of the configuration of an acoustic metadata processing device 2 according to the second embodiment. The acoustic metadata processing device 2 shown in Fig. 8 includes a starting point conversion unit 10, an integrated judgment unit 11, an acoustic metadata combination unit 12', and an acoustic metadata storage unit 13. The acoustic metadata processing device 2 according to the second embodiment differs from the acoustic metadata processing device 1 according to the first embodiment in the processing of the acoustic metadata combination unit. As the rest is the same as the first embodiment, a description thereof will be omitted.

[0080] The acoustic metadata combination unit 12' integrates acoustic metadata where there is a change in the playback position of an audio object just before a certain time, but there is no change in the playback position of the same audio object at that time, by using an interpolation length for the change in playback position, which can be described so that the change is continuous when describing the change in the playback position of an audio object.

[0081] 9 is a flowchart showing an example of a processing procedure in the audio metadata processing device 2. The area enclosed by the dashed line in the figure is different from FIG.

[0082] Integration can be performed using jumpPosition and interpolationLength when the slope at the end of the previous block is different from the slope at the start of the current block (step S19-No) and the current block has the same parameters as the previous block (step S23-Yes).

[0083] In step S24, the audio metadata combining unit 12 combines the previous block and the current block with the settings shown in FIG. 6*3, with the previous block being the portion corresponding to the interpolation that changes from the playback position of the block before previous to the playback position of the current block. That is, the values ​​of the previous block are set to audioBlockFormatID and rtime. The duration is set to a value obtained by adding the duration of the previous block and the duration of the current block. The position is set to the position value of the current block. JumpPosition is set to 1, and interpolationLength is set to the duration value of the previous block. Note that combination is not possible if the previous block already has values ​​for jumpPosition and interpolationLength, but in that case, since the slope at the end of the previous block is 0, there is no need to judge jumpPosition, etc.

[0084] Also, when the inclination at the end of the previous block and the inclination at the start of the current block match (step S19-Yes), a partial change occurs in the block combination process (step S20').

[0085] 10 is a flowchart showing an example of a block combining method in the audio metadata processing device 2. The area within the dashed line in the figure is different from FIG.

[0086] As shown in FIG. 10, when the block has jumpPosition and interpolationLength, the decision as to whether or not to integrate is unchanged from FIG. 2 as can be seen from FIG. 9, but there is no need to divide the block before integration, and integration is possible with jumpPosition and interpolationLength remaining as is. Therefore, when the jumpPosition value of the block is 1 (step S201-Yes), it is integrated with the previous block with the setting of *4 shown in FIG. 6 (step S210). That is, the audioBlockFormatID and rtime are set to the values ​​of the previous block. The duration is set to a value obtained by adding the duration of the previous block and the duration of the block. The position is set to the value of the position of the block. The jumpPosition is set to 1, and the interpolationLength is set to a value obtained by adding the duration of the previous block and the interpolationLength of the block.

[0087] In the above-mentioned first and second embodiments, two parameters, azimuth and elevation, are used to describe the playback position, but even if one of these two parameters is judged to be integrable, if the other parameter is judged to be incompatible, the block may be added as a new block to the block next to the previous block. In this case, the block is integrated with the previous block only when both of the two parameters are judged to be integrable.

[0088] <Example> Next, specific examples of ADMs that have been combined and processed by the audio metadata processing device 1 or the audio metadata processing device 2 will be described below as Examples 1 to 5.

[0089] Example 1 As a first embodiment, an example of the operation of the audio metadata processing device 1 when receiving the sixth to eighth blocks of the audioChannelFormat (three frames of S-ADM) for an S-ADM in which the parameters of each block do not change except for the playback position and the slope of the change in the playback position is always constant will be described. Note that since the audioBlockFormat ID (audioBlockFormatID) changes due to the conversion of the audio metadata, hereinafter the source audioBlockFormatID will be referred to as source AB_yyyyxxxx_zzzzzzzz, and the destination audioBlockFormatID will be referred to as destination AB_yyyyxxxx_zzzzzzzz.

[0090] Fig. 11 is a diagram showing an example of an S-ADM. Fig. 12 is a diagram showing an example of an ADM generated from the S-ADM shown in Fig. 11 by a conventional method. Fig. 13 is a diagram showing an example of an ADM generated from the S-ADM shown in Fig. 11 by an audio metadata processing device 1. Hereinafter, the integration process of the 6th block to the 8th block of audioChannelFormat will be described with reference to Figs. 11 and 13.

[0091] When the S-ADM is received in order starting from frame FF_00000006, the start point conversion unit 10 first changes the time notation (start point) of all blocks. lstart and lduration are time notation methods that start from the start time of each frame of the S-ADM, and represent the start time and block length of the block, respectively. lstart is added to the start time of the frame to calculate rtime, which is a notation method that starts from the start time of the program (audioProgramme), which is the notation method of the ADM. Since the block length is the same in both notations, lduration can be replaced with duration as it is.

[0092] After that, comparison and integration are performed starting from the source block with the smallest number (audioBlockFormatID:AB_00030001_00000006), excluding the initialization block.

[0093] The last eight digits of audioBlockFormatID are an index that distinguishes each block lined up in the time direction, with 00000001 used for the first block and 00000000 for the initialization block. By comparing the parameters of AB_00030001_00000006 with the parameters of the previous initialization block AB_00030001_00000000, it is determined that the change in playback position is 30 degrees, which is less than 180 degrees, and therefore the block in question does not need to be divided.

[0094] Next, the integrated judgment unit 11 checks whether the block is the first block of the channel. Although it is clear from the index that AB_00030001_00000006 is not the first block, in the first embodiment, an initialization block (a block in which initializeBlock = 1 is written) is also received when reception of S-ADM starts from the sixth frame, so the parameters of the initialization block are used to generate the first block (conversion destination AB_00030001_00000001) with duration = 0.

[0095] Next, the integrated judgment unit 11 checks whether there are any parameters that are different from the previous block other than the playback position. In the first embodiment, it can be confirmed that only the playback position is different.

[0096] Next, the integrated judgment unit 11 calculates the gradient ω of the playback position change at the end of the previous block according to the flow shown in FIG. n-1 end Now, since the first block of the channel has been generated from the initialization block, the block preceding the block (source AB_00030001_00000006) is the first block of the channel (destination AB_00030001_00000001), and ω n-1 end =0.

[0097] ω n-1 endis 0, the integration determination unit 11 compares the parameters of the block in question (source AB_00030001_00000006) with the previous block (destination AB_00030001_00000001). Because the playback positions are different, the integration determination unit 11 determines that the block in question cannot be integrated with the previous block. The audio metadata combination unit 12 adds the block in question as a new block next to the previous block (destination AB_00030001_00000002).

[0098] Since processing of all blocks received in frame FF_00000006 of the S-ADM has been completed, processing moves to the block received in frame FF_00000007. As with FF_00000006, the start point conversion unit 10 changes the time notation (start point) of all blocks.

[0099] Thereafter, the process moves to a block with a lower number (source block AB_00030001_00000007) excluding the initialization block. Integrated judgment unit 11 compares with the previous block (destination block AB_00030001_00000002) and confirms that the change in the playback position is less than 180 degrees.

[0100] The previous block (conversion destination AB_00030001_00000002) exists, so the block in question (conversion source AB_00030001_00000007) is not the first block.

[0101] The integrated judgment unit 11 compares the block (source block AB_00030001_00000007) with the previous block (destination block AB_00030001_00000002), and since only the playback position is a different parameter, it determines the gradient ω n-1 end Calculate.

[0102] It has already been confirmed that the previous block (transformed AB_00030001_00000002) is not the first block, and that it does not have the same parameters as the block before that (transformed AB_00030001_00000001). In addition, since there is no description of jumpPosition, the integrated judgment unit 11 determines ω n-1 end = 15 degrees / second.

[0103] The integrated decision unit 11 is n-1 end Since is not 0, the gradient ω of the playback position change at the start of the block is calculated according to FIG. n start It has already been confirmed that the parameters are not the same as those of the previous block (transformed block AB_00030001_00000002). In addition, since there is no description of jumpPosition, the integrated decision unit 11 calculates ω n start = 15 degrees / second.

[0104] Since the slope of the playback position change at the end of the previous block and the start of the current block is the same at 15 degrees / second, integration determination unit 11 determines that the previous block (conversion destination AB_00030001_00000002) and the current block (conversion source AB_00030001_00000007) can be integrated.

[0105] Next, the acoustic metadata combining unit 12 performs the merging process according to Fig. 5. Because there is no jumpPosition description, the acoustic metadata combining unit 12 merges the block in question with the previous block (conversion destination AB_00030001_00000002) with the settings shown in Fig. 6*1. Compared to the previous block before merging, the duration of the block after merging has been added to that of the block in question (conversion source AB_00030001_00000007), and the playback position has been rewritten to that of the block in question.

[0106] The same processing is performed for frame FF_00000008 as for frame FF_00000007. Ultimately, in an example where an ADM is generated without conversion by the acoustic metadata processing device 1 (see FIG. 12), the ADM is described in four blocks, whereas by using the acoustic metadata processing device 1, the ADM can be described in two blocks, thereby reducing the amount of data.

[0107] In this embodiment, the S-ADM was transmitted at a period of one frame every two seconds, however, in preparation for the introduction of object-based audio in broadcasting, it is also being considered to transmit the S-ADM at a period that coincides with the frame period of the video.

[0108] If an S-ADM is used in which the parameter changes are constant or do not change as in Example 1, then for a one-hour program with a video frame rate of 30 fps, approximately 108,000 blocks will be generated (30 blocks x 3,600 seconds). However, by using the audio metadata processing device 1, this can be described in two blocks, which is expected to result in a significant reduction in the amount of data.

[0109] Example 2 As a second embodiment, an example of operation of the audio metadata processing device 1 will be described in which the change threshold N is set to 160 degrees when receiving the first to fifth blocks of the audioChannelFormat for an S-ADM in which the parameters of each block do not change except for the playback position, and the slope of the change in the playback position does change.

[0110] Fig. 14 is a diagram showing an example of an S-ADM. Fig. 15 is a diagram showing an example of an ADM generated from the S-ADM shown in Fig. 14 by the audio metadata processing device 1. Hereinafter, the integration process of the first block to the fifth block of audioChannelFormat will be described with reference to Figs. 14 and 15.

[0111] When the S-ADM is received in order from frame FF_00000001, the origin conversion unit 10 changes the time notation (origin) of all blocks, as in the first embodiment. After that, comparison and integration are performed in order from the source block with the smallest number (audioBlockFormatID:AB_00030001_00000001) excluding the initialization block.

[0112] The last 8 digits of the ID of the block in question (source AB_00030001_00000001) indicate that it is the first block, but since an initialization block has also been received, the initialization block (source AB_00030001_00000000) is treated as the first block of the channel in question (destination AB_00030001_00000001). Therefore, the block in question (source AB_00030001_00000001) is no longer the first block, and comparison of the playback position, etc. proceeds.

[0113] Since the block (source AB_00030001_00000001) has no parameters different from the previous block (destination AB_00030001_00000001) other than the playback position, the integrated judgment unit 11 determines the gradient ω n-1 end Since the previous block (transformed AB_00030001_00000001) is the first block, ω n-1 end =0.

[0114] Since the playback positions of the block in question (source AB_00030001_00000001) and the previous block (destination AB_00030001_00000001) are different, the integration determination unit 11 determines that the block in question cannot be integrated with the previous block. The audio metadata combination unit 12 adds the block in question (source AB_00030001_00000001) as a new block (destination AB_00030001_00000002) next to the previous block (destination AB_00030001_00000001).

[0115] The next block (source block AB_00030001_00000002) is processed in the same manner, and the audio metadata combination unit 12 merges it with the previous block (destination block AB_00030001_00000002) in the same manner as in the first embodiment.

[0116] Furthermore, for the next block (source AB_00030001_00000003), the audio metadata combining unit 12 first merges it with the previous block (destination AB_00030001_00000002) based on the matching slope, and the previous block (destination AB_00030001_00000002) becomes a block in which the playback position is changed by -240 degrees from an azimuth angle of +160 degrees to -80 degrees over duration = 3 seconds (slope is -80 degrees / second). However, the absolute value of the change in azimuth angle, 240 degrees, exceeds N = 160 degrees. Therefore, the division unit 121 divides it again at the point where the change in azimuth angle exceeds 160 degrees, and the slope before and after the division is the same (ω n-1 end =ω n start ), the audio metadata combination unit 12 divides the time length of the block into 2 seconds and 1 second. The audio metadata combination unit 12 then adds the latter half of the divided block (conversion destination AB_00030001_00000002) to the end of the previous block (conversion destination AB_00030001_00000003).

[0117] Furthermore, for the next block (the original block AB_00030001_00000004), the integrated judgment unit 11 calculates the gradient ω n start =-80 degrees / second. The integrated judgment unit 11 calculates ω n start The slope of the playback position change at the end of the previous block (conversion destination AB_00030001_00000003) is ω n-1 end Therefore, it is judged that integration is possible.

[0118] Since jumpPosition is set for the block (source AB_00030001_00000004), the acoustic metadata combining unit 12 divides the block at the end of interpolationLength according to Fig. 5, and merges the first half of the divided block with the previous block (destination AB_00030001_00000003) with the setting *2 shown in Fig. 6. The acoustic metadata combining unit 12 adds the second half of the divided block as a new block (destination AB_00030001_00000003) after the previous block (destination AB_00030001_00000004).

[0119] When the integration judgment unit 11 compares the last block (source AB_00030001_00000005) with the previous block (destination AB_00030001_00000004), it determines that the previous block (destination AB_00030001_00000004) has a slope of 0 and the playback positions of the two blocks match, so that the block (source AB_00030001_00000005) can be integrated with the previous block (destination AB_00030001_00000004) because their slopes match. The audio metadata combination unit 12 integrates the block according to the setting shown in FIG. 6 *1.

[0120] Example 3 As a third embodiment, an example of the operation of the audio metadata processing device 1 when receiving the first to fifth blocks of the audioChannelFormat for S-ADM in which the parameters of each block do not change except for the playback position, and in addition to changes in the playback position, the gain also changes will be described.

[0121] Fig. 16 is a diagram showing an example of an S-ADM. Fig. 17 is an example of an ADM generated from the S-ADM shown in Fig. 16 by the acoustic metadata processing device 1 (α is set to 3%). Fig. 18 is an example of an ADM generated from the S-ADM shown in Fig. 16 by the acoustic metadata processing device 1 (α is set to 5%). The integration process of the first block to the fifth block of audioChannelFormat will be described below with reference to Figs. 16 to 18.

[0122] The initialization block (source AB_00030001_00000000) and the source block with the lowest number excluding the initialization block (source AB_00030001_00000001) are respectively made into the first destination block (destination AB_00030001_00000001) and the next block (destination AB_00030001_00000002), which is the same as in Example 2, so explanation will be omitted.

[0123] From the next block (source block AB_00030001_00000002), conversion will vary depending on the set value of α, the relative error that is the allowable range for determining whether the slopes match. First, with reference to Fig. 17, a case where α is set to 3% will be described.

[0124] The integrated judgment unit 11 detects the gradient ω of the playback position change at the end of the previous block (conversion destination AB_00030001_00000002). n-1 end = 22 deg / sec, and the inclination ω at the start of the block (source AB_00030001_00000002) n start = 23 deg / sec, and calculates the relative error to be about 4.5% from formula (1). Since this exceeds α = 3%, it is determined that the block in question (source AB_00030001_00000002) cannot be integrated. The acoustic metadata integration unit 12 adds the block in question as a new block (destination AB_00030001_00000003).

[0125] Furthermore, for the next block (the source block AB_00030001_00000003), the integrated judgment unit 11 calculates the gradient ω n start = 0 degrees / second, the slope of the playback position change at the end of the previous block (conversion destination AB_00030001_00000003) ω n-1 end = 23 deg / sec, and the relative error is 100% according to formula (1), so it is determined that the block (source AB_00030001_00000003) cannot be integrated. The acoustic metadata combination unit 12 adds the block as a new block (destination AB_00030001_00000004).

[0126] Furthermore, for the next block (source block AB_00030001_00000004), the integrated judgment unit 11 judges whether the gradient of the change in the playback position at the end of the previous block (destination block AB_00030001_00000004) is ω n-1 end = 0 deg / sec, and the playback positions of the block in question (source AB_00030001_00000004) and the previous block (destination AB_00030001_00000004) match, so it is determined that they can be merged. The audio metadata merger 12 merges the block in question with the previous block (destination AB_00030001_00000004) using the settings shown in Fig. 6 *1.

[0127] Furthermore, for the next block (source AB_00030001_00000005), the integration determination unit 11 determines that integration is not possible because the parameters of the playback position are the same as those of the previous block (destination AB_00030001_00000004), but the gain, which is a parameter other than the playback position, is different. The audio metadata combination unit 12 adds this block as a new block (destination AB_00030001_00000005).

[0128] Moreover, a case where the relative error α is set to 5% will be described with reference to Fig. 18. The integral judgment unit 11 judges the gradient ω of the playback position change at the end point of the previous block (conversion destination AB_00030001_00000002) n-1 end= 22 deg / sec and the inclination ω at the start of the block (source AB_00030001_00000002) n start Since the relative error of about 4.5% for α = 23 deg / sec is below α = 5%, it is determined that the block in question (source AB_00030001_00000002) can be integrated due to the matching of inclinations. The audio metadata integration unit 12 integrates the block in question with the previous block (destination AB_00030001_00000002) with the settings shown in Figure 6 *1. Subsequent processing is performed in the same way as when α = 3%, and audioBlockFormatID becomes a value decremented by 1 from the ID when α = 3% is set.

[0129] Example 4 As a fourth embodiment, an example of the operation of the audio metadata processing device 2 when the same S-ADM as in the second embodiment is received will be described.

[0130] FIG. 19 is a diagram showing an example of an ADM generated by the acoustic metadata processing device 2 from the S-ADM shown in FIG. 14 of the second embodiment.

[0131] Processing up to the third block (source AB_00030001_00000003) is carried out in the same manner as in the second embodiment (destination AB_00030001_00000003).

[0132] When processing the next block (source block AB_00030001_00000004), as shown in Figure 9, the acoustic metadata combination unit 12' does not divide the block at the end of the interpolationLength, but instead merges it with the previous block (destination block AB_00030001_00000003) using the setting *4 shown in Figure 6.

[0133] The integration decision unit 11 compares the last block (source AB_00030001_00000005) with the previous block (destination AB_00030001_00000003) and finds that the slope of the previous block (destination AB_00030001_00000003) is ω n-1 end= 0 deg / sec, and the playback positions of the two blocks match, so it is determined that the block in question (source AB_00030001_00000005) can be integrated with the previous block (destination AB_00030001_00000003) due to the matching inclination. The audio metadata integration unit 12' integrates the block in question with the setting shown in Fig. 6 *1.

[0134] Comparing the ADM converted in Example 4 (see Figure 19) with the ADM converted in Example 2 (see Figure 15), it can be seen that the dynamic metadata that was written in four blocks can now be written in three blocks, thereby further reducing the amount of data.

[0135] Example 5 As a fifth embodiment, an example of the operation of the audio metadata processing device 2 when the same S-ADM as in the third embodiment is received will be described.

[0136] Fig. 20 is a diagram showing an example of an ADM generated from the S-ADM shown in Fig. 16 of the third embodiment by the acoustic metadata processing device 2. Here, α is set to 5%.

[0137] The initialization block (source AB_00030001_00000000) and the source block (source AB_00030001_00000001) which is the smallest number excluding the initialization block are respectively set as the first destination block (destination AB_00030001_00000001) and the next block (destination AB_00030001_00000002), which is the same as in the third embodiment. In addition, the next block (source AB_00030001_00000002) is also integrated with the previous block (destination AB_00030001_00000002) as in the third embodiment, since α=5% is set.

[0138] Furthermore, for the next block (source block AB_00030001_00000003), the integrated judgment unit 11 judges whether the gradient of the change in the playback position at the end of the previous block (destination block AB_00030001_00000003) is ω n-1 end= 23 deg / sec, which is greater than 0, so according to the flow in Figure 8, the gradient ω n start = 0 deg / sec. The magnitude of the relative error indicates that the inclinations do not match, but the playback positions of the block in question (source AB_00030001_00000003) and the previous block (destination AB_00030001_00000003) are the same. Therefore, the audio metadata combining unit 12' combines the block in question (source AB_00030001_00000003) with the previous block (destination AB_00030001_00000002) using the settings shown in Figure 6 *3.

[0139] The subsequent processing is performed in the same manner as in the third embodiment, and audioBlockFormatID becomes a value that is further decremented by 1 from the ID in the third embodiment where α=5% is set.

[0140] Comparing the ADM converted in Example 5 (see Figure 20) with the ADM converted in Example 3 (α = 5%) (see Figure 18), it can be seen that dynamic metadata that was written in four blocks can now be written in three blocks, thereby further reducing the amount of data.

[0141] <Program> It is also possible to use a computer capable of executing program instructions to function as the above-mentioned audio metadata processing devices 1, 2. Here, the computer may be a general-purpose computer, a special-purpose computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing the necessary tasks.

[0142] The computer includes a processor, a storage unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), a SoC (System on a Chip), or the like, and may be composed of multiple processors of the same type or different types. The processor reads a program from the storage unit and executes it to control each of the above components and perform various arithmetic processing. Note that at least a part of the processing contents may be realized by hardware. The input unit is an input interface that receives a user's input operation and acquires information based on the user's operation, and is a pointing device, a keyboard, a mouse, or the like. The output unit is an output interface that outputs information, and is a display, a speaker, or the like. The communication interface is an interface for communicating with an external device.

[0143] The program may be recorded on a computer-readable recording medium. By using such a recording medium, the program can be installed on the computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. In addition, the program may be in a form that is downloaded from an external device via a network.

[0144] In this way, according to the present invention, when recording and saving audio metadata in which a large amount of dynamic metadata for each time is described in chronological order in object-based audio, particularly an S-ADM in which dynamic metadata involving changes in the playback position is described, the received S-ADM is converted sequentially and integrated with the ADM received up to the previous frame and reconstructed using the slope of the value change, making it possible to record and save the ADM while suppressing an increase in the amount of data.

[0145] Moreover, by integrating the data using the complementary time of the change in the playback position, the amount of data can be further reduced.

[0146] In addition, by appropriately dividing the data so that the change in azimuth angle does not exceed a certain value, such as 180 degrees, when integrating the data, it is possible to prevent an object-based audio player (renderer) that loads the reconstructed ADM from generating a playback signal that the producer did not intend.

[0147] It will be apparent to those skilled in the art that the present invention also enables offline conversion, such as reading an audio file in which S-ADM has been recorded as an audio signal and re-recording it as ADM.

[0148] Although the above-mentioned embodiment has been described as a representative example, it is obvious to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be interpreted as being limited by the above-mentioned embodiment, and various modifications and changes are possible without departing from the scope of the claims. For example, it is possible to combine multiple building blocks shown in the configuration diagram of the embodiment into one, or to divide one building block. [Explanation of symbols]

[0149] 1,2 Acoustic metadata processing device 10 Origin conversion section 11 Integrated Judgment Department 12,12' Audio metadata binding section 13. Audio Metadata Storage 121 Division

Claims

1. An audio metadata processing device that combines audio metadata in which playback conditions of audio objects for each time are described in chronological order in object-based audio, comprising: an integration determination unit that compares the acoustic metadata at the time with the acoustic metadata up to the time before the time and performs an integration determination as to whether or not to integrate the acoustic metadata at the time with the acoustic metadata up to the time before the time; an acoustic metadata combining unit that combines the acoustic metadata at the time with the acoustic metadata up to the time before the time based on the integration determination, or adds the acoustic metadata at the time to the acoustic metadata up to the time before the time without integrating the acoustic metadata; An audio metadata processing device comprising:

2. The audio metadata processing device according to claim 1 , wherein the integration determination unit determines whether or not to integrate by comparing a rate or amount of change in the playback position of the same audio object in the audio metadata at the time and the audio metadata immediately before the time with a threshold value.

3. 3. The acoustic metadata processing device according to claim 2, wherein the integration determination unit determines whether or not to integrate by comparing a relative error in the rate of change or an absolute error in the amount of change in the acoustic metadata at the time and the acoustic metadata immediately before the time with a threshold value.

4. 4. The acoustic metadata processing device according to claim 1, wherein when the acoustic metadata at a given time is integrated with the acoustic metadata up to the time before the given time, the acoustic metadata combination unit divides the audio object described in the acoustic metadata within a range not exceeding a change threshold when a change in the playback position of the audio object exceeds a change threshold.

5. The acoustic metadata processing device described in any one of claims 1 to 4, wherein the acoustic metadata combination unit integrates acoustic metadata in which there is a change in the playback position of an audio object just before that time but there is no change in the playback position of the same audio object at that time by using a complementary time of a change in playback position, which can be described so that the change is continuous when describing a change in the playback position of an audio object.

6. A program for causing a computer to function as the acoustic metadata processing device according to any one of claims 1 to 5.

Images