Signal processing device, method, and program

Abstract

To improve coding efficiency.SOLUTION: A signal processing apparatus includes an acquisition unit that acquires reverb information including at least one of spatial reverb information specific to a space around an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object, and a reverb processing unit that generates a signal of a reverb component of the audio object on the basis of the reverb information and the audio object signal. This technology can be applied to a signal processing apparatus.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present technology relates to a signal processing device, method, and program, and in particular to a signal processing device, method, and program that are capable of improving encoding efficiency. [Background technology]

[0002] Conventionally, object audio technology has been used in movies, games, etc., and encoding methods capable of handling object audio have also been developed. Specifically, for example, the international standard MPEG (Moving Picture Experts Group)-H Part 3:3D audio standard is known (for example, see Non-Patent Document 1).

[0003] In this encoding method, in addition to the conventional two-channel stereo method and multi-channel stereo methods such as 5.1 channel, it is possible to treat moving sound sources, etc. as independent audio objects and encode the object's position information as metadata along with the signal data of the audio object.

[0004] This allows playback in various viewing environments with different numbers of speakers. It also makes it easy to process the sound of a specific sound source during playback, such as adjusting the volume of the sound of a specific sound source or adding effects to the sound of a specific sound source, which was difficult with conventional encoding methods.

[0005] For example, the standard in Non-Patent Document 1 uses a method called three-dimensional Vector Based Amplitude Panning (VBAP) (hereinafter simply referred to as VBAP) for rendering processing.

[0006] This is a rendering technique commonly known as panning, which involves distributing gain to the three speakers that exist on the surface of a sphere with the listening position as the origin and are closest to the audio object, which also exists on the surface of the sphere.

[0007] This kind of panning of audio objects is based on the premise that all audio objects are on the surface of a sphere with the listening position as the origin, so the sense of distance when an audio object is close to or far from the listening position is controlled only by the gain for the audio object.

[0008] However, in reality, unless one takes into account the fact that the attenuation rate differs depending on the frequency component and reflections in the space in which the audio object exists, the representation of the sense of distance ends up being far from the actual experience.

[0009] In order to reflect these effects in the listening experience, the first thing that comes to mind is to physically calculate the spatial reflection and attenuation and use them as the final output audio signal. However, while this method is effective for video content such as movies, which can be produced over a very long period of time, it is difficult to do so when rendering audio objects in real time.

[0010] Furthermore, the final output, which is obtained by physically calculating spatial reflection and attenuation, does not easily reflect the intent of the content creator. Therefore, particularly in musical works such as music clips, a format that makes it easier for the content creator to reflect their intent, such as applying preferred reverb processing to vocal tracks, is required. [Prior art documents] [Non-patent literature]

[0011] [Non-Patent Document 1] INTERNATIONAL STANDARD ISO / IEC 23008-3 First edition 2015-10-15 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio Summary of the Invention [Problem to be solved by the invention]

[0012] Therefore, for real-time playback, it is desirable to store data such as coefficients required for reverb processing that takes into account spatial reflection and attenuation for each audio object in a file or transmission stream along with the position information of the audio object, and use this to obtain the final output audio signal.

[0013] However, storing reverb processing data required for each audio object in a file or transmission stream for each frame leads to an increase in transmission rate, and therefore data transmission with high coding efficiency is required.

[0014] The present technology has been made in consideration of such circumstances, and is intended to make it possible to improve coding efficiency. [Means for solving the problem]

[0015] According to one aspect of the present technology, there is provided a signal processing device, comprising: an acquisition unit that acquires reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object; a reverb processing unit that generates a signal of a reverb component of the audio object based on the reverb information and the audio object signal; and a rendering unit that performs rendering processing using VBAP. When identification information indicating the past reverb information is acquired by the acquisition unit, the reverb processing unit generates a signal of the reverb component based on the reverb information indicated by the identification information and the audio object signal.

[0016] A signal processing method or program according to one aspect of the present technology acquires reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object, generates a reverb component signal of the audio object based on the reverb information and the audio object signal, and performs rendering processing using VBAP. When identification information indicating the past reverb information is acquired, a signal of the reverb component is generated based on the reverb information indicated by the identification information and the audio object signal.

[0017] In one aspect of the present technology, reverb information including at least one of spatial reverb information specific to the space surrounding an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object are obtained, and a signal of the reverb component of the audio object is generated based on the reverb information and the audio object signal, and a rendering process is performed using VBAP. Furthermore, when identification information indicating the past reverb information is acquired, the reverb component signal is generated based on the reverb information indicated by the identification information and the audio object signal. Effect of the Invention

[0018] According to one aspect of the present technology, it is possible to improve coding efficiency.

[0019] Note that the effects described herein are not necessarily limited to those described herein, and may be any of the effects described in this disclosure. [Brief description of the drawings]

[0020] [Figure 1] FIG. 1 illustrates an example of the configuration of a signal processing device. [Diagram 2] FIG. 2 illustrates an example of the configuration of a rendering processing unit. [Diagram 3] FIG. 13 is a diagram illustrating an example of the syntax of audio object information. [Figure 4] 11 is a diagram illustrating an example of the syntax of object reverb information and spatial reverb information. [Diagram 5] FIG. 1 is a diagram illustrating a localization position of a reverb component. [Figure 6] FIG. 1 is a diagram illustrating an impulse response. [Figure 7] FIG. 2 is a diagram illustrating the relationship between audio objects and a listening position. [Figure 8] 1 is a diagram illustrating a direct sound component, an early reflection sound component, and a rear reverberation component. [Figure 9] 11 is a flowchart illustrating an audio output process. [Figure 10] FIG. 1 illustrates an example of the configuration of an encoding device. [Figure 11] 11 is a flowchart illustrating an encoding process. [Figure 12] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.

[0022] First Embodiment <Configuration example of signal processing device> This technology enables the transmission of reverb parameters with high coding efficiency by adaptively selecting the reverb parameter coding method based on the relationship between the audio object and the listening position.

[0023] FIG. 1 is a diagram showing an example of the configuration of an embodiment of a signal processing device to which the present technology is applied.

[0024] The signal processing device 11 shown in FIG.

[0025] The core decoding processing unit 21 receives and decodes the transmitted input bitstream, and supplies the audio object information and audio object signals obtained as a result to the rendering processing unit 22. In other words, the core decoding processing unit 21 functions as an acquisition unit that acquires the audio object information and audio object signals.

[0026] Here, the audio object signal is an audio signal for reproducing the sound of an audio object.

[0027] The audio object information is metadata of an audio object, that is, an audio object signal, and includes information about the audio object that is required for the processing performed in the rendering processor 22.

[0028] Specifically, the audio object information includes object position information, direct sound gain, object reverb information, object reverb sound gain, spatial reverb information, and spatial reverb gain.

[0029] Here, the object position information is information that indicates the position of an audio object in a three-dimensional space. For example, the object position information includes a horizontal angle that indicates the horizontal position of the audio object as viewed from a reference listening position, a vertical angle that indicates the vertical position of the audio object as viewed from the listening position, and a radius that indicates the distance from the listening position to the audio object.

[0030] The direct sound gain is a gain value used for gain adjustment when generating a direct sound component of the sound of an audio object.

[0031] For example, in the rendering processing unit 22, when rendering an audio object, that is, an audio object signal, a signal of a direct sound component from the audio object, an object-specific reverb sound signal, and a space-specific reverb sound signal are generated.

[0032] In particular, object-specific reverb sound signals and space-specific reverb sound signals are signals of components such as reflected sound and reverberation sound from an audio object, that is, reverb component signals obtained by performing reverb processing on an audio object signal.

[0033] Object-specific reverb sound is the early reflection sound component of the sound of an audio object, and is a sound that is greatly contributed by the state of the audio object, such as the position of the audio object in three-dimensional space. In other words, object-specific reverb sound is a reverb sound that depends on the position of the audio object, and changes significantly depending on the relative positional relationship between the listening position and the audio object.

[0034] In contrast, spatially specific reverberant sound is the rear reverberation component of the sound of an audio object, in which the contribution of the state of the audio object is small and the contribution of the environment surrounding the audio object, i.e., the state of the space surrounding the audio object, is large.

[0035] That is, the space-specific reverb sound changes significantly depending on the relative positional relationship between the listening position and the walls in the space around the audio object, the materials of the walls and floor, etc., but changes very little depending on the relative positional relationship between the listening position and the audio object. Therefore, it can be said that the space-specific reverb sound is a sound that depends on the space around the audio object.

[0036] During rendering processing in the rendering processing unit 22, such direct sound components from the audio objects, object-specific reverb sound components, and space-specific reverb sound components are generated by reverb processing of the audio object signals. The direct sound gain is used to generate signals of such direct sound components.

[0037] The object reverb information is information related to object-specific reverb sound. For example, the object reverb information includes object reverb position information indicating the position of the sound image of the object-specific reverb sound and coefficient information used to generate object-specific reverb sound components during reverb processing.

[0038] Since the object-specific reverb sound is a component specific to an audio object, the object reverb information can be said to be reverb information specific to an audio object that is used to generate the object-specific reverb sound component during reverb processing.

[0039] In the following, the position of the sound image of the object-specific reverb sound in the three-dimensional space indicated by the object reverb position information is also called the object reverb component position. This object reverb component position can also be said to be the arrangement position of the real speaker or virtual speaker that outputs the object-specific reverb sound in the three-dimensional space.

[0040] Moreover, the object reverb sound gain included in the audio object information is a gain value used for adjusting the gain of the object-specific reverb sound.

[0041] The spatial reverb information is information related to a spatially specific reverb sound. For example, the spatial reverb information includes spatial reverb position information indicating the localization position of the sound image of the spatially specific reverb sound, and coefficient information used to generate the spatially specific reverb sound component during reverb processing.

[0042] Since space-specific reverb sound is a space-specific component with a low contribution rate from the audio object, it can be said that the space-specific reverb information is reverb information specific to the space surrounding the audio object, and is used to generate space-specific reverb sound components during reverb processing.

[0043] In the following, the localization position of the sound image of the spatial specific reverb sound in the three-dimensional space indicated by the spatial reverb position information is also referred to as the spatial reverb component position. This spatial reverb component position can also be said to be the arrangement position of the real or virtual speaker that outputs the spatial specific reverb sound in the three-dimensional space.

[0044] The spatial reverb gain is a gain value used to adjust the gain of an object-specific reverb sound.

[0045] The audio object information output from the core decode processor 21 includes at least the object position information out of object position information, direct sound gain, object reverb information, object reverb sound gain, spatial reverb information, and spatial reverb gain.

[0046] The rendering processor 22 generates an output audio signal based on the audio object information and audio object signal supplied from the core decode processor 21, and supplies the output audio signal to a speaker, a recording unit, or the like in the subsequent stages.

[0047] That is, the rendering processing unit 22 performs reverb processing based on the audio object information, and generates a direct sound signal of one or more audio objects, an object-specific reverb sound signal, and a space-specific reverb sound signal.

[0048] The rendering processor 22 then performs rendering processing by VBAP for each of the obtained direct sound, object-specific reverb sound, and space-specific reverb sound signals, and generates output audio signals with a channel configuration that corresponds to the output destination playback device, such as a speaker system, headphones, etc. Furthermore, the rendering processor 22 adds the signals of the same channel of the output audio signals generated for each signal to generate one final output audio signal.

[0049] When sound is played back based on the output audio signal obtained in this way, the sound image of the direct sound of the audio object is localized at the position indicated by the object position information, the sound image of the object-specific reverb sound is localized at the object reverb component position, and the sound image of the space-specific reverb sound is localized at the space reverb component position. This realizes more realistic audio playback with the sense of distance of the audio object appropriately controlled.

[0050] Example of the configuration of the rendering processing unit Next, a more detailed configuration example of the rendering processing unit 22 of the signal processing device 11 shown in FIG. 1 will be described.

[0051] Here, as a specific example, a case where there are two audio objects will be described. Note that the number of audio objects may be any number, and it is possible to handle as many audio objects as the computational resources allow.

[0052] In the following, when distinguishing between the two audio objects, one audio object will be referred to as audio object OBJ1, and the audio object signal of that audio object OBJ1 will be referred to as audio object signal OA1, while the other audio object will be referred to as audio object OBJ2, and the audio object signal of that audio object OBJ2 will be referred to as audio object signal OA2.

[0053] Furthermore, hereinafter, the object position information, direct sound gain, object reverb information, object reverb sound gain, and spatial reverb gain for audio object OBJ1 will also be referred to as object position information OP1, direct sound gain OG1, object reverb information OR1, object reverb sound gain RG1, and spatial reverb gain SG1, among others.

[0054] Similarly, hereinafter, the object position information, direct sound gain, object reverb information, object reverb sound gain, and spatial reverb gain for audio object OBJ2 will also be referred to as object position information OP2, direct sound gain OG2, object reverb information OR2, object reverb sound gain RG2, and spatial reverb gain SG2, among others.

[0055] When there are two audio objects in this way, the rendering processing unit 22 is configured as shown in FIG.

[0056] In the example shown in Figure 2, the rendering processing unit 22 has an amplification unit 51-1, an amplification unit 51-2, an amplification unit 52-1, an amplification unit 52-2, an object-specific reverb processing unit 53-1, an object-specific reverb processing unit 53-2, an amplification unit 54-1, an amplification unit 54-2, a space-specific reverb processing unit 55, and a rendering unit 56.

[0057] The amplification units 51-1 and 51-2 perform gain adjustment on the audio object signals OA1 and OA2 supplied from the core decoding processing unit 21 by the direct sound gains OG1 and OG2 supplied from the core decoding processing unit 21, and supply the resulting direct sound signals of the audio objects to the rendering unit 56.

[0058] In the following description, the amplifier units 51-1 and 51-2 will be simply referred to as amplifier unit 51 unless there is a need to distinguish between them.

[0059] The amplifiers 52-1 and 52-2 perform gain adjustment on the audio object signals OA1 and OA2 supplied from the core decode processor 21 by the object reverb sound gains RG1 and RG2 supplied from the core decode processor 21. This gain adjustment adjusts the volume of the reverb sound specific to each object.

[0060] The amplifiers 52-1 and 52-2 supply the gain-adjusted audio object signals OA1 and OA2 to the object-specific reverberation processors 53-1 and 53-2.

[0061] In the following description, the amplifier unit 52-1 and the amplifier unit 52-2 will be simply referred to as the amplifier unit 52 unless there is a need to particularly distinguish between them.

[0062] The object specific reverberation processor 53-1 performs reverberation processing on the gain-adjusted audio object signal OA1 supplied from the amplifier 52-1, based on the object reverberation information OR1 supplied from the core decode processor 21.

[0063] This reverb process generates one or more object-specific reverb sound signals for the audio object OBJ1.

[0064] In addition, the object-specific reverb processing unit 53-1 generates position information indicating the absolute position of the sound image of each object-specific reverb sound in three-dimensional space based on the object position information OP1 supplied from the core decoding processing unit 21 and the object reverb position information contained in the object reverb information OR1.

[0065] As described above, the object position information OP1 is information consisting of a horizontal angle, a vertical angle, and a radius that indicates the absolute position of the audio object OBJ1 with respect to the listening position in three-dimensional space.

[0066] In contrast, the object reverb position information can be information indicating the absolute position (localization position) of the sound image of the object-specific reverb sound as viewed from the listening position in three-dimensional space, or it can be information indicating the position (localization position) of the sound image of the object-specific reverb sound relative to the audio object OBJ1 in three-dimensional space.

[0067] For example, when the object reverb position information is information indicating the absolute position of the sound image of the object-specific reverb sound as viewed from the listening position in three-dimensional space, the object reverb position information is information consisting of a horizontal angle, a vertical angle, and a radius indicating the absolute position of the sound image of the object-specific reverb sound based on the listening position in three-dimensional space.

[0068] In this case, the object specific reverb processor 53-1 uses the object reverb position information as it is as position information indicating the absolute position of the sound image of the object specific reverb sound.

[0069] On the other hand, when the object reverb position information is information indicating the position of the sound image of the object-specific reverb sound relative to the audio object OBJ1, the object reverb position information is information consisting of a horizontal angle, a vertical angle, and a radius indicating the position of the sound image of the object-specific reverb sound relative to the audio object OBJ1 as viewed from the listening position in three-dimensional space.

[0070] In this case, based on the object position information OP1 and the object reverb position information, the object-specific reverb processing unit 53-1 generates information consisting of a horizontal angle, a vertical angle, and a radius indicating the absolute position of the sound image of the object-specific reverb sound based on the listening position in three-dimensional space, as position information indicating the absolute position of the sound image of the object-specific reverb sound.

[0071] The object-specific reverb processing unit 53-1 supplies to the rendering unit 56 a pair of the object-specific reverb sound signal thus obtained for each of one or more object-specific reverb sounds and the position information of the object-specific reverb sound.

[0072] In this way, by generating an object-specific reverb sound signal and position information through reverb processing, it becomes possible to handle each object-specific reverb sound signal as an independent audio object signal.

[0073] Similarly, the object specific reverberation processor 53-2 performs reverberation processing on the gain-adjusted audio object signal OA2 supplied from the amplifier 52-2, based on the object reverberation information OR2 supplied from the core decode processor 21.

[0074] This reverb process generates one or more object-specific reverb sound signals for the audio object OBJ2.

[0075] In addition, the object-specific reverb processing unit 53-2 generates position information indicating the absolute position of the sound image of each object-specific reverb sound in three-dimensional space based on the object position information OP2 supplied from the core decoding processing unit 21 and the object reverb position information contained in the object reverb information OR2.

[0076] Then, the object-specific reverb processing unit 53-2 supplies the rendering unit 56 with a pair of the object-specific reverb sound signal thus obtained and the position information of the object-specific reverb sound.

[0077] In the following description, when there is no need to particularly distinguish between the object-specific reverb processing unit 53-1 and the object-specific reverb processing unit 53-2, they will also be simply referred to as the object-specific reverb processing unit 53.

[0078] The amplifiers 54-1 and 54-2 perform gain adjustment on the audio object signals OA1 and OA2 supplied from the core decode processor 21 by the spatial reverb gains SG1 and SG2 supplied from the core decode processor 21. This gain adjustment adjusts the volume of each space-specific reverb sound.

[0079] Further, the amplifiers 54-1 and 54-2 supply the gain-adjusted audio object signals OA1 and OA2 to the space-specific reverb processor 55.

[0080] In the following description, the amplifier units 54-1 and 54-2 will be simply referred to as amplifier unit 54 unless there is a need to distinguish between them.

[0081] The space-specific reverb processing unit 55 performs reverb processing on the gain-adjusted audio object signals OA1 and OA2 supplied from the amplifiers 54-1 and 54-2, based on the space reverb information supplied from the core decode processing unit 21. The space-specific reverb processing unit 55 also generates a space-specific reverb sound signal by adding signals obtained by reverb processing on the audio objects OBJ1 and OBJ2. The space-specific reverb processing unit 55 generates one or more space-specific reverb sound signals.

[0082] Furthermore, in a similar manner to the object-specific reverb processing unit 53, the space-specific reverb processing unit 55 generates position information indicating the absolute position of the sound image of the space-specific reverb sound based on the space-specific reverb position information contained in the space-specific reverb information supplied from the core decoding processing unit 21, the object position information OP1, and the object position information OP2.

[0083] This position information is, for example, information consisting of a horizontal angle, a vertical angle, and a radius that indicates the absolute position of the sound image of the space-specific reverberation sound based on the listening position in a three-dimensional space.

[0084] The space-specific reverb processing unit 55 supplies pairs of space-specific reverb sound signals and position information for one or more space-specific reverb sounds obtained in this manner to the rendering unit 56. Note that, like object-specific reverb sounds, these space-specific reverb sounds also have position information and can therefore be treated as independent audio object signals.

[0085] The components from the amplifier 51 to the space-specific reverb processor 55 function as a processing block constituting a reverb processor that is provided before the rendering unit 56 and performs reverb processing based on audio object information and audio object signals.

[0086] The rendering unit 56 performs rendering processing using VBAP based on the supplied sound signals and the positional information of those sound signals, and generates and outputs an output audio signal consisting of signals of each channel of a predetermined channel configuration.

[0087] That is, the rendering unit 56 performs rendering processing using VBAP based on the object position information supplied from the core decoding processing unit 21 and the direct sound signal supplied from the amplification unit 51, and generates output audio signals for each channel for each of the audio objects OBJ1 and audio objects OBJ2.

[0088] In addition, the rendering unit 56 performs rendering processing using VBAP for each pair of object-specific reverb sound signals and position information supplied from the object-specific reverb processing unit 53, and generates output audio signals for each channel for each object-specific reverb sound.

[0089] Furthermore, the rendering unit 56 performs rendering processing using VBAP for each pair of space-specific reverb sound signals and position information supplied from the space-specific reverb processing unit 55, and generates output audio signals for each channel for each space-specific reverb sound.

[0090] Then, the rendering unit 56 adds together the signals of the same channel of the output audio signals obtained for the audio object OBJ1, the audio object OBJ2, the object-specific reverb sound, and the space-specific reverb sound, respectively, to generate a final output audio signal.

[0091] <Example of input bitstream format> Here, an example of the format of the input bit stream supplied to the signal processing device 11 will be described.

[0092] For example, the format (syntax) of the input bitstream is as shown in Fig. 3. In the example shown in Fig. 3, the part with the characters "object_metadata()" is metadata of the audio object, that is, the part with the audio object information.

[0093] This audio object information portion includes object position information for the audio objects, the number of which is the number of audio objects indicated by the characters “num_objects.” In this example, the horizontal angle position_azimuth[i], vertical angle position_elevation[i], and radius position_radius[i] are stored as the object position information for the i-th audio object.

[0094] The audio object information also includes a reverb information flag, indicated by the characters "flag_obj_reverb", which indicates whether or not reverb information such as object reverb information or spatial reverb information is included.

[0095] Here, if the value of the reverb information flag flag_obj_reverb is "1", this indicates that reverb information is included in the audio object information.

[0096] In other words, when the value of the reverb information flag flag_obj_reverb is "1", reverb information including at least either spatial reverb information or object reverb information is stored in the audio object information.

[0097] More specifically, depending on the value of the reuse flag use_prev (described later), the audio object information may contain identification information for identifying past reverb information as reverb information, i.e., a reverb ID (described later), but may not contain object reverb information or spatial reverb information.

[0098] On the other hand, if the value of the reverb information flag flag_obj_reverb is "0", this indicates that the audio object information does not include reverb information.

[0099] When the value of the reverb information flag flag_obj_reverb is "1", the audio object information stores, as reverb information, a direct sound gain indicated by the characters "dry_gain[i]", an object reverb sound gain indicated by the characters "wet_gain[i]", and a spatial reverb gain indicated by the characters "room_gain[i]", each for the number of audio objects.

[0100] The direct sound gain dry_gain[i], the object reverb sound gain wet_gain[i], and the space reverb gain room_gain[i] determine the mixing ratio of the direct sound, the object-specific reverb sound, and the space-specific reverb sound in the output audio signal.

[0101] Furthermore, the audio object information stores a reuse flag represented by the characters "use_prev" as reverb information.

[0102] This reuse flag use_prev is flag information indicating whether or not past object reverb information specified by a reverb ID is to be reused as object reverb information of the i-th audio object.

[0103] Here, a reverb ID is assigned to each piece of object reverb information transmitted in the input bit stream as identification information for identifying (specifying) that object reverb information.

[0104] For example, when the value of the reuse flag use_prev is “1”, it indicates that past object reverb information is to be reused, and in this case the audio object information stores a reverb ID indicating the object reverb information to be reused, indicated by the characters “reverb_data_id[i]”.

[0105] On the other hand, when the value of the reuse flag use_prev is “0”, it indicates that the object reverb information is not reused, and in this case the audio object information stores the object reverb information indicated by the characters “obj_reverb_data(i)”.

[0106] In addition, the audio object information stores a space reverb information flag, indicated by the characters "flag_room_reverb", as reverb information.

[0107] The spatial reverb information flag flag_room_reverb is a flag indicating the presence or absence of spatial reverb information. For example, when the value of the spatial reverb information flag flag_room_reverb is "1", it indicates that spatial reverb information is present, and the spatial reverb information indicated by the characters "room_reverb_data(i)" is stored in the audio object information.

[0108] On the other hand, if the value of the spatial reverb information flag flag_room_reverb is "0", it indicates that there is no spatial reverb information, and in this case, the spatial reverb information is not stored in the audio object information. Note that a reuse flag may be stored for the spatial reverb information as in the case of the object reverb information, so that the spatial reverb information can be reused as appropriate.

[0109] Moreover, the format (syntax) of the object reverb information obj_reverb_data(i) and the spatial reverb information room_reverb_data(i) in the audio object information of the input bit stream is, for example, as shown in FIG.

[0110] In the example shown in FIG. 4, the object reverb information includes a reverb ID indicated by the characters "reverb_data_id", the number of object-specific reverb sound components to be generated indicated by the characters "num_out", and a tap length indicated by the characters "len_ir".

[0111] In this example, it is assumed that the coefficients of the impulse response are stored as coefficient information used to generate object-specific reverberation sound components, and the tap length len_ir indicates the tap length of the impulse response, i.e., the number of coefficients of the impulse response.

[0112] Also, as the object reverb information, object reverb position information of the object-specific reverb sounds is included for the number of object-specific reverb sound components to be generated (num_out).

[0113] That is, the horizontal angle position_azimuth[i], the vertical angle position_elevation[i], and the radius position_radius[i] are stored as the object reverb position information of the i-th object-specific reverb sound component.

[0114] Furthermore, as coefficient information of the i-th object-specific reverberation sound component, impulse response coefficients impulse_response[i][j] are stored in the number corresponding to the tap length len_ir.

[0115] On the other hand, the spatial reverb information includes the number of spatially specific reverb sound components to be generated, which is indicated by the character "num_out", and the tap length, which is indicated by the character "len_ir". This tap length len_ir is the tap length of the impulse response as coefficient information used to generate the spatially specific reverb sound components.

[0116] Also, as the spatial reverb information, spatial reverb position information of the spatial specific reverb sounds is included for the number of spatial specific reverb sound components to be generated (num_out).

[0117] That is, the horizontal angle position_azimuth[i], the vertical angle position_elevation[i], and the radius position_radius[i] are stored as spatial reverb position information of the i-th spatial specific reverb sound component.

[0118] Furthermore, as coefficient information of the i-th space-specific reverberation sound component, impulse response coefficients impulse_response[i][j] are stored in the number corresponding to the tap length len_ir.

[0119] In the examples shown in Fig. 3 and Fig. 4, an example has been described in which an impulse response is used as coefficient information used to generate object-specific reverb sound components and space-specific reverb sound components. That is, an example has been described in which reverb processing is performed using sampling reverb. However, this is not limiting, and reverb processing may also be performed using other techniques such as parametric reverb. Furthermore, these coefficient information may be compressed using lossless coding techniques such as Huffman coding.

[0120] As described above, in the input bit stream, the information required for reverb processing is transmitted divided into information regarding direct sound (direct sound gain), information regarding object-specific reverb sounds such as object reverb information, and information regarding space-specific reverb sounds such as space reverb information.

[0121] Therefore, the information can be mixed and output at an appropriate transmission frequency for each piece of information such as information on direct sound, information on object-specific reverberation sound, and information on space-specific reverberation sound. In other words, in each frame of the audio object signal, only the necessary information such as information on direct sound can be selectively transmitted based on the relationship between the audio object and the listening position. This makes it possible to suppress the bit rate of the input bit stream and realize more efficient information transmission. In other words, it is possible to improve the coding efficiency.

[0122] <About the output audio signal> Next, a direct sound of an audio object, an object-specific reverb sound, and a space-specific reverb sound that are reproduced based on an output audio signal will be described.

[0123] The relationship between the position of an audio object and the position of an object reverb component is, for example, as shown in FIG.

[0124] Here, around a position OBJ11 of one audio object, there are object reverb component positions RVB11 to RVB14 of four object-specific reverb sounds for that audio object.

[0125] Here, the horizontal angle (azimuth) and vertical angle (elevation) indicating object reverb component position RVB11 to object reverb component position RVB14 are shown at the top of the figure. In this example, it can be seen that four object-specific reverb sound components are arranged with the origin O, which is the listening position, as the center.

[0126] The position of the object-specific reverb sound and the type of sound the object-specific reverb sound produces vary greatly depending on the position of the audio object in three-dimensional space. Therefore, object reverb information can be said to be reverb information that depends on the position of the audio object in space.

[0127] Therefore, in the input bitstream, object reverb information is not linked to audio objects, but is managed by reverb IDs.

[0128] When the core decode processor 21 reads out object reverb information from the input bit stream, the core decode processor 21 holds the read object reverb information for a certain period of time. In other words, the core decode processor 21 always holds object reverb information for a predetermined period of time in the past.

[0129] For example, assume that the value of the reuse flag use_prev is "1" at a given time, indicating that the object reverb information is to be reused.

[0130] In this case, the core decoding processor 21 obtains the reverb ID for a predetermined audio object from the input bitstream. That is, the reverb ID is read out.

[0131] The core decode processing unit 21 then reads out the object reverb information identified by the read reverb ID from among the past object reverb information it holds, and reuses that object reverb information as object reverb information for a specified audio object at a specified time.

[0132] By managing object reverb information by reverb ID in this way, for example, object reverb information transmitted as information about audio object OBJ1 can be reused as information about audio object OBJ2. This makes it possible to reduce the number of pieces of object reverb information temporarily stored in the core decoding processing unit 21, that is, the amount of data.

[0133] Generally, when an impulse is emitted into a space, in addition to the direct sound, early reflected sounds are generated by reflections from the floors and walls in the surrounding space, as shown in Figure 6, and later reverberation components are generated by repeated reflections.

[0134] Here, the portion indicated by the arrow Q11 indicates a direct sound component, and this direct sound component corresponds to the direct sound signal obtained by the amplifier unit 51.

[0135] Also, the portion indicated by the arrow Q12 indicates an early reflection sound component, which corresponds to the object-specific reverberation sound signal obtained by the object-specific reverberation processing unit 53. Furthermore, the portion indicated by the arrow Q13 indicates a late reverberation component, which corresponds to the space-specific reverberation sound signal obtained by the space-specific reverberation processing unit 55.

[0136] If such a relationship between the direct sound, the early reflection sound, and the late reverberation components is explained on a two-dimensional plane, it will look like, for example, Figures 7 and 8. In Figures 7 and 8, the same reference numerals are used to designate corresponding parts, and explanations thereof will be omitted as appropriate.

[0137] For example, suppose there are two audio objects OBJ21 and OBJ22 in an indoor space surrounded by walls represented by a rectangular frame, as shown in Fig. 7. Also suppose that a viewer U11 is at a reference viewing position.

[0138] Here, the distance from the viewer U11 to the audio object OBJ21 is R OBJ21 and the distance from the viewer U11 to the audio object OBJ22 is R OBJ22 Let us assume that.

[0139] In such a case, as shown in FIG. 8, the sound generated in the audio object OBJ21 and traveling directly toward the viewer U11 is called the direct sound D of the audio object OBJ21. OBJ21 Similarly, the sound generated by the audio object OBJ22 and traveling directly toward the viewer U11, which is indicated by the dashed arrow in the figure, is called the direct sound D OBJ22 It becomes.

[0140] In addition, in the figure, the sound generated by the audio object OBJ21, reflected once by the wall in the room, etc., and traveling toward the viewer U11 is the initial reflected sound E of the audio object OBJ21, which is depicted by the dotted arrow. OBJ21 Similarly, the sound generated by the audio object OBJ22, which is depicted by the dotted arrow in the figure, is reflected once by the walls of the room and then travels toward the viewer U11. This is called the initial reflected sound E of the audio object OBJ22. OBJ22 It becomes.

[0141] Furthermore, sound S generated by audio object OBJ21 is reflected by the walls of the room and reaches the viewer U11. OBJ21 The sound S is generated by the audio object OBJ22, is repeatedly reflected by the walls of the room, and reaches the viewer U11. OBJ22 The sound components consisting of these are the late reverberation components. Here, the late reverberation components are depicted by solid arrows.

[0142] Here, the distance R OBJ22 is the distance R OBJ21 , and audio object OBJ22 is located closer to viewer U11 than audio object OBJ21.

[0143] Therefore, for the audio object OBJ22, the early reflection sound E is heard by the viewer U11. OBJ22 More direct sound than D OBJ22 Therefore, for the reverb of the audio object OBJ22, the direct sound gain is set to a large value, and the object reverb sound gain and the spatial reverb gain are set to small values, and these gains are stored in the input bitstream.

[0144] In contrast, audio object OBJ21 is located farther from viewer U11 than audio object OBJ22.

[0145] Therefore, for the audio object OBJ21, the direct sound D is heard by the viewer U11. OBJ21 Early reflections E OBJ21 and the late reverberation component sound S OBJ21 Therefore, for the reverb of the audio object OBJ21, the direct sound gain is set to a small value, and the object reverb sound gain and the spatial reverb gain are set to large values, and these gains are stored in the input bit stream.

[0146] Furthermore, when the audio object OBJ21 or the audio object OBJ22 moves, the early reflection sound components change significantly depending on the positional relationship between the audio object and the surrounding space, that is, the walls and floor of the room.

[0147] Therefore, the object reverb information of the audio objects OBJ21 and OBJ22 needs to be transmitted with the same frequency as the object position information. Such object reverb information is information that is highly dependent on the positions of the audio objects.

[0148] On the other hand, since the late reverberation components are highly dependent on the materials of the space, such as the walls and floors, it is possible to ensure a sufficiently subjective quality by transmitting spatial reverb information at the minimum necessary frequency and controlling only the magnitude relationship according to the position of the audio object.

[0149] Therefore, for example, the spatial reverb information is transmitted to the signal processing device 11 less frequently than the object reverb information. In other words, the core decode processing unit 21 acquires the spatial reverb information less frequently than the object reverb information.

[0150] In this technology, the amount of information (data) required for reverb processing can be reduced by dividing the information required for reverb processing into sound components such as direct sound, object-specific reverb sound, and space-specific reverb sound.

[0151] Generally, sampling reverb requires long impulse response data of about 1 second, but by dividing the required information into sound components as in this technology, the impulse response can be realized as a combination of fixed delay and short impulse response data, reducing the amount of data. This makes it possible to reduce the number of biquad filter stages not only in sampling reverb but also in parametric reverb.

[0152] Moreover, with this technology, the information required for reverb processing is divided for each sound component and transmitted, so that the necessary information can be transmitted with the necessary frequency, thereby improving coding efficiency.

[0153] As described above, according to the present technology, when transmitting reverb information for controlling the sense of distance for a panning-based rendering method such as VBAP, it is possible to achieve high transmission efficiency even when a large number of audio objects exist.

[0154] <Description of Audio Output Processing> Next, a specific operation of the signal processing device 11 will be described. That is, hereinafter, audio output processing by the signal processing device 11 will be described with reference to the flowchart of FIG.

[0155] In step S11, the core decoding processing unit 21 decodes (data) the received input bit stream.

[0156] The core decoding processing unit 21 supplies the audio object signal obtained by decoding to amplification unit 51, amplification unit 52, and amplification unit 54, and also supplies the direct sound gain, object reverb sound gain, and spatial reverb gain obtained by decoding to amplification unit 51, amplification unit 52, and amplification unit 54, respectively.

[0157] The core decoding unit 21 also supplies the object reverb information and spatial reverb information obtained by decoding to the object specific reverb processing unit 53 and the spatial specific reverb processing unit 55. Furthermore, the core decoding unit 21 supplies the object position information obtained by decoding to the object specific reverb processing unit 53, the spatial specific reverb processing unit 55, and the rendering unit 56.

[0158] At this time, the core decode processing unit 21 temporarily holds the object reverb information read from the input bit stream.

[0159] In more detail, when the value of the reuse flag use_prev is "1", the core decoding processing unit 21 supplies the object reverb information it holds, which is identified by the reverb ID read from the input bitstream, to the object-specific reverb processing unit 53 as object reverb information of the audio object.

[0160] In step S12, the amplification unit 51 multiplies the audio object signal supplied from the core decoding processing unit 21 by the direct sound gain supplied from the core decoding processing unit 21 to perform gain adjustment to generate a direct sound signal and supplies it to the rendering unit 56.

[0161] In step S13, the object-specific reverb processing unit 53 generates an object-specific reverb sound signal.

[0162] That is, the amplifier 52 multiplies the audio object signal supplied from the core decoding processor 21 by the object reverb sound gain supplied from the core decoding processor 21 to adjust the gain, and supplies the result to the object-specific reverb processor 53.

[0163] The object-specific reverb processor 53 also performs reverb processing on the audio object signal supplied from the amplifier 52, based on the coefficient of the impulse response included in the object reverb information supplied from the core decode processor 21. That is, a convolution process is performed between the coefficient of the impulse response and the audio object signal, to generate an object-specific reverb sound signal.

[0164] Furthermore, the object-specific reverb processing unit 53 generates position information of the object-specific reverb sound based on the object position information supplied from the core decoding processing unit 21 and the object reverb position information included in the object reverb information, and supplies the obtained position information and the object-specific reverb sound signal to the rendering unit 56.

[0165] In step S14, the space-specific reverb processing unit 55 generates a space-specific reverb sound signal.

[0166] That is, the amplifier 54 multiplies the audio object signal supplied from the core decoding processor 21 by the spatial reverb gain supplied from the core decoding processor 21 to adjust the gain, and supplies the result to the spatial specific reverb processor 55.

[0167] Furthermore, the space-specific reverb processor 55 performs reverb processing on the audio object signal supplied from the amplifier 54, based on the coefficient of the impulse response included in the space reverb information supplied from the core decode processor 21. That is, a convolution process is performed between the coefficient of the impulse response and the audio object signal, and the signals obtained for each audio object by the convolution process are added together to generate a space-specific reverb sound signal.

[0168] Furthermore, the spatial-specific reverb processing unit 55 generates position information of the spatial-specific reverb sound based on the object position information supplied from the core decoding processing unit 21 and the spatial reverb position information contained in the spatial reverb information, and supplies the obtained position information and the spatial-specific reverb sound signal to the rendering unit 56.

[0169] In step S15, the rendering unit 56 performs rendering processing and outputs the resulting output audio signal.

[0170] That is, the rendering unit 56 performs rendering processing based on the object position information supplied from the core decode processing unit 21 and the direct sound signal supplied from the amplification unit 51. The rendering unit 56 also performs rendering processing based on the object-specific reverb sound signal and position information supplied from the object-specific reverb processing unit 53, and also performs rendering processing based on the space-specific reverb sound signal and position information supplied from the space-specific reverb processing unit 55.

[0171] The rendering unit 56 then adds the signals obtained by the rendering process of each sound component for each channel to generate a final output audio signal. The rendering unit 56 outputs the output audio signal thus obtained to the subsequent stage, and the audio output process is then completed.

[0172] In this manner, the signal processing device 11 performs reverberation processing and rendering processing based on audio object information including information divided into the direct sound, object-specific reverberation sound, and space-specific reverberation sound components, and generates an output audio signal. In this manner, it is possible to improve the coding efficiency of the input bitstream.

[0173] <Example of the configuration of the encoding device> Next, a coding device that generates and outputs an output bitstream from the input bitstream described above will be described.

[0174] Such an encoding device may be configured, for example, as shown in FIG.

[0175] The encoding device 101 shown in FIG. 10 includes an object signal encoding unit 111, an audio object information encoding unit 112, and a packing unit 113.

[0176] The object signal encoding unit 111 encodes the supplied audio object signal using a predetermined encoding method, and supplies the encoded audio object signal to a packing unit 113 .

[0177] The audio object information encoding unit 112 encodes the supplied audio object information and supplies the encoded information to a packing unit 113 .

[0178] The packing unit 113 stores the coded audio object signal supplied from the object signal coding unit 111 and the coded audio object information supplied from the audio object information coding unit 112 in a bitstream to generate an output bitstream. The packing unit 113 transmits the obtained output bitstream to the signal processing device 11.

[0179] <Description of Encoding Process> Next, a description will be given of the operation of the encoding device 101. That is, the encoding process by the encoding device 101 will be described below with reference to the flowchart in Fig. 11. For example, this encoding process is performed for each frame of an audio object signal.

[0180] In step S 41 , the object signal encoding unit 111 encodes the supplied audio object signal using a predetermined encoding method, and supplies the encoded signal to the packing unit 113 .

[0181] In step S 42 , the audio object information encoding unit 112 encodes the supplied audio object information and supplies the encoded audio object information to the packing unit 113 .

[0182] Here, audio object information, which includes object reverb information and spatial reverb information, is provided and coded in such a way that, for example, spatial reverb information is transmitted to the signal processing device 11 less frequently than object reverb information.

[0183] In step S43, the packing unit 113 stores the coded audio object signal supplied from the object signal coding unit 111 in a bitstream.

[0184] In step S44, the packing unit 113 stores the object position information contained in the encoded audio object information supplied from the audio object information encoding unit 112 in the bitstream.

[0185] In step S45, the packing unit 113 determines whether the encoded audio object information supplied from the audio object information encoding unit 112 includes reverb information.

[0186] Here, if the reverb information does not include either object reverb information or spatial reverb information, it is determined that there is no reverb information.

[0187] If it is determined in step S45 that there is no reverb information, then the process proceeds to step S46.

[0188] In step S46, the packing unit 113 sets the value of the reverb information flag flag_obj_reverb to "0" and stores the reverb information flag flag_obj_reverb in the bitstream. This results in an output bitstream that does not include reverb information being obtained. After the output bitstream is obtained, the process proceeds to step S54.

[0189] On the other hand, if it is determined in step S45 that reverb information is present, then the process proceeds to step S47.

[0190] In step S47, the packing unit 113 sets the value of the reverb information flag flag_obj_reverb to "1" and stores in the bitstream the reverb information flag flag_obj_reverb and the gain information included in the encoded audio object information supplied from the audio object information encoding unit 112. Here, the above-mentioned direct sound gain dry_gain[i], object reverb sound gain wet_gain[i], and spatial reverb gain room_gain[i] are stored in the bitstream as the gain information.

[0191] In step S48, the packing unit 113 determines whether or not to reuse the object reverb information.

[0192] For example, if the encoded audio object information supplied from the audio object information encoding unit 112 does not include object reverb information but does include a reverb ID, it is determined that reuse is to be performed.

[0193] If it is determined in step S48 that reuse is to be performed, then the process proceeds to step S49.

[0194] In step S49, the packing unit 113 sets the value of the reuse flag use_prev to "1" and stores in the bitstream the reuse flag use_prev and the reverb ID included in the encoded audio object information supplied from the audio object information encoding unit 112. After the reverb ID is stored, the process proceeds to step S51.

[0195] On the other hand, if it is determined in step S48 that reuse is not to be performed, then the process proceeds to step S50.

[0196] In step S50, the packing unit 113 sets the value of the reuse flag use_prev to "0" and stores in the bitstream the reuse flag use_prev and the object reverb information included in the encoded audio object information supplied from the audio object information encoding unit 112. After the object reverb information is stored, the process proceeds to step S51.

[0197] After the process of step S49 or step S50 is performed, the process of step S51 is performed.

[0198] That is, in step S51, the packing unit 113 determines whether or not the encoded audio object information supplied from the audio object information encoding unit 112 includes spatial reverb information.

[0199] If it is determined in step S51 that spatial reverb information is present, then the process proceeds to step S52.

[0200] In step S52, the packing unit 113 sets the value of the spatial reverb information flag flag_room_reverb to “1”, and stores the spatial reverb information flag flag_room_reverb and the spatial reverb information contained in the encoded audio object information supplied from the audio object information encoding unit 112 in the bitstream.

[0201] As a result, an output bitstream containing spatial reverb information is obtained. Once the output bitstream is obtained, the process proceeds to step S54.

[0202] On the other hand, if it is determined in step S51 that there is no spatial reverb information, then the process proceeds to step S53.

[0203] In step S53, the packing unit 113 sets the value of the spatial reverb information flag flag_room_reverb to "0" and stores the spatial reverb information flag flag_room_reverb in the bitstream. This results in an output bitstream that does not include spatial reverb information being obtained. After the output bitstream is obtained, the process proceeds to step S54.

[0204] After the process of step S46, step S52, or step S53 is performed to obtain an output bitstream, the process of step S54 is performed. Note that the output bitstream obtained by these processes is, for example, a bitstream in the format shown in Fig. 3 and Fig. 4.

[0205] In step S54, the packing unit 113 outputs the obtained output bitstream, and the encoding process ends.

[0206] In this manner, the encoding device 101 stores audio object information, which appropriately includes information divided into the direct sound, object-specific reverb sound, and space-specific reverb sound components, in a bitstream and outputs the audio object information. In this manner, it is possible to improve the encoding efficiency of the output bitstream.

[0207] In the above, an example has been described in which gain information such as a direct sound gain, an object reverb sound gain, and a spatial reverb gain is provided as audio object information, but these gain information may be generated on the decoding side.

[0208] In such a case, for example, the signal processing device 11 generates a direct sound gain, an object reverb sound gain, and a spatial reverb gain based on object position information, object reverb position information, spatial reverb position information, and the like included in the audio object information.

[0209] Example of computer configuration The above-mentioned series of processes can be executed by hardware or software. When the series of processes is executed by software, the program constituting the software is installed in a computer. Here, the computer includes a computer built into dedicated hardware, and a general-purpose personal computer, for example, capable of executing various functions by installing various programs.

[0210] FIG. 12 is a block diagram showing an example of the hardware configuration of a computer that executes the above-mentioned series of processes according to a program.

[0211] In the computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are interconnected via a bus 504.

[0212] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0213] The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, etc. The output unit 507 includes a display, a speaker, etc. The recording unit 508 includes a hard disk, a non-volatile memory, etc. The communication unit 509 includes a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.

[0214] In a computer configured as described above, the CPU 501 loads, for example, a program recorded in the recording unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and executes the program, thereby performing the series of processes described above.

[0215] The program executed by the computer (CPU 501) can be provided by being recorded on a removable recording medium 511 such as a package medium, for example. The program can also be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.

[0216] In the computer, the program can be installed in the recording unit 508 via the input / output interface 505 by mounting the removable recording medium 511 in the drive 510. The program can also be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. Alternatively, the program can be installed in the ROM 502 or the recording unit 508 in advance.

[0217] In addition, the program executed by the computer may be a program in which processing is performed chronologically in the order described in this specification, or it may be a program in which processing is performed in parallel or at the required timing, such as when called.

[0218] Furthermore, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the gist of the present technology.

[0219] For example, the present technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0220] Furthermore, each step described in the above flow chart can be executed by one device, or can be shared and executed by a plurality of devices.

[0221] Furthermore, when a single step includes multiple processes, the multiple processes included in the single step can be executed by a single device, or can be shared and executed by multiple devices.

[0222] Furthermore, the present technology can also be configured as follows.

[0223] (1) an acquisition unit that acquires reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object; a reverb processing unit that generates a reverb component signal of the audio object based on the reverb information and the audio object signal; A signal processing device comprising: (2) The spatial reverb information is acquired less frequently than the object reverb information. A signal processing device as described in (1). (3) When identification information indicating the past reverb information is acquired by the acquisition unit, the reverb processing unit generates a signal of the reverb component based on the reverb information indicated by the identification information and the audio object signal. A signal processing device according to (1) or (2). (4) the identification information is information indicating the object reverb information, The reverb processing unit generates a signal of the reverb component based on the object reverb information indicated by the identification information, the spatial reverb information, and the audio object signal. A signal processing device according to (3). (5) The object reverb information is information that depends on the position of the audio object. A signal processing device according to any one of (1) to (4). (6) The reverb processing unit includes: generating a signal of the reverb component specific to the space based on the spatial reverb information and the audio object signal; generating a signal of the reverb component specific to the audio object based on the object reverb information and the audio object signal; A signal processing device according to any one of (1) to (5). (7) A signal processing device, obtaining reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to said audio object, and an audio object signal of said audio object; generating a signal of a reverb component of the audio object based on the reverb information and the audio object signal; Signal processing methods. (8) obtaining reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to said audio object, and an audio object signal of said audio object; generating a signal of a reverb component of the audio object based on the reverb information and the audio object signal; A program that causes a computer to execute a process including steps. [Explanation of symbols]

[0224] 11 signal processing device, 21 core decode processing unit, 22 rendering processing unit, 51-1, 51-2, 51 amplification unit, 52-1, 52-2, 52 amplification unit, 53-1, 53-2, 53 object specific reverberation processing unit, 54-1, 54-2, 54 amplification unit, 55 space specific reverberation processing unit, 56 rendering unit, 101 encoding device, 111 object signal encoding unit, 112 audio object information encoding unit, 113 packing unit

Claims

1. an acquisition unit that acquires reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to the audio object, and an audio object signal of the audio object; a reverb processing unit that generates a reverb component signal of the audio object based on the reverb information and the audio object signal; A rendering part that performs rendering processing using VBAP Equipped with When identification information indicating the past reverb information is acquired by the acquisition unit, the reverb processing unit generates a signal of the reverb component based on the reverb information indicated by the identification information and the audio object signal. Signal processing device.

2. The object reverb information is information that depends on the position of the audio object. The signal processing device according to claim 1 .

3. The reverb processing unit includes: generating a signal of the reverb component specific to the space based on the spatial reverb information and the audio object signal; generating a signal of the reverb component specific to the audio object based on the object reverb information and the audio object signal; The signal processing device according to claim 1 .

4. A signal processing device, obtaining reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to said audio object, and an audio object signal of said audio object; generating a signal of a reverb component of the audio object based on the reverb information and the audio object signal; Rendering process using VBAP Steps include: When identification information indicating the past reverb information is acquired, a signal of the reverb component is generated based on the reverb information indicated by the identification information and the audio object signal. Signal processing methods.

5. obtaining reverb information including at least one of spatial reverb information specific to a space surrounding an audio object and object reverb information specific to said audio object, and an audio object signal of said audio object; generating a signal of a reverb component of the audio object based on the reverb information and the audio object signal; Rendering process using VBAP causing a computer to execute a process including steps; When identification information indicating the past reverb information is acquired, a signal of the reverb component is generated based on the reverb information indicated by the identification information and the audio object signal. program.

Images