Audio signal downmixing method, audio signal downmixing device and program

Abstract

To provide a technique for obtaining a monaural signal useful for signal processing, such as coding, from a two-channel sound signal.SOLUTION: A sound signal downmixing device obtains a downmixed signal, which is a mixed signal of a left channel input sound signal and a right channel input sound signal, and includes: a left-right relationship information acquisition unit 185 that obtains preceding channel information, which is information indicating whether the left channel input sound signal or the right channel input sound signal is preceding, and a left-right correlation coefficient, which is the correlation coefficient between the left channel input sound signal and the right channel input sound signal; and a downmix unit 112 that obtains a downmix signal by weighted averaging the left channel input sound signal and the right channel input sound signal so that the preceding channel input sound signal among the left channel input sound signal and the right channel input sound signal is largely included as the left-right correlation coefficient is larger, based on the preceding channel information and the left-right correlation coefficient.SELECTED DRAWING: Figure 22

Description

[Technical field]

[0001] The present invention relates to a technique for obtaining a monaural sound signal from a two-channel sound signal in order to encode the sound signal in monaural, encode the sound signal using a combination of monaural and stereo encoding, process the sound signal in monaural, or process a stereo sound signal using a monaural sound signal. [Background technology]

[0002] A technology for obtaining a monaural sound signal from two-channel sound signals and for embedding / decoding the two-channel sound signals and the monaural sound signal is disclosed in Patent Document 1. Patent Document 1 discloses a technology for obtaining a monaural signal by averaging an input left channel sound signal and an input right channel sound signal for each corresponding sample, encoding the monaural signal (monaural encoding) to obtain a monaural code, decoding the monaural code (monaural decoding) to obtain a monaural locally decoded signal, and encoding the difference (prediction residual signal) between the input sound signal and a prediction signal obtained from the monaural locally decoded signal for each of the left channel and the right channel. In the technology of Patent Document 1, for each channel, a signal obtained by delaying and giving an amplitude ratio to a monaural locally decoded signal is used as a predicted signal, and a predicted signal having a delay and amplitude ratio that minimizes the error between an input sound signal and the predicted signal is selected, or a predicted signal having a delay difference and amplitude ratio that maximizes the cross-correlation between an input sound signal and the monaural locally decoded signal is used, and a predicted signal is subtracted from the input sound signal to obtain a predicted residual signal, and the predicted residual signal is subjected to encoding / decoding, thereby suppressing deterioration in sound quality of the decoded sound signal of each channel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2006-070751 publication Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology of Patent Document 1, the delay and amplitude ratio given to the monaural locally decoded signal when obtaining a predicted signal are optimized, thereby improving the coding efficiency of each channel. However, in the technology of Patent Document 1, the monaural locally decoded signal is obtained by encoding and decoding a monaural signal obtained by averaging a left channel sound signal and a right channel sound signal. In other words, the technology of Patent Document 1 has a problem in that it does not devise a way to obtain a monaural signal useful for signal processing such as coding processing from two channel sound signals. An object of the present invention is to provide a technique for obtaining a monaural signal useful for signal processing such as encoding processing from a two-channel sound signal. [Means for solving the problem]

[0005] One aspect of the present invention is a sound signal downmixing method for obtaining a downmix signal, which is a signal obtained by mixing a first channel input sound signal and a second channel input sound signal, the method comprising the steps of: acquiring preceding channel information, which is information indicating which of the first channel input sound signal and the second channel input sound signal is preceding; Magnitude of correlation and a downmix step of obtaining a downmix signal from the preceding channel and the other channel based on a degree determined based on a relation coefficient, which is a coefficient indicating In the downmix step, the first channel input sound signal and the second channel input sound signal, whichever is earlier, are given a greater weight than the other signal. . Effect of the Invention

[0008] According to the present invention, a monaural signal useful for signal processing such as encoding processing can be obtained from a two-channel sound signal. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of an encoding device according to a first reference embodiment and a second embodiment. [Diagram 2] 4 is a flowchart showing an example of processing by the encoding device of the first reference embodiment. [Diagram 3] FIG. 1 is a block diagram showing an example of a decoding device according to a first reference embodiment. [Figure 4] 10 is a flowchart showing an example of processing by a decoding device according to a first reference embodiment. [Diagram 5] 11 is a flowchart showing an example of processing by a left channel subtraction gain estimating section and a right channel subtraction gain estimating section of the first reference embodiment. [Figure 6] 11 is a flowchart showing an example of processing by a left channel subtraction gain estimating section and a right channel subtraction gain estimating section of the first reference embodiment. [Figure 7] 11 is a flowchart showing an example of processing by a left channel subtraction gain decoding unit and a right channel subtraction gain decoding unit of the first reference embodiment. [Figure 8] 11 is a flowchart showing an example of processing by a left channel subtraction gain estimating section and a right channel subtraction gain estimating section of the first reference embodiment. [Figure 9] 11 is a flowchart showing an example of processing by a left channel subtraction gain estimating section and a right channel subtraction gain estimating section of the first reference embodiment. [Figure 10] FIG. 11 is a block diagram showing an example of an encoding device according to a second reference embodiment and a first embodiment. [Figure 11] 13 is a flowchart showing an example of processing by an encoding device of the second reference embodiment. [Figure 12] FIG. 13 is a block diagram showing an example of a decoding device according to a second reference embodiment. [Figure 13] 13 is a flowchart showing an example of processing by a decoding device according to a second reference embodiment. [Figure 14] 4 is a flowchart showing an example of processing performed by the encoding device of the first embodiment. [Figure 15] 10 is a flowchart showing an example of processing performed by an encoding device according to the second embodiment. [Figure 16] FIG. 13 is a block diagram illustrating an example of an encoding device according to a third embodiment. [Figure 17] 13 is a flowchart showing an example of processing of an encoding device according to a third embodiment. [Figure 18] FIG. 13 is a block diagram showing an example of a sound signal encoding device according to a fourth embodiment. [Figure 19] 13 is a flowchart showing an example of processing performed by a sound signal encoding device according to a fourth embodiment. [Figure 20] FIG. 13 is a block diagram showing an example of a sound signal processing device according to a fourth embodiment. [Figure 21] 13 is a flowchart showing an example of processing by the sound signal processing device of the fourth embodiment. [Figure 22] FIG. 13 is a block diagram showing an example of a sound signal downmixing device according to a fourth embodiment. [Figure 23] 13 is a flowchart showing an example of processing performed by the sound signal downmixing device of the fourth embodiment. [Figure 24] FIG. 2 is a diagram illustrating an example of a functional configuration of a computer that realizes each device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] First, let me explain the notation used in the specification. A superscript "^" such as ^x for a certain letter x should be written directly above the "x". However, due to the restrictions on notation used in the specification, it is sometimes written as ^x. <First reference form> Before describing the embodiments of the invention, an encoding device and a decoding device in a form that is the basis for implementing the invention of the second embodiment and the invention of the first embodiment will be described as a first reference embodiment and a second reference embodiment. Note that in the specification and claims, the encoding device may be referred to as an audio signal encoding device, the encoding method as an audio signal encoding method, the decoding device as an audio signal decoding device, and the decoding method as an audio signal decoding method.

[0011] ≪Encoding device 100≫ As shown in Fig. 1, the encoding device 100 of the first reference embodiment includes a downmix unit 110, a left channel subtraction gain estimation unit 120, a left channel signal subtraction unit 130, a right channel subtraction gain estimation unit 140, a right channel signal subtraction unit 150, a monaural encoding unit 160, and a stereo encoding unit 170. The encoding device 100 encodes an input two-channel stereo time domain sound signal in frame units of a predetermined time length, for example, 20 ms, and obtains and outputs a monaural code CM, a left channel subtraction gain code Cα, a right channel subtraction gain code Cβ, and a stereo code CS, which will be described later. The two-channel stereo time domain sound signal input to the encoding device is, for example, a digital voice signal or audio signal obtained by collecting sounds such as speech or music with two microphones and performing AD conversion, and is composed of a left channel input sound signal and a right channel input sound signal. The codes output by the encoding device, i.e., the monaural code CM, the left channel subtraction gain code Cα, the right channel subtraction gain code Cβ, and the stereo code CS, are input to a decoding device. The encoding device 100 performs the processes of steps S110 to S170 illustrated in FIG. 2 for each frame.

[0012] [Downmix section 110] The downmix unit 110 receives the left channel input sound signal input to the encoding device 100 and the right channel input sound signal input to the encoding device 100. The downmix unit 110 obtains a downmix signal, which is a signal obtained by mixing the left channel input sound signal and the right channel input sound signal, from the input left channel input sound signal and right channel input sound signal, and outputs the downmix signal (step S110).

[0013] For example, if the number of samples per frame is T, the downmix unit 110 receives the left channel input sound signal x L (1), x L (2), ..., x L (T) and the right channel input sound signal x R (1), x R (2), ..., x RThe downmix unit 110 receives the left channel input sound signal and the right channel input sound signal, and converts the input sound signal into a downmix signal x M (1), x M (2), ..., x M (T) is obtained and output. That is, if each sample number is t, then x M (t)=(x L (t)+x R (t) / 2.

[0014] [Left channel subtraction gain estimation unit 120] The left channel subtraction gain estimation unit 120 receives the left channel input sound signal x L (1), x L (2), ..., x L (T) and the downmix signal x output by the downmix unit 110 M (1), x M (2), ..., x M (T) is input. The left channel subtraction gain estimation unit 120 obtains and outputs a left channel subtraction gain α and a left channel subtraction gain code Cα, which is a code representing the left channel subtraction gain α, from the input left channel input sound signal and downmix signal (step S120). The left channel subtraction gain estimation unit 120 obtains the left channel subtraction gain α and the left channel subtraction gain code Cα by a well-known method such as the method for obtaining the amplitude ratio g in Patent Document 1 and the method for encoding the amplitude ratio g, or a newly invented method based on the principle of minimizing the quantization error. The principle of minimizing the quantization error and the method based on this principle will be described later.

[0015] [Left channel signal subtraction unit 130] The left channel signal subtraction unit 130 subtracts the left channel input sound signal x L (1), x L (2), ..., x L(T) and the downmix signal x output by the downmix unit 110 M (1), x M (2), ..., x M (T) and the left channel subtraction gain α output by the left channel subtraction gain estimation unit 120. The left channel signal subtraction unit 130 subtracts the sample value x M (t) multiplied by the left channel subtraction gain α, α×x M (t) is the sample value x of the input sound signal in the left channel. L The value x subtracted from (t) L (t)-α×x M (t) is the left channel difference signal y L (1), y L (2), ..., y L (T) and output it (step S130). L (t)=x L (t)-α×x M In the encoding device 100, in order to avoid delay and an amount of calculation processing required for obtaining a locally decoded signal, the left channel signal subtraction unit 130 subtracts the unquantized downmix signal x obtained by the downmix unit 110, instead of the quantized downmix signal which is the locally decoded signal of the monaural encoding. M However, when left channel subtraction gain estimation unit 120 obtains left channel subtraction gain α by a well-known method such as that illustrated in Patent Document 1, rather than a method based on the principle of minimizing quantization error, a means for obtaining a locally decoded signal corresponding to monaural code CM is provided downstream of monaural coding unit 160 of coding device 100 or within monaural coding unit 160, and left channel signal subtraction unit 130 calculates downmix signal x M (1), x M (2), ..., x M Instead of (T), a quantized downmix signal ^x, which is a locally decoded signal of monaural coding, is used, as in the conventional coding device of Patent Document 1. M (1), ^x M (2), ..., ^x M (T) may be used to obtain the left channel difference signal.

[0016] [Right channel subtraction gain estimation unit 140] The right channel subtraction gain estimation unit 140 receives the right channel input sound signal x R (1), x R (2), ..., x R (T) and the downmix signal x output by the downmix unit 110 M (1), x M (2), ..., x M (T) is input. The right channel subtraction gain estimation unit 140 obtains and outputs a right channel subtraction gain β and a right channel subtraction gain code Cβ, which is a code representing the right channel subtraction gain β, from the input right channel input sound signal and downmix signal (step S140). The right channel subtraction gain estimation unit 140 obtains the right channel subtraction gain β and the right channel subtraction gain code Cβ by a well-known method such as the method for obtaining the amplitude ratio g in Patent Document 1 and the method for encoding the amplitude ratio g, or a newly invented method based on the principle of minimizing the quantization error. The principle of minimizing the quantization error and the method based on this principle will be described later.

[0017] [Right channel signal subtraction unit 150] The right channel signal subtraction unit 150 subtracts the right channel input sound signal x R (1), x R (2), ..., x R (T) and the downmix signal x output by the downmix unit 110 M (1), x M (2), ..., x M (T) and the right channel subtraction gain β output by the right channel subtraction gain estimation unit 140. The right channel signal subtraction unit 150 subtracts the sample value x M (t) multiplied by the right channel subtraction gain β, β×x M (t) is the sample value x of the input sound signal on the right channel. R The value x subtracted from (t) R (t)-β×xM (t) is the right channel difference signal y R (1), y R (2), ..., y R (T) and output it (step S150). R (t)=x R (t)-β×x M In order to avoid delay or the amount of calculation required to obtain a locally decoded signal in encoding device 100, right channel signal subtraction unit 150, like left channel signal subtraction unit 130, subtracts the unquantized downmix signal x obtained by downmix unit 110 instead of the quantized downmix signal that is the locally decoded signal of monaural encoding. M However, when right channel subtraction gain estimation unit 140 obtains right channel subtraction gain β by a well-known method such as that illustrated in Patent Document 1, rather than a method based on the principle of minimizing quantization error, a means for obtaining a locally decoded signal corresponding to monaural code CM is provided downstream of monaural coding unit 160 in coding device 100 or within monaural coding unit 160, and right channel signal subtraction unit 150 subtracts downmix signal x M (1), x M (2), ..., x M Instead of (T), a quantized downmix signal ^x, which is a locally decoded signal of monaural coding, is used, as in the conventional coding device of Patent Document 1. M (1), ^x M (2), ..., ^x M (T) may be used to obtain the right channel difference signal.

[0018] [Monaural encoding unit 160] The monaural encoding unit 160 receives the downmix signal x output from the downmix unit 110. M (1), x M (2), ..., x M The mono encoding unit 160 encodes the input downmix signal in a predetermined encoding format. MIn other words, the input downmix signal x M (1), x M (2), ..., x M (T) to b M The 1-bit monaural code CM is obtained and output. Any encoding method may be used, for example, an encoding method according to the 3GPP EVS standard may be used.

[0019] [Stereo encoding unit 170] The stereo encoding unit 170 receives the left channel difference signal y L (1), y L (2), ..., y L (T) and the right channel difference signal y R (1), y R (2), ..., y R The stereo encoding unit 170 sums the input left channel difference signal and right channel difference signal by a predetermined encoding method to obtain b s The left channel difference signal y L (1), y L (2), ..., y L (T) and the right channel difference signal y R (1), y R (2), ..., y R (T) and the total b S The stereo code CS of 128 bits is obtained and output. Any encoding method may be used, for example, a stereo encoding method corresponding to the stereo decoding method of the MPEG-4 AAC standard may be used, or a method for encoding the input left channel difference signal and right channel difference signal independently may be used, and the combination of all the codes obtained by encoding may be used as the stereo code CS.

[0020] When the input left channel difference signal and the right channel difference signal are encoded independently, the stereo encoding unit 170 encodes the left channel difference signal as b L bit, and the right channel difference signal is denoted by b R That is, the stereo encoding unit 170 encodes the input left channel difference signal y L (1), y L (2), ..., y L (T) to b L The left channel difference code CL of 10 bits is obtained, and the right channel difference signal y R (1), y R (2), ..., y R (T) to b R The left channel difference code CL and the right channel difference code CR are combined and output as the stereo code CS. L Bit and b R The sum of the bits is b S It is a bit.

[0021] When the input left channel difference signal and the right channel difference signal are jointly coded in one coding scheme, the stereo coding unit 170 converts the left channel difference signal and the right channel difference signal into a sum of b S That is, the stereo encoding unit 170 encodes the input left channel difference signal y L (1), y L (2), ..., y L (T) and the right channel difference signal y R (1), y R (2), ..., y R (T) and b S The 16-bit stereo code CS is obtained and output.

[0022] <Decryption device 200> As shown in Fig. 3, the decoding device 200 of the first reference embodiment includes a monaural decoding unit 210, a stereo decoding unit 220, a left channel subtraction gain decoding unit 230, a left channel signal adding unit 240, a right channel subtraction gain decoding unit 250, and a right channel signal adding unit 260. The decoding device 200 decodes the input monaural code CM, the left channel subtraction gain code Cα, the right channel subtraction gain code Cβ, and the stereo code CS in units of frames having the same time length as the corresponding encoding device 100, and obtains and outputs a frame-by-frame two-channel stereo time-domain decoded sound signal (a left channel decoded sound signal and a right channel decoded sound signal, which will be described later). The decoding device 200 may also output a monaural time-domain decoded sound signal (a monaural decoded sound signal, which will be described later), as indicated by a dashed line in Fig. 3. The decoded sound signal output by the decoding device 200 is made audible by, for example, being DA converted and being played on a speaker. The decoding device 200 performs the process of steps S210 to S260 illustrated in FIG. 4 for each frame.

[0023] [Monaural decoding unit 210] The monaural code CM input to the decoding apparatus 200 is input to the monaural decoding section 210. The monaural decoding section 210 decodes the input monaural code CM using a predetermined decoding method to generate a monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M (T) is obtained and output (step S210). As the predetermined decoding method, a decoding method corresponding to the encoding method used in the monaural encoding unit 160 of the corresponding encoding device 100 is used. The number of bits of the monaural code CM is b M It is.

[0024] [Stereo Decoding Unit 220] The stereo code CS input to the decoding apparatus 200 is input to the stereo decoding unit 220. The stereo decoding unit 220 decodes the input stereo code CS using a predetermined decoding method to generate a left channel decoded differential signal ^y L (1), ^y L (2), ..., ^y L (T) and the right channel decoded differential signal ^y R(1), ^y R (2), ..., ^y R (T) and output it (step S220). As the predetermined decoding method, a decoding method corresponding to the encoding method used in the stereo encoding unit 170 of the corresponding encoding device 100 is used. The total number of bits of the stereo code CS is b S It is.

[0025] [Left channel subtraction gain decoding unit 230] The left channel subtraction gain decoding unit 230 receives the left channel subtraction gain code Cα input to the decoding device 200. The left channel subtraction gain decoding unit 230 decodes the left channel subtraction gain code Cα to obtain and output the left channel subtraction gain α (step S230). The left channel subtraction gain decoding unit 230 decodes the left channel subtraction gain code Cα using a decoding method corresponding to the method used by the left channel subtraction gain estimation unit 120 of the corresponding encoding device 100 to obtain the left channel subtraction gain α. A method for the left channel subtraction gain decoding unit 230 to decode the left channel subtraction gain code Cα to obtain the left channel subtraction gain α in a case where the left channel subtraction gain estimation unit 120 of the corresponding encoding device 100 obtains the left channel subtraction gain α and the left channel subtraction gain code Cα using a method based on the principle of minimizing quantization error will be described later.

[0026] [Left channel signal adder 240] The left channel signal adder 240 receives the monaural decoded sound signal ^x output by the monaural decoding unit 210. M (1), ^x M (2), ..., ^x M (T) and the left channel decoded differential signal ^y L (1), ^y L (2), ..., ^y L (T) and the left channel subtraction gain α output by left channel subtraction gain decoding unit 230. Left channel signal adder 240 receives sample value ^y L (t) and the sample value ^x of the monaural decoded sound signal M (t) multiplied by the left channel subtraction gain α is α×^xM (t) and the sum of ^y L (t)+α×^x M (t) is converted into the left channel decoded sound signal ^x L (1), ^x L (2), ..., ^x L (T) is obtained and output (step S240). L (t)=^y L (t)+α×^x M (t).

[0027] [Right channel subtraction gain decoding unit 250] The right channel subtraction gain decoding unit 250 receives the right channel subtraction gain code Cβ input to the decoding device 200. The right channel subtraction gain decoding unit 250 decodes the right channel subtraction gain code Cβ to obtain and output the right channel subtraction gain β (step S250). The right channel subtraction gain decoding unit 250 decodes the right channel subtraction gain code Cβ using a decoding method corresponding to the method used by the right channel subtraction gain estimation unit 140 of the corresponding encoding device 100 to obtain the right channel subtraction gain β. A method in which the right channel subtraction gain decoding unit 250 decodes the right channel subtraction gain code Cβ to obtain the right channel subtraction gain β in a case in which the right channel subtraction gain estimation unit 140 of the corresponding encoding device 100 obtains the right channel subtraction gain β and the right channel subtraction gain code Cβ using a method based on the principle of minimizing quantization error will be described later.

[0028] [Right channel signal adder 260] The right channel signal adder 260 receives the monaural decoded sound signal ^x output by the monaural decoding unit 210. M (1), ^x M (2), ..., ^x M (T) and the right channel decoded differential signal ^y R (1), ^y R (2), ..., ^y R (T) and the right channel subtraction gain β output by right channel subtraction gain decoding unit 250. Right channel signal adding unit 260 receives sample value ^yR (t) and the sample value ^x of the monaural decoded sound signal M (t) multiplied by the right channel subtraction gain β is β×^x M (t) and the sum of ^y R (t)+β×^x M The sequence by (t) is converted into the right channel decoded sound signal ^x R (1), ^x R (2), ..., ^x R (T) and output it (step S260). R (t)=^y R (t)+β×^x M (t).

[0029] [Principle of minimizing quantization error] The principle of minimizing the quantization error will be described below. When the left channel difference signal and the right channel difference signal input to the stereo encoding unit 170 are encoded together in one encoding method, the number of bits b used to encode the left channel difference signal is L and the number of bits b used to encode the right channel difference signal. R Although it is possible that b is not explicitly defined, in the following, it is assumed that the number of bits used to encode the left channel difference signal is b L and the number of bits used to encode the right channel difference signal is b R In the following, the left channel will be mainly described, but the same applies to the right channel.

[0030] The above-described encoding device 100 receives a left channel input sound signal x L (1), x L (2), ..., x L From each sample value of (T), the downmix signal x M (1), x M (2), ..., x M The left channel difference signal y L (1), y L (2), ..., y L (T) to bL The downmix signal x is encoded with M (1), x M (2), ..., x M (T) to b M The above-mentioned decoding device 200 encodes the data with b L Bit code to left channel decoded difference signal ^y L (1), ^y L (2), ..., ^y L (T) (hereinafter also referred to as the “quantized left channel difference signal”), and M From the bit code to the mono decoded sound signal ^x M (1), ^x M (2), ..., ^x M (T) (hereinafter also referred to as the “quantized downmix signal”), the quantized downmix signal ^x obtained by the decoding is M (1), ^x M (2), ..., ^x M The quantized left channel difference signal ^y obtained by multiplying each sample value of (T) by the left channel subtraction gain α is L (1), ^y L (2), ..., ^y L By adding each sample value of (T) to the left channel decoded sound signal, the left channel decoded sound signal ^x L (1), ^x L (2), ..., ^x L Encoding apparatus 100 and decoding apparatus 200 should be designed so that the energy of the quantization error in the decoded sound signal in the left channel obtained by the above process is small.

[0031] The energy of the quantization error (hereinafter, for convenience, referred to as the "quantization error caused by encoding") in the decoded signal obtained by encoding and decoding the input signal is roughly proportional to the energy of the input signal in many cases, and tends to be exponentially smaller with respect to the number of bits per sample used for encoding. Therefore, the average energy per sample of the quantization error caused by encoding the left channel difference signal is a positive number σ L2 The average energy per sample of the quantization error caused by the coding of the downmix signal is a positive number σ M 2 It can be estimated using the following equation (1-0-2).

number

number

[0032] Here, let us assume that the left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x M Assume that the sample values ​​are close enough to be considered as the same sequence. For example, the left channel input sound signal x L (1), x L (2), ..., x L (T) and the right channel input signal x R (1), x R (2), ..., x R This condition corresponds to the case where (T) is obtained by capturing sounds emitted by a sound source equidistant from two microphones in an environment with little background noise or reverberation. Under this condition, the left channel difference signal y L (1), y L (2), ..., y L Each sample value of (T) is the downmix signal x M (1), x M (2), ..., x M The energy of the left channel difference signal is equivalent to the value obtained by multiplying each sample value of (T) by (1-α). Therefore, the energy of the left channel difference signal is (1-α) times the energy of the downmix signal. 2 Since it can be expressed as a factor of σ L 2 is the above σ M2 Using (1-α) 2 ×σ M 2 Therefore, the average energy per sample of the quantization error caused by encoding the left channel difference signal can be estimated as shown in the following equation (1-1).

number

number

[0033] Assuming that there is no correlation between the quantization error generated by encoding the left channel difference signal and the quantization error of the series of values ​​obtained by multiplying each sample value of the quantized downmix signal obtained by decoding by the left channel subtraction gain α, the average energy per sample of the quantization error of the decoded sound signal of the left channel is estimated by the sum of equations (1-1) and (1-2). The left channel subtraction gain α that minimizes the energy of the quantization error of the decoded sound signal of the left channel is obtained by the following equation (1-3).

number

[0034] That is, the left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x MIn order to minimize the quantization error of the decoded sound signal of the left channel under the condition that the sample values ​​of (T) are close enough to be considered as the same sequence, the left channel subtraction gain estimation unit 120 can calculate the left channel subtraction gain α by equation (1-3). The left channel subtraction gain α obtained by equation (1-3) is a value that is greater than 0 and less than 1, and is the number of bits used for the two encodings, b L and b M is equal to 0.5, and the number of bits b L is the number of bits b for encoding the downmix signal. M The larger the value, the closer to 0 than 0.5. The number of bits b M is the number of bits b for encoding the left channel difference signal. L The more, the closer to 1 the value is than 0.5.

[0035] The same is true for the right channel, where the right channel input sound signal x R (1), x R (2), ..., x R (T) and downmix signal x M (1), x M (2), ..., x M In order to minimize the quantization error of the right channel decoded sound signal under the condition that the sample values ​​of (T) are close enough to be considered to be the same sequence, right channel subtraction gain estimation unit 140 may calculate the right channel subtraction gain β by the following equation (1-3-2).

number

[0036] Next, the left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x M The principle of minimizing the energy of the quantization error in the decoded sound signal of the left channel, including the case where (T) cannot be regarded as the same sequence, will be described.

[0037] Left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x M The normalized dot product value r of (T) L is expressed by the following equation (1-4).

number

[0038] Left channel input sound signal x L (1), x L (2), ..., x L (T) is x for each sample number t. L (t)=r L x M (t)+(x L (t)-r L x M (t)), where x L (t)-r L x M The sequence composed of each value of (t) is expressed as an orthogonal signal x L '(1), x L '(2), ..., x L '(T), then according to the decomposition, each sample value y L (t)=x L (t)-αx M (t) is the downmix signal x M (1), x M (2), ..., x M Each sample value x of (T) M (t) is the normalized dot product value r L and the left channel subtraction gain α was used (r L -α) to obtain the value (r L -α)×x M (t) and each sample value x of the quadrature signal L '(t) and the sum (r L -α)×x M (t)+x L '(t) is equivalent to the quadrature signal x L '(1), x L '(2), ..., x L'(T) is the downmix signal x M (1), x M (2), ..., x M (T), that is, the inner product is zero. Therefore, the energy of the left channel difference signal is calculated by multiplying the energy of the downmix signal by (r L -α) 2 The left channel difference signal is expressed as the sum of the energy of the quadrature signal and the energy of the quadrature signal. L The average energy per sample of the quantization error introduced by coding with bits is a positive number σ 2 can be estimated using the following equation (1-5):

number

[0039] Assuming that there is no correlation between the quantization error generated by encoding the left channel difference signal and the quantization error of the series of values ​​obtained by multiplying each sample value of the quantized downmix signal obtained by decoding by the left channel subtraction gain α, the average energy per sample of the quantization error of the left channel decoded sound signal is estimated by the sum of equations (1-5) and (1-2). The left channel subtraction gain α that minimizes the energy of the quantization error of the left channel decoded sound signal is obtained by the following equation (1-6).

number

[0040] That is, in order to minimize the quantization error of the decoded sound signal of the left channel, the left channel subtraction gain estimation unit 120 needs to obtain the left channel subtraction gain α by the equation (1-6). In other words, considering the principle of minimizing the energy of the quantization error, the left channel subtraction gain α is calculated by subtracting the normalized inner product value r L and the number of bits used for encoding, b L and b MThe correction factor should be greater than 0 and less than 1, and should be multiplied by the number of bits b L and the number of bits for encoding the downmix signal b M is the same, and the number of bits b L is the number of bits b for encoding the downmix signal. M The number of bits b for encoding the left channel difference signal is greater than 0.5 and closer to 0. L is the number of bits b for encoding the downmix signal. M The smaller the value, the closer it is to 1 than 0.5.

[0041] The same applies to the right channel. In order to minimize the quantization error of the decoded sound signal of the right channel, right channel subtraction gain estimating section 140 should find right channel subtraction gain β by the following equation (1-6-2).

number

number

[0042] [Estimation and decoding of subtraction gain based on the principle of minimizing quantization error] A specific example of estimation and decoding of subtraction gain based on the principle of minimizing the quantization error described above will be described. In each example, a left channel subtraction gain estimation unit 120 and a right channel subtraction gain estimation unit 140 that estimate subtraction gains in encoding device 100, and a left channel subtraction gain decoding unit 230 and a right channel subtraction gain decoding unit 250 that decode the subtraction gains in decoding device 200 will be described.

[0043] [Example 1] Example 1 shows the input sound signal x in the left channel. L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x M The principle of minimizing the energy of the quantization error of the decoded sound signal of the left channel, including the case where (T) cannot be considered as the same sequence, and the principle of minimizing the energy of the quantization error of the decoded sound signal of the right channel input sound signal x R (1), x R (2), ..., x R (T) and downmix signal x M (1), x M (2), ..., x M This is based on the principle of minimizing the energy of the quantization error in the decoded sound signal of the right channel, even in cases where (T) cannot be regarded as the same sequence.

[0044] [[[Left channel subtraction gain estimator 120]]] The left channel subtraction gain estimation unit 120 includes a left channel subtraction gain candidate α cand (a) and the code Cα corresponding to the candidate cand A plurality of pairs (A pairs, a=1, . . . , A) of pairs with (a) are stored in advance. Left channel subtraction gain estimation section 120 performs the following steps S120-11 to S120-14 shown in FIG.

[0045] The left channel subtraction gain estimation unit 120 first estimates the left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x M From (T), the normalized inner product value r L (step S120-11). Furthermore, the left channel subtraction gain estimation unit 120 obtains the left channel difference signal y L (1), y L (2), ..., y L The number of bits b used to encode (T) L Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the left channel correction coefficient c is calculated using the following equation (1-7). L (step S120-12).

number

[0046] In addition, in the stereo encoding unit 170, the left channel difference signal y L (1), y L (2), ..., y L The number of bits b used to encode (T) L If is not explicitly determined, the number of bits b of the stereo code CS output by the stereo coding unit 170 is s (i.e., b s / 2) to number of bits b L The left channel correction coefficient c L is not the value obtained by equation (1-7) itself, but is a value greater than 0 and less than 1, and the left channel difference signal y L (1), y L (2), ..., y L The number of bits b used to encode (T) L and the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M is the same, it is 0.5, and the number of bits b L is the number of bits b M The more the number of bits, the closer it is to 0 than 0.5, and the more bits L is the number of bits b M The smaller the value, the closer it is to 0.5 or to 1. The same applies to the examples described below.

[0047] [Right channel subtraction gain estimation unit 140] The right channel subtraction gain estimation unit 140 estimates the right channel subtraction gain candidate β cand (b) and the code Cβ corresponding to the candidate cand A plurality of pairs (B pairs, b=1, . . . , B) of pairs with (b) are stored in advance. Right channel subtraction gain estimation section 140 performs the following steps S140-11 to S140-14 shown in FIG.

[0048] The right channel subtraction gain estimation unit 140 first estimates the input sound signal x R (1), x R (2), ..., x R (T) and downmix signal x M (1), x M (2), ..., x M From (T), the normalized inner product value r R (step S140-11). Also, the right channel subtraction gain estimation unit 140 obtains the right channel difference signal y R (1), y R (2), ..., y R The number of bits b used to encode (T) R Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the right channel correction coefficient c is calculated using the following equation (1-7-2). R (step S140-12).

number

[0049] In addition, in the stereo encoding unit 170, the right channel difference signal y R (1), y R (2), ..., y R The number of bits b used to encode (T) R If is not explicitly determined, the number of bits b of the stereo code CS output by the stereo coding unit 170 is s (i.e., b s / 2) to number of bits b R The right channel correction coefficient c R is not the value obtained by equation (1-7-2) itself, but is a value greater than 0 and less than 1, and the right channel difference signal y R (1), y R (2), ..., y R The number of bits b used to encode (T) R and the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M is the same, it is 0.5, and the number of bits b R is the number of bits b M The more the number of bits, the closer it is to 0 than 0.5, and the more bits R is the number of bits b M The smaller the value, the closer it is to 0.5 or to 1. The same applies to the examples described below.

[0050] [[[Left channel subtraction gain decoding unit 230]]] The left channel subtraction gain decoding unit 230 receives the left channel subtraction gain candidates α , which are the same as those stored in the left channel subtraction gain estimation unit 120 of the corresponding encoding device 100. cand (a) and the code Cα corresponding to the candidate cand A plurality of sets (A sets, a=1, . . . , A) of sets with (a) are stored in advance. The left channel subtraction gain decoding unit 230 converts the stored code Cα cand (1), ..., Cα cand A candidate for the left channel subtraction gain corresponding to the input left channel subtraction gain code Cα among (A) is obtained as the left channel subtraction gain α (step S230-11).

[0051] Right Channel Subtraction Gain Decoder 250 The right channel subtraction gain decoding unit 250 receives the right channel subtraction gain candidates β cand (b) and the code Cβ corresponding to the candidate cand A plurality of pairs (B pairs, b=1, . . . , B) of pairs of (b) are stored in advance. The right channel subtraction gain decoding unit 250 uses the stored code Cβ cand (1), ..., Cβ cand A candidate for the right channel subtraction gain corresponding to the input right channel subtraction gain code Cβ among (B) is obtained as the right channel subtraction gain β (step S250-11).

[0052] In addition, the same subtraction gain candidates and codes may be used for the left and right channels. The above-mentioned A and B may be set to the same values, and the left channel subtraction gain candidate α stored in the left channel subtraction gain estimation unit 120 and the left channel subtraction gain decoding unit 230 may be set to the same values. cand (a) and the code Cα corresponding to the candidate cand (a) and the right channel subtraction gain candidates β cand (b) and the code Cβ corresponding to the candidatecand The pair with (b) may be the same.

[0053] [Modification of Example 1] The number of bits b used to code the left channel difference signal in the coding device 100 is L is the number of bits used in decoding the left channel differential signal in the decoding device 200, and b is the number of bits used in encoding the downmix signal in the encoding device 100. M Since the value of is the number of bits used in decoding the downmix signal in the decoding device 200, the correction coefficient c L The same value can be calculated by both the encoding device 100 and the decoding device 200. Therefore, the normalized inner product value r L is the target of encoding and decoding, and the quantized value ^r L Correction factor c L The left channel subtraction gain α may be obtained by multiplying the left channel subtraction gain α by α. The same is true for the right channel. This embodiment will be described as a modification of Example 1.

[0054] [[[Left channel subtraction gain estimator 120]]] The left channel subtraction gain estimation unit 120 includes a candidate r Lcand (a) and the code Cα corresponding to the candidate cand A plurality of sets (A sets, a=1, ..., A) of sets with (a) are stored in advance. As shown in Fig. 6, left channel subtraction gain estimation section 120 performs steps S120-11 and S120-12 described in Example 1, and steps S120-15 and S120-16 described below.

[0055] First, the left channel subtraction gain estimation unit 120 performs the step S120-11 of the left channel subtraction gain estimation unit 120 in the first example to estimate the left channel input sound signal x L (1), x L (2), ..., x L (T) and downmix signal x M (1), x M (2), ..., x MFrom (T), the normalized inner product value r L (Step S120-11). The left channel subtraction gain estimation unit 120 then obtains the stored candidates r Lcand (1), ..., r Lcand The normalized inner product value r obtained in step S120-11 of (A) L The closest candidate (normalized inner product value r L quantized value of)^r L and store the code Cα cand (1), ..., Cα cand The closest candidate ^r in (A) L As the left channel subtraction gain code Cα (step S120-15), the left channel subtraction gain estimation unit 120 obtains a code corresponding to the left channel difference signal y L (1), y L (2), ..., y L The number of bits b used to encode (T) L Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the left channel correction coefficient c is calculated by the formula (1-7). L (step S120-12). Next, the left channel subtraction gain estimation unit 120 obtains the quantized value ^r of the normalized inner product value obtained in step S120-15. L and the left channel correction coefficient c obtained in step S120-12 L The value obtained by multiplying this by α is obtained as the left channel subtraction gain α (step S120-16).

[0056] [Right channel subtraction gain estimation unit 140] The right channel subtraction gain estimation unit 140 includes a candidate r Rcand(b) and the code Cβ corresponding to the candidate cand A plurality of pairs (B pairs, b=1, ..., B) of pairs with (b) are stored in advance. Right channel subtraction gain estimation section 140 performs steps S140-11 and S140-12 described in Example 1, as well as steps S140-15 and S140-16 described below, as shown in FIG.

[0057] First, the right channel subtraction gain estimation unit 140 performs the step S140-11 of the right channel subtraction gain estimation unit 140 in the first example to estimate the input sound signal x R (1), x R (2), ..., x R (T) and downmix signal x M (1), x M (2), ..., x M From (T), the normalized inner product value r R (Step S140-11). Next, the right channel subtraction gain estimation unit 140 obtains the stored candidates r Rcand (1), ..., r Rcand The normalized inner product value r obtained in step S140-11 of (B) R The closest candidate (normalized inner product value r R quantized value of)^r R and the stored code Cβ cand (1), ..., Cβ cand (B) The closest candidate ^r R As the right channel subtraction gain code Cβ (step S140-15), the right channel subtraction gain estimation unit 140 obtains a code corresponding to the right channel difference signal y R (1), y R (2), ..., y R The number of bits b used to encode (T) R Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M(2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the right channel correction coefficient c is calculated by the formula (1-7-2). R (step S140-12). Next, the right channel subtraction gain estimation unit 140 obtains the quantized value ^r of the normalized inner product value obtained in step S140-15. R and the right channel correction coefficient c obtained in step S140-12 R The value obtained by multiplying this by β is obtained as the right channel subtraction gain β (step S140-16).

[0058] [[[Left channel subtraction gain decoding unit 230]]] The left channel subtraction gain decoding unit 230 receives the same candidates r of normalized inner product values ​​for the left channel as those stored in the left channel subtraction gain estimation unit 120 of the corresponding encoding device 100. Lcand (a) and the code Cα corresponding to the candidate cand A plurality of pairs (A pairs, a=1, . . . , A) of pairs with (a) are stored in advance. Left channel subtraction gain decoding section 230 performs the following steps S230-12 to S230-14 shown in FIG.

[0059] The left channel subtraction gain decoder 230 extracts the stored code Cα cand (1), ..., Cα cand The candidate for the normalized inner product value of the left channel corresponding to the input left channel subtraction gain code Cα in (A) is decoded value ^r L (step S230-12). Furthermore, the left channel subtraction gain decoding unit 230 obtains the left channel decoded difference signal ^y L (1), ^y L (2), ..., ^y L The number of bits b used to decode (T) L Then, the monaural decoding unit 210 decodes the monaural sound signal ^x M (1), ^x M (2), ..., ^x M The number of bits b used to decode (T) MUsing the number of samples per frame T, the left channel correction coefficient c is calculated by the formula (1-7). L (step S230-13). Next, left channel subtraction gain decoding section 230 obtains the decoded value ^r of the normalized inner product value obtained in step S230-12. L and the left channel correction coefficient c obtained in step S230-13 L The value obtained by multiplying this by α is obtained as the left channel subtraction gain α (step S230-14).

[0060] When the stereo code CS is a combination of the left channel differential code CL and the right channel differential code CR, the left channel decoded differential signal ^y L (1), ^y L (2), ..., ^y L The number of bits b used to decode (T) L is the number of bits of the left channel differential code CL. L (1), ^y L (2), ..., ^y L The number of bits b used to decode (T) L If is not explicitly determined, the number of bits b of the stereo code CS input to the stereo decoding unit 220 is s (i.e., b s / 2) to number of bits b L In the monaural decoding unit 210, the monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M The number of bits b used to decode (T) M is the number of bits of the monaural code CM. The left channel correction coefficient c L is not the value obtained by equation (1-7) itself, but is a value greater than 0 and less than 1, and the left channel decoded difference signal ^y L (1), ^y L (2), ..., ^y L The number of bits b used to decode (T) L and the monaural decoded sound signal ^x M (1), ^x M(2), ..., ^x M The number of bits b used to decode (T) M is the same, it is 0.5, and the number of bits b L is the number of bits b M The more the number of bits, the closer it is to 0 than 0.5, and the more bits L is the number of bits b M The smaller the value, the closer to 1 than 0.5 may be used.

[0061] Right Channel Subtraction Gain Decoder 250 The right channel subtraction gain decoding unit 250 receives the same right channel normalized inner product value candidates r Rcand (b) and the code Cβ corresponding to the candidate cand A plurality of pairs (B pairs, b=1, . . . , B) of pairs with (b) are stored in advance. Right channel subtraction gain decoding section 250 performs the following steps S250-12 to S250-14 shown in FIG.

[0062] The right channel subtraction gain decoder 250 extracts the stored code Cβ cand (1), ..., Cβ cand The candidate of the normalized inner product value of the right channel corresponding to the input right channel subtraction gain code Cβ in (B) is decoded value ^r R (step S250-12). Also, in stereo decoding unit 220, right channel subtraction gain decoding unit 250 obtains right channel decoded difference signal ^y R (1), ^y R (2), ..., ^y R The number of bits b used to decode (T) R Then, the monaural decoding unit 210 decodes the monaural sound signal ^x M (1), ^x M (2), ..., ^x M The number of bits b used to decode (T) M Using the number of samples per frame T, the right channel correction coefficient c is calculated by the formula (1-7-2). R(step S250-13). Right channel subtraction gain decoding unit 250 then obtains the decoded value ^r of the normalized inner product value obtained in step S250-12. R and the right channel correction coefficient c obtained in step S250-13 R The value obtained by multiplying this by β is obtained as the right channel subtraction gain β (step S250-14).

[0063] When the stereo code CS is a combination of the left channel differential code CL and the right channel differential code CR, the stereo decoding unit 220 decodes the right channel decoded differential signal ^y R (1), ^y R (2), ..., ^y R The number of bits b used to decode (T) R is the number of bits of the right channel differential code CR. R (1), ^y R (2), ..., ^y R The number of bits b used to decode (T) R If is not explicitly determined, the number of bits b of the stereo code CS input to the stereo decoding unit 220 is s (i.e., b s / 2) to number of bits b R In the monaural decoding unit 210, the monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M The number of bits b used to decode (T) M is the number of bits of the monaural code CM. The right channel correction coefficient c R is not the value obtained by equation (1-7-2) itself, but is a value greater than 0 and less than 1, and the right channel decoded differential signal ^y R (1), ^y R (2), ..., ^y R The number of bits b used to decode (T) R and the monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M The number of bits b used to decode (T) Mis the same, it is 0.5, and the number of bits b R is the number of bits b M The more the number of bits, the closer it is to 0 than 0.5, and the more bits R is the number of bits b M The smaller the value, the closer to 1 than 0.5 may be used.

[0064] In addition, the same normalized inner product value candidates and codes may be used for the left and right channels. The above-mentioned A and B may be set to the same values, and the normalized inner product value candidates r for the left channel stored in the left channel subtraction gain estimation unit 120 and the left channel subtraction gain decoding unit 230 may be set to the same values. Lcand (a) and the code Cα corresponding to the candidate cand (a) and the candidate r Rcand (b) and the code Cβ corresponding to the candidate cand The pair with (b) may be the same.

[0065] Note that the code Cα is called the left channel subtraction gain code because it is substantially a code corresponding to the left channel subtraction gain α and for the purpose of consistency of wording in the explanation of the encoding device 100 and the decoding device 200, but since it represents a normalized inner product value, it may also be called the left channel inner product code, etc. The same is true for the code Cβ, which may also be called the right channel inner product code, etc.

[0066] [Example 2] An example in which a value taking into consideration the input value of a past frame as the normalized inner product value will be described as Example 2. Example 2 does not strictly guarantee optimality within a frame, i.e., minimization of the energy of the quantization error in the left channel decoded sound signal and minimization of the energy of the quantization error in the right channel decoded sound signal, but reduces sudden fluctuations between frames of the left channel subtraction gain α and the right channel subtraction gain β, thereby reducing noise generated in the decoded sound signal due to the fluctuations. That is, Example 2 takes into consideration the auditory quality of the decoded sound signal in addition to reducing the energy of the quantization error in the decoded sound signal.

[0067] In Example 2, the encoding side, i.e., left channel subtraction gain estimator 120 and right channel subtraction gain estimator 140, is different from Example 1, but the decoding side, i.e., left channel subtraction gain decoder 230 and right channel subtraction gain decoder 250, is the same as Example 1. Below, the differences between Example 2 and Example 1 will be mainly described.

[0068] [[[Left channel subtraction gain estimator 120]]] As shown in FIG. 8, left channel subtraction gain estimation section 120 performs the following steps S120-111 to S120-113 and steps S120-12 to S120-14 described in Example 1.

[0069] The left channel subtraction gain estimation unit 120 first estimates the left channel input sound signal x L (1), x L (2), ..., x L (T) and the input downmix signal x M (1), x M (2), ..., x M (T) and the dot product value E used in the previous frame L (-1) and the inner product value E to be used in the current frame is calculated using the following equation (1-8). L (0) is obtained (step S120-111).

number

[0070] The left channel subtraction gain estimation unit 120 also estimates the input downmix signal x M (1), x M (2), ..., x M (T) and the energy of the downmix signal used in the previous frame E M (-1) and the energy E of the downmix signal used in the current frame is calculated by the following equation (1-9). M (0) is obtained (step S120-112).

number

[0071] Next, the left channel subtraction gain estimation unit 120 calculates the inner product value E L (0) and the energy E of the downmix signal used in the current frame obtained in step S120-112. M (0) is used to normalize the dot product value r L is obtained using the following equation (1-10) (step S120-113).

number

[0072] Left channel subtraction gain estimation unit 120 also performs step S120-12, and then subtracts the normalized inner product value r obtained in step S120-11. L Instead of the normalized inner product value r obtained in step S120-113 described above, L Step S120-13 is performed using the above, and then step S120-14 is performed.

[0073] In addition, the above ε L and ε M The closer to 1, the normalized inner product value r L is likely to include the influence of the left channel input sound signal and downmix signal of the past frame, and the normalized dot product value r L or the normalized dot product value r L The resulting left channel subtraction gain α has smaller frame-to-frame variation.

[0074] [Right channel subtraction gain estimation unit 140] As shown in FIG. 8, right channel subtraction gain estimation section 140 performs the following steps S140-111 to S140-113 and steps S140-12 to S140-14 described in example 1.

[0075] The right channel subtraction gain estimation unit 140 first estimates the input sound signal x R (1), x R (2), ..., x R (T) and the input downmix signal x M (1), x M (2), ..., x M (T) and the dot product value E used in the previous frame R (-1) and the inner product value E to be used in the current frame is calculated using the following formula (1-8-2). R (0) is obtained (step S140-111).

number

[0076] The right channel subtraction gain estimation unit 140 also estimates the input downmix signal x M (1), x M (2), ..., x M (T) and the energy of the downmix signal used in the previous frame E M (-1), and the energy E of the downmix signal used in the current frame is calculated by using the formula (1-9). M (0) (step S140-112). The right channel subtraction gain estimation unit 140 obtains the energy E M (0) is defined as the energy E M The left channel subtraction gain estimation unit 120 also calculates the energy E M Therefore, only one of step S120-112 performed by left channel subtraction gain estimation section 120 and step S140-112 performed by right channel subtraction gain estimation section 140 may be performed.

[0077] Next, right channel subtraction gain estimation unit 140 calculates the inner product value E R (0), and the energy E of the downmix signal used in the current frame obtained in step S140-112. M (0) is used to normalize the dot product value r R is obtained using the following equation (1-10-2) (step S140-113).

number

[0078] Right channel subtraction gain estimation unit 140 also performs step S140-12, and then subtracts the normalized inner product value r obtained in step S140-11. R Instead of the normalized inner product value r obtained in step S140-113 described above, R Step S140-13 is performed using the above, and then step S140-14 is performed.

[0079] In addition, the above ε R and ε M The closer to 1, the normalized inner product value r R is likely to include the influence of the right channel input sound signal and downmix signal of the past frame, and the normalized dot product value r R or the normalized dot product value r R The resulting right channel subtraction gain β has smaller frame-to-frame variation.

[0080] [Modification of Example 2] The same modification as the modification of Example 1 can be applied to Example 2. This embodiment will be described as a modification of Example 2. The modification of Example 2 differs from the modification of Example 1 on the encoding side, i.e., left channel subtraction gain estimation unit 120 and right channel subtraction gain estimation unit 140, but the decoding side, i.e., left channel subtraction gain decoding unit 230 and right channel subtraction gain decoding unit 250, are the same as the modification of Example 1. The difference between the modification of Example 2 and the modification of Example 1 is the same as in Example 2, so the modification of Example 2 will be described below with reference to the modification of Example 1 and Example 2 as appropriate.

[0081] [[[Left channel subtraction gain estimator 120]]] The left channel subtraction gain estimation unit 120 includes the normalized inner product value candidates r of the left channel, similar to the left channel subtraction gain estimation unit 120 of the modified example of the first embodiment. Lcand (a) and the code Cα corresponding to the candidate candA plurality of sets (A sets, a=1, ..., A) of sets with (a) are stored in advance. As shown in Fig. 9, the left channel subtraction gain estimation unit 120 performs steps S120-111 to S120-113, which are the same as in Example 2, and steps S120-12, S120-15, and S120-16, which are the same as in the modified example of Example 1. Specifically, the steps are as follows.

[0082] The left channel subtraction gain estimation unit 120 first estimates the left channel input sound signal x L (1), x L (2), ..., x L (T) and the input downmix signal x M (1), x M (2), ..., x M (T) and the dot product value E used in the previous frame L (-1), and the inner product value E to be used in the current frame is calculated by equation (1-8). L (0) (step S120-111). The left channel subtraction gain estimation unit 120 also obtains the input downmix signal x M (1), x M (2), ..., x M (T) and the energy of the downmix signal used in the previous frame E M (-1), and the energy E of the downmix signal used in the current frame is calculated by using the formula (1-9). M Next, the left channel subtraction gain estimation unit 120 obtains the inner product value E L (0) and the energy E of the downmix signal used in the current frame obtained in step S120-112. M (0), the normalized inner product value r L (step S120-113). The left channel subtraction gain estimation unit 120 then obtains the stored candidates r Lcand (1), ..., r Lcand The normalized inner product value r obtained in step S120-113 of (A) LThe closest candidate (normalized inner product value r L quantized value of)^r L and store the code Cα cand (1), ..., Cα cand The closest candidate ^r in (A) L The left channel subtraction gain estimation unit 120 obtains a code corresponding to the left channel difference signal y L (1), y L (2), ..., y L The number of bits b used to encode (T) L Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the left channel correction coefficient c L (step S120-12). Next, the left channel subtraction gain estimation unit 120 obtains the quantized value ^r of the normalized inner product value obtained in step S120-15. L and the left channel correction coefficient c obtained in step S120-12 L The value obtained by multiplying this by α is obtained as the left channel subtraction gain α (step S120-16).

[0083] [Right channel subtraction gain estimation unit 140] The right channel subtraction gain estimation unit 140 includes the candidates r Rcand (b) and the code Cβ corresponding to the candidate cand A plurality of sets (B sets, b=1, ..., B) of sets with (b) are stored in advance. As shown in Fig. 9, right channel subtraction gain estimation unit 140 performs steps S140-111 to S140-113, which are the same as in Example 2, and steps S140-12, S140-15, and S140-16, which are the same as in the modified example of Example 1. Specifically, the steps are as follows.

[0084] The right channel subtraction gain estimation unit 140 first estimates the input sound signal x R (1), x R (2), ..., x R (T) and the input downmix signal x M (1), x M (2), ..., x M (T) and the dot product value E used in the previous frame R (-1), and the inner product value E to be used in the current frame is calculated by equation (1-8-2). R (0) (step S140-111). The right channel subtraction gain estimation unit 140 also obtains the input downmix signal x M (1), x M (2), ..., x M (T) and the energy of the downmix signal used in the previous frame E M (-1), and the energy E of the downmix signal used in the current frame is calculated by using the formula (1-9). M Next, the right channel subtraction gain estimation unit 140 obtains the inner product value E R (0), and the energy E of the downmix signal used in the current frame obtained in step S140-112. M (0), the normalized inner product value r R (step S140-113). The right channel subtraction gain estimation unit 140 then obtains the stored candidates r Rcand (1), ..., r Rcand The normalized inner product value r obtained in step S140-113 of (B) R The closest candidate (normalized inner product value r R quantized value of)^r R and the stored code Cβ cand (1), ..., Cβ cand (B) The closest candidate ^r RThe right channel subtraction gain estimation unit 140 obtains a code corresponding to the right channel difference signal y R (1), y R (2), ..., y R The number of bits b used to encode (T) R Then, the monaural encoding unit 160 encodes the downmix signal x M (1), x M (2), ..., x M The number of bits b used to encode (T) M Using the number of samples per frame T, the right channel correction coefficient c is calculated by the formula (1-7-2). R (step S140-12). Next, the right channel subtraction gain estimation unit 140 obtains the quantized value ^r of the normalized inner product value obtained in step S140-15. R and the right channel correction coefficient c obtained in step S140-12 R The value obtained by multiplying this by β is obtained as the right channel subtraction gain β (step S140-16).

[0085] [Example 3] For example, when a sound such as speech or music contained in a left channel input sound signal is different from a sound such as speech or music contained in a right channel input sound signal, the downmix signal may contain both a component of the left channel input sound signal and a component of the right channel input sound signal, so that the larger the value used as the left channel subtraction gain α, the more the left channel decoded sound signal sounds as if it contains a sound originating from the right channel input sound signal that should not be heard in the first place, and the larger the value used as the right channel subtraction gain β, the more the right channel decoded sound signal sounds as if it contains a sound originating from the left channel input sound signal that should not be heard in the first place. Therefore, although minimization of the energy of the quantization error of the decoded sound signal is not strictly guaranteed, the left channel subtraction gain α and the right channel subtraction gain β may be set to values ​​smaller than those obtained by Example 1 in consideration of the hearing quality. Similarly, the left channel subtraction gain α and the right channel subtraction gain β may be set to values ​​smaller than those obtained by Example 2.

[0086] Specifically, for the left channel, in Examples 1 and 2, the normalized dot product value r L and the left channel correction factor c L Multiplication value of c L ×r L In Example 3, the quantized value of the normalized inner product value r L and the left channel correction factor c L and a predetermined value λ greater than 0 and less than 1. L Multiplication value λ L ×c L ×r L The quantized value of is the left channel subtraction gain α. Therefore, as in examples 1 and 2, the multiplication value c L ×r L is encoded by the left channel subtraction gain estimation unit 120 and decoded by the left channel subtraction gain decoding unit 230, and the left channel subtraction gain code Cα is multiplied by the value c L ×r L The left channel subtraction gain estimator 120 and the left channel subtraction gain decoder 230 calculate the multiplication value c L ×r L quantization value of and λ L Alternatively, the normalized dot product value r L and the left channel correction factor c L and a predetermined value λ L Multiplication value λ L ×c L ×r L is encoded by the left channel subtraction gain estimation unit 120 and decoded by the left channel subtraction gain decoding unit 230, and the left channel subtraction gain code Cα is multiplied by the multiplication value λ L ×c L ×r L It may represent a quantized value of

[0087] Similarly, for the right channel, in Examples 1 and 2, the normalized dot product value r R and the right channel correction factor c R Multiplication value of c R ×r RIn Example 3, the quantized value of the right channel subtraction gain β is used as the normalized inner product value r R and the right channel correction factor c R and a predetermined value λ greater than 0 and less than 1. R Multiplication value λ R ×c R ×r R The quantized value of is the right channel subtraction gain β. Therefore, as in examples 1 and 2, the multiplication value c R ×r R is encoded by the right channel subtraction gain estimation unit 140 and decoded by the right channel subtraction gain decoding unit 250, and the right channel subtraction gain code Cβ is multiplied by the value c R ×r R The right channel subtraction gain estimator 140 and the right channel subtraction gain decoder 250 calculate the multiplication value c R ×r R quantization value of and λ R Alternatively, the normalized dot product value r may be multiplied by r to obtain the right channel subtraction gain β. R and the left channel correction factor c R and a predetermined value λ R Multiplication value λ R ×c R ×r R is encoded by the right channel subtraction gain estimation unit 140 and decoded by the right channel subtraction gain decoding unit 250, and the right channel subtraction gain code Cβ is multiplied by the multiplication value λ R ×c R ×r R It should be noted that λ may be expressed as a quantized value. R is λ L should be set to the same value as

[0088] [Modification of Example 3] As mentioned above, the correction factor c L can be calculated as the same value in both the encoding device 100 and the decoding device 200. Therefore, as in the modified example 1 and the modified example 2, the normalized inner product value r L is encoded by the left channel subtraction gain estimation unit 120 and decoded by the left channel subtraction gain decoding unit 230, the inner product value r LThe left channel subtraction gain estimator 120 and the left channel subtraction gain decoder 230 calculate the normalized inner product value r L quantized value of and the left channel correction coefficient c L and a predetermined value λ greater than 0 and less than 1. L Alternatively, the normalized dot product value r L and a predetermined value λ greater than 0 and less than 1. L Multiplication value λ L ×r L is encoded by the left channel subtraction gain estimation unit 120 and decoded by the left channel subtraction gain decoding unit 230, and the left channel subtraction gain code Cα is multiplied by the multiplication value λ L ×r L , so that the left channel subtraction gain estimator 120 and the left channel subtraction gain decoder 230 calculate the multiplication value λ L ×r L quantized value of and the left channel correction coefficient c L to obtain the left channel subtraction gain α.

[0089] The same is true for the right channel, with the correction factor c R can be calculated as the same value in both the encoding device 100 and the decoding device 200. Therefore, as in the modified example 1 and the modified example 2, the normalized inner product value r R is encoded by the right channel subtraction gain estimation unit 140 and decoded by the right channel subtraction gain decoding unit 250, and the right channel subtraction gain code Cβ is normalized to obtain the inner product value r R The right channel subtraction gain estimator 140 and the right channel subtraction gain decoder 250 calculate the normalized inner product value r R and the right channel correction coefficient c R and a predetermined value λ greater than 0 and less than 1. R Alternatively, the normalized dot product value r R and a predetermined value λ greater than 0 and less than 1. R Multiplication value λ R ×r Ris encoded by the right channel subtraction gain estimation unit 140 and decoded by the right channel subtraction gain decoding unit 250, and the right channel subtraction gain code Cβ is multiplied by the multiplication value λ R ×r R The right channel subtraction gain estimator 140 and the right channel subtraction gain decoder 250 calculate the multiplication value λ R ×r R and the right channel correction coefficient c R to obtain the right channel subtraction gain β.

[0090] [Example 4] The hearing quality problem described at the beginning of Example 3 occurs when the correlation between the left channel input sound signal and the right channel input sound signal is small, and this problem does not occur so much when the correlation between the left channel input sound signal and the right channel input sound signal is large. Therefore, in Example 4, instead of the predetermined value in Example 3, a left-right correlation coefficient γ, which is the correlation coefficient between the left channel input sound signal and the right channel input sound signal, is used, so that the greater the correlation between the left channel input sound signal and the right channel input sound signal, the higher the priority is given to reducing the energy of the quantization error in the decoded sound signal, and the smaller the correlation between the left channel input sound signal and the right channel input sound signal, the higher the priority is given to suppressing deterioration of hearing quality.

[0091] In Example 4, the encoding side is different from Examples 1 and 2, but the decoding side, i.e., left channel subtraction gain decoding section 230 and right channel subtraction gain decoding section 250, are the same as in Examples 1 and 2. Below, the differences between Example 4 and Examples 1 and 2 will be described.

[0092] [[[Left-right relationship information estimation unit 180]]] The encoding device 100 of example 4 also includes a left-right relationship information estimation unit 180, as indicated by a dashed line in Fig. 1. The left-right relationship information estimation unit 180 receives a left channel input sound signal and a right channel input sound signal input to the encoding device 100. The left-right relationship information estimation unit 180 obtains and outputs a left-right correlation coefficient γ from the input left channel input sound signal and right channel input sound signal (step S180).

[0093] The left-right correlation coefficient γ is a correlation coefficient between the left channel input sound signal and the right channel input sound signal, and is expressed as the sample sequence x L (1), x L (2), ..., x L (T) and the sample sequence of the right channel input sound signal x R (1), x R (2), ..., x R (T), or a correlation coefficient taking into account a time difference, for example, a correlation coefficient γ between a sample sequence of an input sound signal on the left channel and a sample sequence of an input sound signal on the right channel that is shifted τ samples behind the sample sequence. τ may be also possible.

[0094] This τ is information equivalent to the difference (so-called arrival time difference) between the arrival time from a sound source that mainly emits sound in a space to the left channel microphone and the arrival time from the sound source to the right channel microphone, assuming that the sound signal obtained by AD converting the sound picked up by a left channel microphone placed in the space is the left channel input sound signal, and the sound signal obtained by AD converting the sound picked up by a right channel microphone placed in the space is the right channel input sound signal, and is hereinafter referred to as the left-right time difference. The left-right time difference τ may be calculated by any known method, such as the method described in the left-right relationship information estimation unit 181 of the second reference embodiment. That is, the above-mentioned correlation coefficient γ τ is information equivalent to the correlation coefficient between a sound signal that has arrived at the left channel microphone from a sound source and been picked up, and a sound signal that has arrived at the right channel microphone from the same sound source and been picked up.

[0095] [[[Left channel subtraction gain estimator 120]]] Instead of step S120-13, the left channel subtraction gain estimation unit 120 uses the normalized inner product value r obtained in step S120-11 or step S120-113. L and the left channel correction coefficient c obtained in step S120-12. Land the left-right correlation coefficient γ obtained in step S180 to obtain a value (step S120-13"). Next, instead of step S120-14, the left channel subtraction gain estimation unit 120 multiplies the stored left channel subtraction gain candidates α cand (1), ..., α cand The multiplication value γ×c obtained in step S120-13″ of (A) L ×r L The closest candidate (multiplication value γ × c L ×r L The quantized value of (α) is obtained as the left channel subtraction gain α, and the stored code Cα cand (1), ..., Cα cand The code corresponding to the left channel subtraction gain α in (A) is obtained as the left channel subtraction gain code Cα (step S120-14″).

[0096] [Right channel subtraction gain estimation unit 140] Instead of step S140-13, the right channel subtraction gain estimation unit 140 estimates the normalized inner product value r obtained in step S140-11 or step S140-113. R and the right channel correction coefficient c obtained in step S140-12. R and the left-right correlation coefficient γ obtained in step S180 to obtain a value (step S140-13"). Next, instead of step S140-14, the right channel subtraction gain estimation unit 140 multiplies the stored right channel subtraction gain candidates β cand (1), ..., β cand The multiplication value γ×c obtained in step S140-13″ of (B) R ×r R The closest candidate (multiplication value γ × c R ×r R The quantized value of (β) is obtained as the right channel subtraction gain β, and the stored code Cβ cand (1), ..., Cβ cand The code corresponding to the right channel subtraction gain β in (B) is obtained as the right channel subtraction gain code Cβ (step S140-14″).

[0097] [Modification of Example 4] As mentioned above, the correction factor c L The same value can be calculated by both the encoding device 100 and the decoding device 200. Therefore, the normalized inner product value r L and the multiplication value of the left-right correlation coefficient γ×r L is encoded by the left channel subtraction gain estimation unit 120 and decoded by the left channel subtraction gain decoding unit 230, and the left channel subtraction gain code Cα is multiplied by the multiplication value γ×r L The left channel subtraction gain estimator 120 and the left channel subtraction gain decoder 230 calculate the multiplication value γ×r L quantized value of and the left channel correction coefficient c L to obtain the left channel subtraction gain α.

[0098] The same is true for the right channel, with the correction factor c R The same value can be calculated by both the encoding device 100 and the decoding device 200. Therefore, the normalized inner product value r R and the multiplication value of the left-right correlation coefficient γ×r R is encoded by the right channel subtraction gain estimation unit 140 and decoded by the right channel subtraction gain decoding unit 250, and the right channel subtraction gain code Cβ is multiplied by the multiplication value γ×r R The right channel subtraction gain estimator 140 and the right channel subtraction gain decoder 250 calculate the multiplication value γ×r R and the right channel correction coefficient c R to obtain the right channel subtraction gain β.

[0099] <Second reference form> An encoding device and a decoding device according to the second embodiment will be described.

[0100] ≪Encoding device 101≫ 10, the encoding device 101 of the second reference embodiment includes a downmixing unit 110, a left channel subtraction gain estimating unit 120, a left channel signal subtracting unit 130, a right channel subtraction gain estimating unit 140, a right channel signal subtracting unit 150, a monaural encoding unit 160, a stereo encoding unit 170, a left-right relationship information estimating unit 181, and a time shifting unit 191. The encoding device 101 of the second reference embodiment differs from the encoding device 100 of the first reference embodiment in that it includes the left-right relationship information estimating unit 181 and the time shifting unit 191, that the left channel subtraction gain estimating unit 120, the left channel signal subtracting unit 130, the right channel subtraction gain estimating unit 140, and the right channel signal subtracting unit 150 use the signal output by the time shifting unit 191 instead of the signal output by the downmixing unit 110, and that it also outputs a left-right time difference code Cτ described later in addition to the above-mentioned codes. Other configurations and operations of the encoding device 101 of the second reference embodiment are the same as those of the encoding device 100 of the first reference embodiment. The encoding device 101 of the second reference embodiment performs the processes of steps S110 to S191 illustrated in Fig. 11 for each frame. Below, the differences between the encoding device 101 of the second reference embodiment and the encoding device 100 of the first reference embodiment will be described.

[0101] [Left-right relationship information estimation unit 181] The left-channel input sound signal input to the encoding device 101 and the right-channel input sound signal input to the encoding device 101 are input to the left-right relationship information estimation unit 181. From the input left-channel input sound signal and the input right-channel input sound signal, the left-right relationship information estimation unit 181 obtains and outputs a left-right time difference τ and a left-right time difference code Cτ that is a code representing the left-right time difference τ (step S181).

[0102] The left-right time difference τ is information corresponding to the difference between the arrival time from a sound source that mainly emits sound in a space to a left channel microphone and the arrival time from the sound source to a right channel microphone (so-called arrival time difference) when it is assumed that the sound signal obtained by AD converting the sound picked up by a left channel microphone arranged in a certain space is the left channel input sound signal, and the sound signal obtained by AD converting the sound picked up by a right channel microphone arranged in the space is the right channel input sound signal. Note that in order to include not only the arrival time difference but also information on which microphone the sound arrives at first in the left-right time difference τ, the left-right time difference τ can take both positive and negative values ​​based on one of the input sound signals. In other words, the left-right time difference τ is information indicating how far in advance the same sound signal is included in the left channel input sound signal or the right channel input sound signal. In the following, when the same sound signal is included in the input sound signal of the left channel before the input sound signal of the right channel, it is also said that the left channel is leading, and when the same sound signal is included in the input sound signal of the right channel before the input sound signal of the left channel, it is also said that the right channel is leading.

[0103] The left-right time difference τ may be calculated by any known method. For example, the left-right relationship information estimation unit 181 calculates a predetermined time difference τ max From τ min up to (e.g., τ max is a positive number, τ min is a negative number) cand For the left channel input sound signal sample sequence and the number of candidate samples τ cand A value (hereinafter referred to as a correlation value) γ cand Calculate the correlation value γ cand The number of candidate samples τ that maximizes candis obtained as the left-right time difference τ. That is, in this example, when the left channel is leading, the left-right time difference τ is a positive value, and when the right channel is leading, the left-right time difference τ is a negative value, and the absolute value of the left-right time difference τ is a value (the number of leading samples) that indicates how far the leading channel is ahead of the other channel. For example, the correlation value γ cand When calculating τ cand If is positive, the partial sample sequence x of the input sound signal on the right channel R (1+τ cand ), x R (2+τ cand ), ..., x R (T) and the number of candidate samples τ cand A partial sample sequence x of the input sound signal on the left channel that is shifted forward from the partial sample sequence by L (1), x L (2), ..., x L (T-τ cand ) and the absolute value of the correlation coefficient is the correlation value γ cand Calculate as τ cand If is negative, the partial sample sequence x of the left channel input sound signal is L (1-τ cand ), x L (2-τ cand ), ..., x L (T) and the number of candidate samples - τ cand A partial sample sequence x of the input sound signal on the right channel, which is shifted forward from the partial sample sequence by R (1), x R (2), ..., x R (T+τ cand ) and the absolute value of the correlation coefficient is the correlation value γ cand Of course, the correlation value γ cand In calculating the above, one or more samples of a past input sound signal consecutive to the sample sequence of the input sound signal of the current frame may also be used. In this case, the sample sequences of the input sound signals of past frames may be stored in a memory unit (not shown) in the left-right relationship information estimation unit 181 for a predetermined number of frames.

[0104] Alternatively, instead of the absolute value of the correlation coefficient, the correlation value γ cand In this example, the left-right relation information estimation unit 181 first calculates the left channel input sound signal x L (1), x L (2), ..., x L (T) and the right channel input sound signal x R (1), x R (2), ..., x R By Fourier transforming each of (T) as shown in the following equations (3-1) and (3-2), the frequency spectrum X at each frequency k from 0 to T-1 is obtained. L (k) and X R Obtain (k).

number

number

number

number

number

number

[0105] The left-right relationship information estimation unit 181 may encode the left-right time difference τ by a predetermined encoding method to obtain a left-right time difference code Cτ that can uniquely identify the left-right time difference τ. As the predetermined encoding method, a well-known encoding method such as scalar quantization may be used. Note that the number of each of the predetermined candidate samples is τ max From τ min τ max From τ min It may contain fractional or decimal values ​​between τ max From τ min In addition, it is not necessary to include any integer value between τ max =-τ min In addition, when a special input sound signal in which one of the channels always precedes the other is treated as a target, τ max Also τ min Let τ be a positive number. max Also τ min may also be a negative number.

[0106] In addition, when the encoding device 101 estimates the subtraction gain based on the principle of minimizing the quantization error of Example 4 or the modified example of Example 4 described in the first embodiment, the left-right relationship information estimation unit 181 further estimates a correlation value between a sample sequence of an input sound signal of the left channel and a sample sequence of an input sound signal of the right channel that is shifted backward from the sample sequence by the left-right time difference τ, that is, τ max From τ min The number of candidate samples up to τ cand The correlation value γ cand The maximum value among these is output as the left-right correlation coefficient γ (step S180).

[0107] [Time shift section 191] The time shift unit 191 receives the downmix signal x output from the downmix unit 110. M (1), x M (2), ..., x MThe time shift unit 191 receives the left-right time difference τ (T) and the left-right time difference τ output by the left-right relationship information estimation unit 181. If the left-right time difference τ is a positive value (i.e., if the left-right time difference τ indicates that the left channel is leading), the time shift unit 191 shifts the downmix signal x M (1), x M (2), ..., x M (T) is output as is to left channel subtraction gain estimation unit 120 and left channel signal subtraction unit 130 (i.e., it is determined to use it in left channel subtraction gain estimation unit 120 and left channel signal subtraction unit 130), and the downmix signal is delayed by |τ| samples (the number of samples corresponding to the absolute value of left-right time difference τ, the number of samples corresponding to the magnitude represented by left-right time difference τ) to obtain signal x M (1-|τ|), x M (2-|τ|), ..., x M The delayed downmix signal x M' (1), x M' (2), ..., x M' (T) is output to the right channel subtraction gain estimation unit 140 and the right channel signal subtraction unit 150 (i.e., it is determined to use it in the right channel subtraction gain estimation unit 140 and the right channel signal subtraction unit 150), and if the left-right time difference τ is a negative value (i.e., if the left-right time difference τ indicates that the right channel is ahead), a signal x M (1-|τ|), x M (2-|τ|), ..., x M The delayed downmix signal x M' (1), x M' (2), ..., x M' (T) is output to the left channel subtraction gain estimation unit 120 and the left channel signal subtraction unit 130 (i.e., it is determined to be used in the left channel subtraction gain estimation unit 120 and the left channel signal subtraction unit 130), and the downmix signal x M (1), x M (2), ..., x M(T) is output as is to the right channel subtraction gain estimation unit 140 and the right channel signal subtraction unit 150 (i.e., it is determined to use it in the right channel subtraction gain estimation unit 140 and the right channel signal subtraction unit 150), and if the left-right time difference τ is 0 (i.e., if the left-right time difference τ indicates that neither channel is leading), the downmix signal x M (1), x M (2), ..., x M (T) is output as it is to the left channel subtraction gain estimation unit 120, the left channel signal subtraction unit 130, the right channel subtraction gain estimation unit 140, and the right channel signal subtraction unit 150 (i.e., it is determined to use it in the left channel subtraction gain estimation unit 120, the left channel signal subtraction unit 130, the right channel subtraction gain estimation unit 140, and the right channel signal subtraction unit 150) (step S191). That is, for the channel having the shorter arrival time among the left channel and the right channel, the input downmix signal is output as it is to the subtraction gain estimation unit of the channel and the signal subtraction unit of the channel, and for the channel having the longer arrival time among the left channel and the right channel, the input downmix signal is delayed by the absolute value |τ| of the left and right time difference τ and is output to the subtraction gain estimation unit of the channel and the signal subtraction unit of the channel. In addition, since time shift unit 191 uses the downmix signal of a past frame to obtain a delayed downmix signal, a storage unit (not shown) in time shift unit 191 stores the downmix signal input in the past frame for a predetermined number of frames. In addition, when left channel subtraction gain estimation unit 120 and right channel subtraction gain estimation unit 140 obtain the left channel subtraction gain α and the right channel subtraction gain β by a well-known method such as that exemplified in Patent Document 1, rather than a method based on the principle of minimizing quantization error, a means for obtaining a local decoded signal corresponding to the monaural code CM is provided downstream of monaural coding unit 160 of coding device 101 or within monaural coding unit 160, and time shift unit 191 outputs the downmix signal x M (1), x M (2), ..., x M Instead of (T), a quantized downmix signal ^x, which is a locally decoded signal of mono coding, is used. M(1), ^x M (2), ..., ^x M (T) to perform the above-mentioned processing. In this case, the time shift unit 191 may M (1), x M (2), ..., x M Instead of (T), we use the quantized downmix signal ^x M (1), ^x M (2), ..., ^x M (T) and output the delayed downmix signal x M' (1), x M' (2), ..., x M' Instead of (T), we use the delayed quantized downmix signal ^x M' (1), ^x M' (2), ..., ^x M' Output (T).

[0108] [Left channel subtraction gain estimator 120, left channel signal subtractor 130, right channel subtraction gain estimator 140, right channel signal subtractor 150] The left channel subtraction gain estimating unit 120, the left channel signal subtracting unit 130, the right channel subtraction gain estimating unit 140, and the right channel signal subtracting unit 150 perform the same operations as those described in the first embodiment on the downmix signal x output by the downmixing unit 110. M (1), x M (2), ..., x M (T), the downmix signal x input from the time shift unit 191 is M (1), x M (2), ..., x M (T) or delayed downmix signal x M' (1), x M' (2), ..., x M' That is, the left channel subtraction gain estimation unit 120, the left channel signal subtraction unit 130, the right channel subtraction gain estimation unit 140, and the right channel signal subtraction unit 150 perform the downmix signal x determined by the time shift unit 191 using the left channel subtraction gain estimation unit 120, the left channel signal subtraction gain estimation unit 130, the right channel signal subtraction gain estimation unit 140, and the right channel signal subtraction unit 150 perform the downmix signal x determined by the time shift unit 191 using the left channel subtraction gain estimation unit 130, the right channel signal subtraction gain estimation unit 140, and the right channel signal subtraction unit 150 perform the downmix signal x M (1), x M(2), ..., x M (T) or delayed downmix signal x M' (1), x M' (2), ..., x M' (T), the same operation as that described in the first embodiment is performed. M (1), x M (2), ..., x M Instead of (T), we use the quantized downmix signal ^x M (1), ^x M (2), ..., ^x M (T) and output the delayed downmix signal x M' (1), x M' (2), ..., x M' Instead of (T), we use the delayed quantized downmix signal ^x M' (1), ^x M' (2), ..., ^x M' When the left channel subtraction gain estimation unit 120, the left channel signal subtraction unit 130, the right channel subtraction gain estimation unit 140, and the right channel signal subtraction unit 150 output the quantized downmix signal ^x M (1), ^x M (2), ..., ^x M (T) or delayed quantized downmix signal ^x M' (1), ^x M' (2), ..., ^x M' The above-mentioned processing is carried out using (T).

[0109] <Decryption device 201> As shown in Fig. 12, the decoding device 201 of the second reference embodiment includes a monaural decoding unit 210, a stereo decoding unit 220, a left channel subtraction gain decoding unit 230, a left channel signal adding unit 240, a right channel subtraction gain decoding unit 250, a right channel signal adding unit 260, a left / right time difference decoding unit 271, and a time shift unit 281. The decoding device 201 of the second reference embodiment differs from the decoding device 200 of the first reference embodiment in that a left / right time difference code Cτ described later is also input in addition to the above-mentioned codes, that the decoding device 201 of the second reference embodiment includes a left / right time difference decoding unit 271 and a time shift unit 281, and that the left channel signal adding unit 240 and the right channel signal adding unit 260 use the signal output by the time shift unit 281 instead of the signal output by the monaural decoding unit 210. The other configurations and operations of the decoding device 201 of the second reference embodiment are the same as those of the decoding device 200 of the first reference embodiment. The decoding device 201 of the second reference embodiment performs, for each frame, the processes from step S210 to step S281 illustrated in Fig. 13. Below, the differences between the decoding device 201 of the second reference embodiment and the decoding device 200 of the first reference embodiment will be described.

[0110] [Left / right time difference decoding unit 271] The left / right time difference code Cτ input to the decoding device 201 is input to the left / right time difference decoding unit 271. The left / right time difference decoding unit 271 decodes the left / right time difference code Cτ using a predetermined decoding method to obtain and output the left / right time difference τ (step S271). As the predetermined decoding method, a decoding method corresponding to the encoding method used by the left / right relationship information estimation unit 181 of the corresponding encoding device 101 is used. The left / right time difference τ obtained by the left / right time difference decoding unit 271 is the same value as the left / right time difference τ obtained by the left / right relationship information estimation unit 181 of the corresponding encoding device 101, and τ max From τ min The value can be any value in the range from

[0111] [Time shift section 281] The time shift unit 281 receives the monaural decoded sound signal ^x output from the monaural decoding unit 210. M (1), ^x M (2), ..., ^x MThe time shift unit 281 receives the monaural decoded sound signal ^x(T) and the left-right time difference τ output by the left-right time difference decoding unit 271. If the left-right time difference τ is a positive value (i.e., if the left-right time difference τ indicates that the left channel is leading), the time shift unit 281 decodes the monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M (T) is output as is to left channel signal adder 240 (i.e., it is determined to be used in left channel signal adder 240), and the monaural decoded sound signal is delayed by |τ| samples to obtain signal ^x M (1-|τ|), ^x M (2-|τ|), ..., ^x M Delayed mono decoded sound signal ^x where (T-|τ|) M' (1), ^x M' (2), ..., ^x M' (T) is output to right channel signal adder 260 (i.e., it is determined to be used in right channel signal adder 260), and if the left-right time difference τ is a negative value (i.e., if the left-right time difference τ indicates that the right channel is ahead), a signal ^x M (1-|τ|), ^x M (2-|τ|), ..., ^x M Delayed mono decoded sound signal ^x where (T-|τ|) M' (1), ^x M' (2), ..., ^x M' (T) is output to left channel signal adder 240 (i.e., it is determined to be used in left channel signal adder 240), and monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M (T) is output as is to the right channel signal adder 260 (i.e., it is determined to use it in the right channel signal adder 260), and if the left-right time difference τ is 0 (i.e., if the left-right time difference τ indicates that none of the channels are ahead), the monaural decoded sound signal ^x M (1), ^x M (2), ..., ^x M(T) is output as is to left channel signal adder 240 and right channel signal adder 260 (i.e., it is determined to use it in left channel signal adder 240 and right channel signal adder 260) (step S281). Note that, because time shifter 281 uses a monaural decoded sound signal of a past frame to obtain a delayed monaural decoded sound signal, a storage unit (not shown) in time shifter 281 stores monaural decoded sound signals input in past frames for a predetermined number of frames.

[0112] [Left channel signal adder 240, right channel signal adder 260] Left channel signal adder 240 and right channel signal adder 260 perform the same operations as those described in the first embodiment on monaural decoded sound signal ^x output by monaural decoding unit 210. M (1), ^x M (2), ..., ^x M Instead of (T), the monaural decoded sound signal ^x input from the time shift unit 281 M (1), ^x M (2), ..., ^x M (T) or delayed mono decoded sound signal ^x M' (1), ^x M' (2), ..., ^x M' That is, left channel signal adder 240 and right channel signal adder 260 use monaural decoded sound signal ^x determined by time shifter 281 (steps S240, S260). M (1), ^x M (2), ..., ^x M (T) or delayed mono decoded sound signal ^x M' (1), ^x M' (2), ..., ^x M' (T) is used to perform the same operations as those described in the first embodiment.

[0113] First Embodiment The first embodiment is a modification of the encoding device 101 of the second reference embodiment in which a downmix signal is generated taking into consideration the relationship between the left channel input sound signal and the right channel input sound signal. The encoding device of the first embodiment will be described below. Note that the code obtained by the encoding device of the first embodiment can be decoded by the decoding device 201 of the second reference embodiment, so a description of the decoding device will be omitted.

[0114] ≪Encoding device 102≫ As shown in Fig. 10, the encoding device 102 of the first embodiment includes a downmixing unit 112, a left channel subtraction gain estimating unit 120, a left channel signal subtracting unit 130, a right channel subtraction gain estimating unit 140, a right channel signal subtracting unit 150, a monaural encoding unit 160, a stereo encoding unit 170, a left / right relationship information estimating unit 182, and a time shifting unit 191. The encoding device 102 of the first embodiment differs from the encoding device 101 of the second reference embodiment in that it includes a left / right relationship information estimating unit 182 instead of the left / right relationship information estimating unit 181, includes a downmixing unit 112 instead of the downmixing unit 110, and, as shown by the dashed lines in Fig. 10, the left / right relationship information estimating unit 182 obtains and outputs left / right correlation coefficient γ and preceding channel information, and the output left / right correlation coefficient γ and preceding channel information are input to the downmixing unit 112 and used. The other configurations and operations of the encoding device 102 of the first embodiment are the same as those of the encoding device 101 of the second reference embodiment. The encoding device 102 of the first embodiment performs, for each frame, the processes from step S112 to step S191 illustrated in Fig. 14. Below, the differences between the encoding device 102 of the first embodiment and the encoding device 101 of the second reference embodiment will be described.

[0115] [Left-right relationship information estimation unit 182] The left-channel input sound signal input to the encoding device 102 and the right-channel input sound signal input to the encoding device 102 are input to the left-right relationship information estimation unit 182. From the input left-channel input sound signal and the input right-channel input sound signal, the left-right relationship information estimation unit 182 obtains and outputs the left-right time difference τ, the left-right time difference code Cτ which is a code representing the left-right time difference τ, the left-right correlation coefficient γ, and preceding channel information (step S182). The process by which the left-right relationship information estimation unit 182 obtains the left-right time difference τ and the left-right time difference code Cτ is similar to that of the left-right relationship information estimation unit 181 of the second reference embodiment.

[0116] The left-right correlation coefficient γ is information corresponding to the correlation coefficient between a sound signal that arrives from a sound source at a microphone for the left channel and is picked up, and a sound signal that arrives from the sound source at a microphone for the right channel and is picked up, in the assumption described above in the description of the left-right relation information estimation unit 181 in the second reference embodiment. The preceding channel information is information corresponding to which microphone the sound emitted from the sound source arrives at first, is information indicating whether the same sound signal is contained first in the input sound signal of the left channel or the input sound signal of the right channel, and is information indicating which channel, the left channel or the right channel, is preceding.

[0117] In the example described above in the description of the left-right relationship information estimation unit 181 of the second embodiment, the left-right relationship information estimation unit 182 calculates a correlation value between a sample sequence of an input sound signal of the left channel and a sample sequence of an input sound signal of the right channel that is shifted later than the sample sequence by the left-right time difference τ, that is, τ max From τ min The number of candidate samples up to τ cand The correlation value γ candThe left-right relationship information estimation unit 182 obtains and outputs the maximum value of the left-right correlation coefficient γ when the left-right time difference τ is a positive value, and obtains and outputs information indicating that the left channel is leading as leading channel information when the left-right time difference τ is a negative value, and obtains and outputs information indicating that the right channel is leading as leading channel information when the left-right time difference τ is 0. The left-right relationship information estimation unit 182 may obtain and output information indicating that the left channel is leading as leading channel information, or may obtain and output information indicating that the right channel is leading as leading channel information when the left-right time difference τ is 0, but it is preferable to obtain and output information indicating that none of the channels are leading as leading channel information.

[0118] [Downmix section 112] The downmix unit 112 receives as input the left channel input sound signal input to the encoding device 102, the right channel input sound signal input to the encoding device 102, the left / right correlation coefficient γ output by the left / right relationship information estimation unit 182, and the preceding channel information output by the left / right relationship information estimation unit 182. The downmix unit 112 obtains and outputs a downmix signal by performing a weighted average of the left channel input sound signal and the right channel input sound signal so that the input sound signal of the preceding channel out of the left channel input sound signal and the right channel input sound signal is included in the downmix signal to a greater extent as the left / right correlation coefficient γ increases (step S112).

[0119] For example, if the absolute value or normalized value of the correlation coefficient is used as the correlation value as in the example described above in the description of the left-right relationship information estimation unit 181 of the second reference embodiment, the obtained left-right correlation coefficient γ is a value between 0 and 1, so the downmix unit 112 uses a weight determined by the left-right correlation coefficient γ for each corresponding sample number t to obtain the left channel input sound signal x L (t) and the right channel input sound signal x R (t) is weighted and added to the downmix signal x MSpecifically, when the leading channel information is information indicating that the left channel is leading, that is, when the left channel is leading, the downmix unit 112 sets x M (t) = ((1 + γ) / 2) × x L (t) + ((1-γ) / 2) × x R (t), if the leading channel information is information indicating that the right channel is leading, that is, if the right channel is leading, then x M (t) = ((1-γ) / 2) × x L (t) + ((1 + γ) / 2) × x R (t), as the downmix signal x M By the downmix unit 112 obtaining the downmix signal in this manner, the downmix signal is closer to a signal obtained by averaging the left channel input sound signal and the right channel input sound signal as the left-right correlation coefficient γ is smaller, that is, the correlation between the left channel input sound signal and the right channel input sound signal is smaller, and the downmix signal is closer to the input sound signal of the preceding channel out of the left channel input sound signal and the right channel input sound signal as the left-right correlation coefficient γ is larger, that is, the correlation between the left channel input sound signal and the right channel input sound signal is larger.

[0120] When none of the channels are ahead, the downmix unit 112 preferably averages the left channel input sound signal and the right channel input sound signal to obtain and output a downmix signal so that the left channel input sound signal and the right channel input sound signal are included in the downmix signal with the same weight. Therefore, when the leading channel information indicates that none of the channels are ahead, the downmix unit 112 calculates the left channel input sound signal x L (t) and the right channel input sound signal x R (t) averaged x M (t)=(x L (t)+x R (t) / 2 is the downmix signal x M Let (t).

[0121] <Second embodiment> The encoding device 100 of the first reference embodiment may also be modified to generate a downmix signal in consideration of the relationship between the left channel input sound signal and the right channel input sound signal, and this embodiment will be described as the second embodiment. Note that the code obtained by the encoding device of the second embodiment can be decoded by the decoding device 200 of the first reference embodiment, so a description of the decoding device will be omitted.

[0122] ≪Encoding device 103≫ As shown in Fig. 1, the encoding device 103 of the second embodiment includes a downmixing unit 112, a left channel subtraction gain estimating unit 120, a left channel signal subtracting unit 130, a right channel subtraction gain estimating unit 140, a right channel signal subtracting unit 150, a monaural encoding unit 160, a stereo encoding unit 170, and a left-right relationship information estimating unit 183. The encoding device 103 of the second embodiment differs from the encoding device 100 of the first reference embodiment in that it includes a downmixing unit 112 instead of the downmixing unit 110, and includes a left-right relationship information estimating unit 183 as shown by a dashed line in Fig. 1, which obtains and outputs a left-right correlation coefficient γ and preceding channel information, and the output left-right correlation coefficient γ and preceding channel information are input to the downmixing unit 112 and used. The other configurations and operations of the encoding device 103 of the second embodiment are the same as those of the encoding device 100 of the first reference embodiment. Moreover, the operation of the downmixing unit 112 of the encoding device 103 of the second embodiment is the same as the operation of the downmixing unit 112 of the encoding device 102 of the first embodiment. The encoding device 103 of the second embodiment performs the processes of steps S112 to S183 illustrated in Fig. 15 for each frame. Below, the differences between the encoding device 103 of the second embodiment and the encoding device 100 of the first reference embodiment and the encoding device 102 of the first embodiment will be described.

[0123] [Left-right relationship information estimation unit 183] The left-channel input sound signal input to the encoding device 103 and the right-channel input sound signal input to the encoding device 103 are input to the left-right relationship information estimation unit 183. The left-right relationship information estimation unit 183 obtains and outputs a left-right correlation coefficient γ and preceding channel information from the input left-channel input sound signal and right-channel input sound signal (step S183).

[0124] The left-right correlation coefficient γ and preceding channel information obtained and output by the left-right relationship information estimation unit 183 are the same as those described in the first embodiment. That is, the left-right relationship information estimation unit 183 may be the same as the left-right relationship information estimation unit 182, except that the left-right relationship information estimation unit 183 does not need to obtain and output the left-right time difference τ and the left-right time difference code Cτ.

[0125] For example, the left-right relationship information estimation unit 183 calculates τ max From τ min The number of candidate samples up to τ cand For the left channel input sound signal sample sequence and each candidate sample number τ cand and a correlation value γ cand The maximum value of the left-right correlation coefficient γ is obtained and output, and τ when the correlation value is at its maximum value is cand When the correlation value is a positive value, information indicating that the left channel is leading is obtained and output as leading channel information, and τ cand When τ is a negative value, the left-right relationship information estimation unit 183 obtains and outputs information indicating that the right channel is leading as leading channel information. cand When is 0, information indicating that the left channel is leading may be obtained and output as leading channel information, or information indicating that the right channel is leading may be obtained and output as leading channel information, but it is preferable to obtain and output information indicating that neither channel is leading as leading channel information.

[0126] <Third embodiment> A configuration may be adopted in which a downmix signal is obtained by taking into consideration the relationship between the left channel input sound signal and the right channel input sound signal, even for an encoding device that stereo-encodes the input sound signal of each channel instead of the differential signal of each channel, and this configuration will be described as the third embodiment.

[0127] <<Encoding device 104>> As shown in Fig. 16, the encoding device 104 of the third embodiment includes a left-right relationship information estimation unit 183, a downmixing unit 112, a monaural encoding unit 160, and a stereo encoding unit 174. The encoding device 104 of the third embodiment performs, for each frame, processes of steps S183, S112, S160, and S174 illustrated in Fig. 17. The encoding device 104 of the third embodiment will be described below with appropriate reference to the description of the second embodiment.

[0128] [Left-right relationship information estimation unit 183] The left / right relationship information estimation unit 183 is the same as the left / right relationship information estimation unit 183 in the second embodiment. The left / right relationship information estimation unit 183 receives the left channel input sound signal input to the encoding device 104 and the right channel input sound signal input to the encoding device 104. The left / right relationship information estimation unit 183 obtains and outputs a left / right correlation coefficient γ, which is a correlation coefficient between the left channel input sound signal and the right channel input sound signal, and preceding channel information, which is information indicating whether the left channel input sound signal or the right channel input sound signal is preceding, from the input left channel input sound signal and right channel input sound signal (step S183).

[0129] [Downmix section 112] The downmixing unit 112 is the same as the downmixing unit 112 of the second embodiment. The downmixing unit 112 receives the left channel input sound signal input to the encoding device 104, the right channel input sound signal input to the encoding device 104, the left / right correlation coefficient γ output by the left / right relationship information estimation unit 183, and the preceding channel information output by the left / right relationship information estimation unit 183. The downmixing unit 112 obtains and outputs a downmix signal by performing a weighted average of the left channel input sound signal and the right channel input sound signal so that the input sound signal of the preceding channel out of the left channel input sound signal and the right channel input sound signal is included in the downmix signal to a greater extent as the left / right correlation coefficient γ increases (step S112).

[0130] For example, let the sample number be t, and the input sound signal in the left channel be x L (t), and the input sound signal on the right channel is x R (t) and the downmix signal is x M (t), the downmix unit 112 calculates x for each sample number t when the preceding channel information indicates that the left channel is preceding. M (t) = ((1 + γ) / 2) × x L (t) + ((1-γ) / 2) × x R (t) to obtain a downmix signal, and if the leading channel information indicates that the right channel is leading, then for each sample number t, M (t) = ((1-γ) / 2) × x L (t) + ((1 + γ) / 2) × x R (t) to obtain a downmix signal, and if the leading channel information indicates that no channel is leading, then for each sample number t, M (t)=(x L (t)+x R The downmix signal is obtained by (t) / 2.

[0131] [Monaural encoding unit 160] The mono encoding unit 160 is the same as the mono encoding unit 160 in the second embodiment. The downmix signal output by the downmix unit 112 is input to the mono encoding unit 160. The mono encoding unit 160 encodes the input downmix signal to obtain a mono code CM and outputs the mono code CM (step S160). The mono encoding unit 160 may use any encoding method, for example, an encoding method such as the 3GPP EVS standard may be used. The encoding method may be an encoding method that performs encoding processing independently of the stereo encoding unit 174 described later, that is, an encoding method that performs encoding processing without using the stereo code CS' obtained by the stereo encoding unit 174 or information obtained in the encoding processing performed by the stereo encoding unit 174, or an encoding method that performs encoding processing using the stereo code CS' obtained by the stereo encoding unit 174 or information obtained in the encoding processing performed by the stereo encoding unit 174.

[0132] [Stereo encoding unit 174] The stereo encoding unit 174 receives the left channel input sound signal input to the encoding device 104 and the right channel input sound signal input to the encoding device 104. The stereo encoding unit 174 encodes the input left channel input sound signal and the right channel input sound signal to obtain and output a stereo code CS' (step S174). The stereo encoding unit 174 may use any encoding method, for example, a stereo encoding method corresponding to the stereo decoding method of the MPEG-4 AAC standard may be used, or an encoding method that independently encodes the input left channel input sound signal and the input right channel input sound signal may be used, and the stereo code CS' may be a combination of all the codes obtained by encoding. The encoding method may be an encoding method that performs encoding processing independently of mono encoding unit 160, i.e., an encoding method that performs encoding processing without using the mono code CM obtained by mono encoding unit 160 or information obtained in the encoding processing performed by mono encoding unit 160, or an encoding method that performs encoding processing using the mono code CM obtained by mono encoding unit 160 or information obtained in the encoding processing performed by mono encoding unit 160.

[0133] <Fourth embodiment> As can be seen from the above description of the embodiment, as long as the encoding device at least encodes a downmix signal obtained from a left channel input sound signal and a right channel input sound signal to obtain a code, whatever the encoding device may be, a configuration may be adopted to obtain a downmix signal taking into consideration the relationship between the left channel input sound signal and the right channel input sound signal. Furthermore, as long as the signal processing device at least processes a downmix signal obtained from a left channel input sound signal and a right channel input sound signal to obtain a signal processing result, regardless of the type of signal processing device, a configuration may be adopted to obtain a downmix signal taking into consideration the relationship between the left channel input sound signal and the right channel input sound signal. Furthermore, as a downmix device used in a stage preceding these encoding devices and signal processing devices, a configuration may be adopted to obtain a downmix signal taking into consideration the relationship between the left channel input sound signal and the right channel input sound signal. These forms will be described as the fourth embodiment.

[0134] ≪Sound signal encoding device 105≫ As shown in Fig. 18, the sound signal encoding device 105 of the fourth embodiment includes a left-right relationship information estimation unit 183, a downmixing unit 112, and an encoding unit 195. The sound signal encoding device 105 of the fourth embodiment performs, for each frame, processes of steps S183, S112, and S195 illustrated in Fig. 19. The sound signal encoding device 105 of the fourth embodiment will be described below with appropriate reference to the description of the second embodiment.

[0135] [Left-right relationship information estimation unit 183] The left / right relationship information estimation unit 183 is the same as the left / right relationship information estimation unit 183 in the second embodiment, and obtains and outputs, from the input left channel input sound signal and right channel input sound signal, the left / right correlation coefficient γ which is the correlation coefficient between the left channel input sound signal and the right channel input sound signal, and leading channel information which is information indicating whether the left channel input sound signal or the right channel input sound signal is leading (step S183).

[0136] [Downmix section 112] The downmix unit 112 is the same as the downmix unit 112 in the second embodiment, and obtains and outputs a downmix signal by weighting and averaging the left channel input sound signal and the right channel input sound signal so that the input sound signal of the preceding channel out of the left channel input sound signal and the right channel input sound signal is included more significantly in the downmix signal as the left-right correlation coefficient γ is larger (step S112).

[0137] [Encoding unit 195] At least the downmix signal output by the downmix unit 112 is input to the encoding unit 195. The encoding unit 195 at least encodes the input downmix signal to obtain and output a sound signal code (step S195). The encoding unit 195 may also encode the left channel input sound signal and the right channel input sound signal, and may output the code obtained by this encoding together with the sound signal code. In this case, as indicated by the dashed line in Fig. 18, the left channel input sound signal and the right channel input sound signal are also input to the encoding unit 195.

[0138] <Sound signal processing device 305> As shown in Fig. 20, the sound signal processing device 305 of the fourth embodiment includes a left-right relationship information estimation unit 183, a downmixing unit 112, and a signal processing unit 315. The sound signal processing device 305 of the fourth embodiment performs, for each frame, the processes of steps S183, S112, and S315 illustrated in Fig. 21. Below, the sound signal processing device 305 of the fourth embodiment will be described in terms of differences from the sound signal encoding device 105 of the fourth embodiment.

[0139] [Signal processing unit 315] At least the downmix signal output by the downmix unit 112 is input to the signal processing unit 315. The signal processing unit 315 performs at least signal processing on the input downmix signal to obtain and output the signal processing result (step S315). The signal processing unit 315 may also perform signal processing on a left channel input sound signal and a right channel input sound signal to obtain the signal processing result. In this case, as shown by the dashed line in FIG. 20, the left channel input sound signal and the right channel input sound signal are also input to the signal processing unit 315. The signal processing unit 315 may, for example, perform signal processing using the downmix signal on the input sound signal of each channel to obtain an output sound signal of each channel as a signal processing result, or may perform this signal processing on a left channel decoded sound signal and a right channel decoded sound signal obtained by decoding the code CS' obtained by the stereo encoding unit 174 of the third embodiment by a decoding device including a decoding unit corresponding to the stereo encoding unit 174. In other words, it is not essential that the left channel input sound signal and the right channel input sound signal input to the sound signal processing device 305 are digital voice signals or acoustic signals obtained by picking up sound from two microphones respectively and performing AD conversion. The left channel input sound signal and the right channel input sound signal input to the sound signal processing device 305 may be a left channel decoded sound signal and a right channel decoded sound signal obtained by decoding a code, or may be sound signals obtained in any way as long as they are stereo two-channel sound signals.

[0140] In cases where the left channel input sound signal and the right channel input sound signal input to the sound signal processing device 305 are the decoded left channel sound signal and the decoded right channel sound signal obtained by decoding the code in another device, either or both of the left and right correlation coefficient γ and the preceding channel information obtained by the left and right relationship information estimation unit 183 may be obtained in another device. When either or both of the left and right correlation coefficient γ and the preceding channel information are obtained in another device, either or both of the left and right correlation coefficient γ and the preceding channel information obtained in the other device may be input to the sound signal processing device 305, as shown by the dashed dotted line in Fig. 20. In this case, the left and right relationship information estimation unit 183 may obtain the left and right correlation coefficient γ or the preceding channel information that was not input to the sound signal processing device 305, and when both of the left and right correlation coefficient γ and the preceding channel information are input to the sound signal processing device 305, the sound signal processing device 305 does not need to include the left and right relationship information estimation unit 183 and does not need to perform step S183. That is, as shown by the two-dot chain line in Fig. 20, the sound signal processing device 305 is provided with a left-right relationship information acquisition unit 185, which acquires and outputs a left-right correlation coefficient γ that is a correlation coefficient between the left channel input sound signal and the right channel input sound signal, and preceding channel information that is information indicating which of the left channel input sound signal and the right channel input sound signal is preceding (step S185). Note that the left-right relationship information estimation unit 183 and step S183 of each device described above can also be said to be within the scope of the left-right relationship information acquisition unit 185 and step S185.

[0141] <Sound signal downmix device 405> As shown in Fig. 22, the sound signal downmixing device 405 of the fourth embodiment includes a left-right relationship information acquisition unit 185 and a downmixing unit 112. The sound signal downmixing device 405 performs the processes of step S185 and step S112 illustrated in Fig. 23 for each frame. The sound signal downmixing device 405 will be described below with reference to the description of the second embodiment as appropriate. Note that, like the sound signal processing device 305, the left channel input sound signal and the right channel input sound signal input to the sound signal downmixing device 405 may be digital audio signals or acoustic signals obtained by picking up sound with two microphones and AD converting the sound signals, or may be decoded left channel sound signals and decoded right channel sound signals obtained by decoding codes, or may be sound signals obtained in any manner as long as they are stereo two-channel sound signals.

[0142] [Left-right relationship information acquisition unit 185] The left-right relationship information acquisition unit 185 obtains and outputs the left-right correlation coefficient γ, which is the correlation coefficient between the left channel input sound signal and the right channel input sound signal, and leading channel information, which is information indicating whether the left channel input sound signal or the right channel input sound signal is leading (step S185).

[0143] When both the left-right correlation coefficient γ and the preceding channel information are obtained from another device, as shown by the dashed dotted line in FIG. 22 , the left-right relationship information acquisition unit 185 obtains the left-right correlation coefficient γ and the preceding channel information input to the sound signal downmixing device 405 from the other device, and outputs them to the downmixing unit 112.

[0144] When both the left-right correlation coefficient γ and the preceding channel information are not obtained by another device, the left-right relationship information acquisition unit 185 includes a left-right relationship information estimation unit 183, as indicated by a dashed line in Fig. 22. The left-right relationship information estimation unit 183 obtains the left-right correlation coefficient γ and the preceding channel information from the left channel input sound signal and the right channel input sound signal in the same way as the left-right relationship information estimation unit 183 in the second embodiment, and outputs the obtained information to the downmixing unit 112.

[0145] When either the left-right correlation coefficient γ or the preceding channel information is not obtained by another device, the left-right relationship information acquisition unit 185 includes a left-right relationship information estimation unit 183, as shown by a dashed line in Fig. 22. The left-right relationship information estimation unit 183 of the left-right relationship information acquisition unit 185 acquires the left-right correlation coefficient γ not obtained by another device or the preceding channel information not obtained by another device from the left channel input sound signal and the right channel input sound signal in the same manner as the left-right relationship information estimation unit 183 of the second embodiment, and outputs the acquired left-right correlation coefficient γ or the preceding channel information to the downmixing unit 112. As for the left-right correlation coefficient γ obtained by another device or the preceding channel information obtained by another device, the left-right relationship information acquisition unit 185 outputs the left-right correlation coefficient γ or the preceding channel information input to the sound signal downmixing device 405 from the other device to the downmixing unit 112, as shown by a dashed line in Fig. 22.

[0146] [Downmix section 112] The downmix unit 112 is the same as the downmix unit 112 in the second embodiment, and based on the preceding channel information and left-right correlation coefficient acquired by the left-right relationship information acquisition unit 185, obtains and outputs a downmix signal by weighting and averaging the left channel input sound signal and the right channel input sound signal so that the input sound signal of the preceding channel out of the left channel input sound signal and the right channel input sound signal is included more significantly in the downmix signal as the left-right correlation coefficient γ is larger (step S112).

[0147] For example, let the sample number be t, and the input sound signal in the left channel be x L (t), and the input sound signal on the right channel is x R (t) and the downmix signal is x M (t), the downmix unit 112 calculates x for each sample number t when the preceding channel information indicates that the left channel is preceding. M (t) = ((1 + γ) / 2) × x L (t) + ((1-γ) / 2) × x R (t) to obtain a downmix signal, and if the leading channel information indicates that the right channel is leading, then for each sample number t, M(t) = ((1-γ) / 2) × x L (t) + ((1 + γ) / 2) × x R (t) to obtain a downmix signal, and if the leading channel information indicates that no channel is leading, then for each sample number t, M (t)=(x L (t)+x R The downmix signal is obtained by (t) / 2.

[0148] <Programs and recording media> The processing of each of the above-mentioned encoding devices, decoding devices, audio signal encoding device, audio signal processing device, and audio signal downmixing device may be realized by a computer, in which case the processing contents of the functions to be possessed by each device are described by a program. Then, by loading this program into the storage unit 1020 of the computer 1000 shown in Fig. 24 and operating the arithmetic processing unit 1010, input unit 1030, output unit 1040, etc., various processing functions of the above-mentioned devices are realized on the computer.

[0149] The program describing the processing contents can be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a non-transitory recording medium, specifically, a magnetic recording device, an optical disk, etc.

[0150] The program may be distributed, for example, by selling, transferring, lending, etc. portable recording media such as DVDs and CD-ROMs on which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of a server computer and transferring the program from the server computer to other computers via a network.

[0151] A computer that executes such a program, for example, first stores a program recorded on a portable recording medium or a program transferred from a server computer in the auxiliary recording unit 1050, which is a non-transient storage device of the computer. Then, when executing a process, the computer reads the program stored in the auxiliary recording unit 1050, which is a non-transient storage device of the computer, into the storage unit 1020, and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from the portable recording medium into the storage unit 1020 and execute a process according to the program, or may execute a process according to the received program each time a program is transferred from the server computer to the computer. Alternatively, the server computer may not transfer the program to the computer, but may execute the above-mentioned process by a so-called ASP (Application Service Provider) type service that realizes a processing function only by issuing an execution instruction and obtaining a result. Note that the program in this embodiment includes information used for processing by an electronic computer and equivalent to a program (data that is not a direct instruction to a computer but has a nature that specifies the processing of the computer, etc.).

[0152] In addition, in this embodiment, the present device is configured by executing a predetermined program on a computer, but at least a part of the processing contents may be realized by hardware.

[0153] Needless to say, other modifications can be made without departing from the spirit of the present invention.

Claims

1. 1. A sound signal downmixing method for obtaining a downmix signal which is a signal obtained by mixing a first channel input sound signal and a second channel input sound signal, comprising the steps of: acquiring preceding channel information which is information indicating which of the first channel input sound signal and the second channel input sound signal is preceding; a downmix step of obtaining the downmix signal from the preceding channel and another channel based on a degree determined based on the preceding channel information and a correlation coefficient which is a coefficient indicating a magnitude of correlation between the first channel input sound signal and the second channel input sound signal; having In the downmixing step, a leading signal of the first channel input sound signal and the second channel input sound signal is given a greater weight than the other signal. Sound signal downmix method.

2. 1. A sound signal downmixing device for obtaining a downmix signal which is a signal obtained by mixing a first channel input sound signal and a second channel input sound signal, a unit for acquiring preceding channel information which is information indicating which of the first channel input sound signal and the second channel input sound signal is preceding; a downmix unit that obtains the downmix signal from the preceding channel and another channel based on a degree determined based on the preceding channel information and a correlation coefficient that is a coefficient indicating a magnitude of correlation between the first channel input sound signal and the second channel input sound signal; having In the downmixing unit, a leading signal of the first channel input sound signal and the second channel input sound signal is weighted more heavily than the other signal. Sound signal downmix device.

3. A program for causing a computer to execute each step of a sound signal downmixing method for obtaining a downmix signal, which is a signal obtained by mixing a first channel input sound signal and a second channel input sound signal, comprising: The audio signal downmixing method includes: acquiring preceding channel information which is information indicating which of the first channel input sound signal and the second channel input sound signal is preceding; a downmix step of obtaining the downmix signal from the preceding channel and another channel based on a degree determined based on the preceding channel information and a correlation coefficient which is a coefficient indicating a magnitude of correlation between the first channel input sound signal and the second channel input sound signal; having In the downmixing step, a leading signal of the first channel input sound signal and the second channel input sound signal is given a greater weight than the other signal. program.

Images