Speech and audio encoding device, speech and audio decoding device, speech and audio encoding method, and speech and audio decoding method

By converting input signals to the frequency domain, estimating energy envelopes, and grouping them based on energy and norm variance, the method enhances bit allocation efficiency and sound quality in audio-acoustic encoding.

JP7581265B2Active Publication Date: 2024-11-12FRAUNHOFER GESELLSCHAFT ZUR FORDERUNG DER ANGEWANDTEN FORSCHUNG EV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2022003475
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2012-12-13
Filing Date
2022-01-13
Publication Date
2024-11-12
Estimated Expiration
2033-11-26

AI Technical Summary

Technical Problem

Existing bit allocation methods in audio-acoustic encoding do not consider input signal characteristics, leading to inefficient bit allocation and suboptimal sound quality.

Method used

An audio-acoustic encoding device and method that converts input signals to the frequency domain, estimates an energy envelope, groups quantized energy envelopes, and allocates bits to these groups and subbands based on energy and norm variance, enhancing bit allocation efficiency.

Benefits of technology

This approach enables efficient bit allocation and improves sound quality by prioritizing bit allocation to perceptually important frequency bands.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007581265000009
    Figure 0007581265000009
  • Figure 0007581265000010
    Figure 0007581265000010
  • Figure 0007581265000011
    Figure 0007581265000011
Patent Text Reader

Abstract

Provided are a speech and audio encoding device, speech and audio decoding device, speech and audio encoding method, and speech and audio decoding method that perform efficient bit allocation and improve sound quality. In a bit allocation unit (107) of a speech and audio coding device, a dominant frequency band identification unit (301) identifies dominant frequency bands in which the norm coefficient value has a maximum value in the spectrum of an input speech and audio signal, and dominant group determination units (302-1 to 302-N) and a non-dominant group determination unit (303) group all subbands into dominant groups including dominant frequency bands and non-dominant groups not including dominant frequency bands. A group bit allocation unit (308) allocates bits to each group based on the energy and norm variance of each group, and a subband bit allocation unit (309) further allocates the bits allocated to each group to each subband in accordance with the ratio of the norm to the energy of the group.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a speech and audio coding device, a speech and audio decoding device, a speech and audio coding method, and a speech and audio decoding method using a transform coding scheme. [Background technology]

[0002] There is a technology standardized by the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) that can efficiently encode full band (FB) voice or music signals from 0.02 to 20 kHz. This technology converts the input signal into the frequency domain and encodes the band up to 20 kHz (transform coding).

[0003] Here, transform coding is a coding method that uses a time-frequency transformation such as the Discrete Cosine Transform (DCT) or the Modified Discrete Cosine Transform (MDCT) to convert an input signal from the time domain to the frequency domain, enabling the signal to be mapped in a way that accurately corresponds to the auditory characteristics.

[0004] In transform coding, the spectral coefficients are divided into multiple frequency subbands, and in coding each subband, more quantization bits are allocated to bands that are perceptually important to the human ear, thereby improving the overall sound quality.

[0005] In order to achieve this goal, efficient bit allocation methods have been studied, and for example, a technique disclosed in Non-Patent Document 1 is known. The bit allocation method disclosed in Patent Document 1 will be described below with reference to Figs. 1 and 2.

[0006] 1 is a block diagram showing the configuration of the speech and audio coding device disclosed in Patent Document 1. An input signal sampled at 48 kHz is input to a transient detector 11 and a transform unit 12 of the speech and audio coding device.

[0007] The transient detector 11 detects from the input signal either a transient frame corresponding to the beginning or end of speech, or a stationary frame corresponding to other speech segments, and the transform unit 12 applies a high frequency resolution transform or a low frequency resolution transform to the frame of the input signal depending on whether the frame detected by the transient detector 11 is a transient frame or a stationary frame, to obtain spectral coefficients (or transform coefficients).

[0008] The norm estimation unit 13 divides the spectral coefficients obtained by the conversion unit 12 into bands with different bandwidths, and estimates the norm (or energy) of each of the divided bands.

[0009] The norm quantization unit 14 obtains a spectral envelope consisting of the norms of all the bands based on the norm of each band estimated by the norm estimation unit 13, and quantizes the obtained spectral envelope.

[0010] The spectrum normalization unit 15 normalizes the spectrum coefficients obtained by the conversion unit 12 using the norm quantized by the norm quantization unit 14 .

[0011] The norm adjustment unit 16 adjusts the norm quantized by the norm quantization unit 14 based on adaptive spectral weighting.

[0012] The bit allocating unit 17 uses the quantization norm adjusted by the norm adjusting unit 16 to allocate available bits to each band within the frame.

[0013] The lattice vector encoding unit 18 performs lattice vector encoding on the spectral coefficients normalized by the spectrum normalization unit 15 using the bits allocated for each band by the bit allocation unit 17 .

[0014] The noise level adjustment unit 19 estimates the level of the spectral coefficient before encoding in the lattice vector encoding unit 18, and encodes the estimated level, thereby obtaining a noise level adjustment index.

[0015] The multiplexer 20 multiplexes the frame structure of the input signal acquired by the conversion unit 12, i.e., the transient signal flag indicating whether it is a stationary frame or a transient frame, the norm quantized by the norm quantization unit 14, the lattice code vector obtained by the lattice vector encoding unit 18, and the noise level adjustment index obtained by the noise level adjustment unit 19, to form a bit stream, and transmits the bit stream to the speech and audio decoding device.

[0016] 2 is a block diagram showing the configuration of the speech and audio decoding device disclosed in Patent Document 1. A bit stream transmitted from a speech and audio encoding device is received in the speech and audio decoding device, and is demultiplexed by a demultiplexer 21.

[0017] The norm inverse quantization unit 22 inverse quantizes the quantized norm to obtain a spectral envelope consisting of the norms of all bands, and the norm adjustment unit 23 adjusts the norm inverse quantized by the norm inverse quantization unit 22 based on adaptive spectral weighting.

[0018] The bit allocator 24 allocates available bits for each band in the frame using the norm adjusted by the norm adjuster 23. That is, the bit allocator 24 recalculates the bit allocation required to decode the lattice vector code of the normalized spectral coefficients.

[0019] The lattice decoding unit 25 decodes the transient signal flag, and based on the frame configuration indicated by the decoded transient signal flag and the bits allocated by the bit allocating unit 24, decodes the lattice code vector to obtain spectral coefficients.

[0020] The spectral fill generator 26 regenerates low-frequency spectral coefficients to which bits have not been allocated, using a codebook created based on the spectral coefficients decoded by the lattice decoder 25. The spectral fill generator 26 also adjusts the levels of the regenerated spectral coefficients, using a noise level adjustment index. Furthermore, the spectral fill generator 26 regenerates high-frequency uncoded spectral coefficients using low-frequency coded spectral coefficients.

[0021] An adder 27 sums the decoded and regenerated spectral coefficients to produce normalized spectral coefficients.

[0022] The envelope shaping unit 28 applies the spectral envelope dequantized by the norm dequantization unit 22 to the normalized spectral coefficients generated by the adder 27 to generate full-band spectral coefficients.

[0023] The inverse transform unit 29 applies an inverse transform such as an inverse modified discrete cosine transform (IMDCT) to the full-band spectrum coefficients generated by the envelope shaping unit 28 to convert them into time-domain signals. Here, in the case of stationary frames, an inverse transform with a high frequency resolution is applied, and in the case of transient frames, an inverse transform with a low frequency resolution is applied. It is used.

[0024] In G.719, the spectral coefficients are divided into spectral groups. Each spectral group is divided into bands of equal length sub-vectors, as shown in Figure 3. The sub-vectors have different lengths between groups, which increase with increasing frequency. For the resolution of the transform, lower frequencies use a higher frequency resolution and higher frequencies use a lower frequency resolution. As stated in G.719, the grouping allows for efficient use of the available bit budget during encoding.

[0025] In addition, in G.719, the bit allocation method is the same in the encoding device and the decoding device. Here, the bit allocation method will be explained with reference to FIG.

[0026] As shown in FIG. 4, in step (hereinafter abbreviated as "ST") 31, the quantized norm is adjusted before bit allocation in order to adjust the psychoacoustic weighting and masking effects.

[0027] In ST32, the subband with the maximum norm among all the subbands is identified, and in ST33, one bit is assigned for each spectral coefficient in the subband with the maximum norm, i.e., bits are assigned as many as the number of spectral coefficients.

[0028] In ST34, the norm is decreased according to the allocated bits, and in ST35, it is determined whether the remaining number of allocatable bits is 8 bits or more. If the remaining number of allocatable bits is 8 bits or more, the process returns to ST32, and if the remaining number of allocatable bits is less than 8 bits, the bit allocation procedure ends.

[0029] Thus, the bit allocation method allocates the available bits in a frame among the subbands using the adjusted quantization norm, and the normalized spectral coefficients are then coded by lattice vector coding with the bits allocated to each subband. [Prior art documents] [Patent documents]

[0030] [Non-Patent Document 1] ITU-T Recommendation G.719, “Low-complexity full-band audio coding for high-quality conversation applications,” ITU-T, 2009. Summary of the Invention [Problem to be solved by the invention]

[0031] However, in the above bit allocation method, since the input signal characteristics are not taken into consideration when grouping the spectral bands, efficient bit allocation cannot be performed, and there is a problem that further improvement in sound quality cannot be expected.

[0032] An object of the present invention is to provide a speech and audio encoding device, a speech and audio decoding device, a speech and audio encoding method, and a speech and audio decoding method that perform efficient bit allocation and improve sound quality. [Means for solving the problem]

[0033] The speech and audio coding device of the present invention includes a conversion means for converting an input signal from a time domain to a frequency domain, an estimation means for estimating an energy envelope representing an energy level for each of a plurality of subbands into which a frequency spectrum of the input signal is divided, a quantization means for quantizing the energy envelope, a group determination means for grouping the quantized energy envelope into a plurality of groups, a first bit allocation means for allocating bits to the plurality of groups, and a subband determination means for determining the bits allocated to the plurality of groups for each group. and encoding means for encoding the frequency spectrum using the bits allocated to the subbands.

[0034] The speech and audio decoding device of the present invention is configured to include an inverse quantization means for inverse quantizing a quantized spectral envelope, a group determination means for grouping the quantized spectral envelope into a plurality of groups, a first bit allocation means for allocating bits to the plurality of groups, a second bit allocation means for allocating the bits allocated to the plurality of groups to subbands for each group, a decoding means for decoding a frequency spectrum of a speech and audio signal using the bits allocated to the subbands, an envelope shaping means for applying the inverse quantized spectral envelope to the decoded frequency spectrum to reproduce a decoded spectrum, and an inverse transform means for inverse transforming the decoded spectrum from the frequency domain to the time domain.

[0035] The speech and audio coding method of the present invention converts an input signal from a time domain to a frequency domain, estimates an energy envelope representing an energy level for each of a plurality of subbands into which a frequency spectrum of the input signal is divided, quantizes the energy envelope, groups the quantized energy envelopes into a plurality of groups, assigns bits to the plurality of groups, assigns the bits assigned to the plurality of groups to a subband for each group, and encodes the frequency spectrum using the bits assigned to the subbands.

[0036] The speech and audio decoding method of the present invention includes dequantizing a quantized spectral envelope, grouping the quantized spectral envelope into a plurality of groups, allocating bits to the plurality of groups, allocating the bits allocated to the plurality of groups to a subband for each group, decodes a frequency spectrum of a speech and audio signal using the bits allocated to the subbands, applying the dequantized spectral envelope to the decoded frequency spectrum to reproduce a decoded spectrum, and inverse transforming the decoded spectrum from the frequency domain to the time domain. Effect of the Invention

[0037] According to the present invention, efficient bit allocation can be achieved, thereby improving sound quality. [Brief description of the drawings]

[0038] [Figure 1] A block diagram showing the configuration of a speech and audio encoding device disclosed in Patent Document 1. [Diagram 2] A block diagram showing the configuration of a speech and audio decoding device disclosed in Patent Document 1. [Diagram 3] FIG. 1 is a diagram showing grouping of spectral coefficients in a stationary mode disclosed in Patent Document 1. [Figure 4] Flow diagram showing the bit allocation method disclosed in Patent Document 1 [Diagram 5] FIG. 1 is a block diagram showing a configuration of a speech and audio encoding device according to an embodiment of the present invention; [Figure 6] FIG. 1 is a block diagram showing a configuration of a speech and audio decoding device according to an embodiment of the present invention; [Figure 7] A block diagram showing the internal configuration of the bit allocation unit shown in FIG. [Figure 8] FIG. 1 is a diagram for explaining a grouping method according to an embodiment of the present invention; [Figure 9] Diagram showing norm variance DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0039] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0040] (One embodiment) 5 is a block diagram showing a configuration of a speech and audio coding apparatus 100 according to an embodiment of the present invention. An input signal sampled at 48 kHz is input to a transient detector 101 and a transform unit 102 of the speech and audio coding apparatus 100.

[0041] The transient detector 101 detects from the input signal either a transient frame corresponding to the beginning or end of speech, or a stationary frame corresponding to other speech segments, and outputs the detection result to the conversion unit 102. The conversion unit 102 applies high frequency resolution conversion or low frequency resolution conversion to the frames of the input signal depending on whether the detection result output from the transient detector 101 is a transient frame or a stationary frame, obtains spectral coefficients (or conversion coefficients), and outputs them to the norm estimation unit 103 and the spectrum normalization unit 105. The conversion unit 102 also outputs to the multiplexer 110 the frame configuration that is the detection result output from the transient detector 101, i.e., a transient signal flag indicating whether it is a stationary frame or a transient frame.

[0042] The norm estimation unit 103 divides the spectral coefficients output from the conversion unit 102 into bands with different bandwidths, and estimates the norm (or energy) of each of the divided bands. The norm estimation unit 103 outputs the estimated norm of each band to the norm quantization unit 104.

[0043] The norm quantization unit 104 calculates a spectral envelope consisting of the norms of all bands based on the norms of each band output from the norm estimation unit 103, quantizes the calculated spectral envelope, and outputs the quantized spectral envelope to the spectral normalization unit 105 and the norm adjustment unit 106.

[0044] The spectral normalization unit 105 normalizes the spectral coefficients output from the conversion unit 102 using the quantized spectral envelope output from the norm quantization unit 104, and outputs the normalized spectral coefficients to the lattice vector encoding unit .

[0045] The norm adjustment unit 106 adjusts the quantized spectral envelope output from the norm quantization unit 104 based on adaptive spectral weighting, and outputs the adjusted quantized spectral envelope to the bit allocation unit 107 .

[0046] The bit allocating unit 107 allocates available bits for each band in the frame using the adjusted quantized spectral envelope output from the norm adjusting unit 106, and outputs the allocated bits to the lattice vector coding unit 108. The bit allocating unit 107 will be described in detail later.

[0047] The lattice vector encoding unit 108 performs lattice vector encoding on the spectral coefficients normalized by the spectrum normalization unit 105 using the bits allocated for each band by the bit allocation unit 107, and outputs the lattice code vector to the noise level adjustment unit 109 and the multiplexer 110.

[0048] The noise level adjustment unit 109 estimates the levels of the spectral coefficients before encoding in the lattice vector encoding unit 108, and encodes the estimated levels. In this way, a noise level adjustment index is obtained. The noise level adjustment index is output to the multiplexer 110.

[0049] The multiplexer 110 multiplexes the transient signal flag output from the conversion unit 102, the quantized spectral envelope output from the norm quantization unit 104, the lattice code vector output from the lattice vector encoding unit 108, and the noise level adjustment index output from the noise level adjustment unit 109 to form a bit stream, and transmits the bit stream to the speech and audio decoding device.

[0050] 6 is a block diagram showing a configuration of a speech and audio decoding apparatus 200 according to an embodiment of the present invention. A bitstream transmitted from the speech and audio encoding apparatus 100 is received in the speech and audio decoding apparatus 200, and is demultiplexed by the demultiplexer 201.

[0051] The norm inverse quantization unit 202 inverse quantizes the quantized spectral envelope (i.e., norm) output from the multiplexer, finds a spectral envelope made up of the norms of all bands, and outputs the found spectral envelope to the norm adjustment unit 203.

[0052] The norm adjustment unit 203 adjusts the spectral envelope output from the norm inverse quantization unit 202 based on adaptive spectral weighting, and outputs the adjusted spectral envelope to the bit allocation unit 204 .

[0053] The bit allocator 204 allocates available bits for each band in the frame using the spectral envelope output from the norm adjuster 203. That is, the bit allocator 204 recalculates the bit allocation required to decode the lattice vector code of the normalized spectral coefficients. The allocated bits are output to the lattice decoder 205.

[0054] The lattice decoding unit 205 decodes the lattice code vector output from the demultiplexer 201 based on the frame structure indicated by the transient signal flag output from the demultiplexer 201 and the bits output from the bit allocation unit 204 to obtain spectral coefficients. The spectral coefficients are output to the spectral filter generator 206 and the adder 207.

[0055] The spectral fill generator 206 regenerates low-frequency spectral coefficients to which no bits have been allocated, using a codebook created based on the spectral coefficients output from the lattice decoder 205. The spectral fill generator 206 also adjusts the levels of the regenerated spectral coefficients, using the noise level adjustment index output from the demultiplexer 201. The spectral fill generator 206 also regenerates high-frequency uncoded spectral coefficients, using low-frequency coded spectral coefficients. The level-adjusted low-frequency spectral coefficients and the regenerated high-frequency spectral coefficients are output to an adder 207.

[0056] The adder 207 combines the spectral coefficients output from the lattice decoding unit 205 and the spectral coefficients output from the spectral filter generator 206 to generate normalized spectral coefficients, and outputs the normalized spectral coefficients to the envelope shaping unit 208.

[0057] The envelope shaping unit 208 generates full-band spectral coefficients (corresponding to a decoded spectrum) by applying the spectral envelope output from the norm inverse quantization unit 202 to the normalized spectral coefficients generated by the adder 207. The generated full-band spectral coefficients are output to the inverse transform unit 209.

[0058] The inverse transform unit 209 applies an inverse transform such as an inverse modified discrete cosine transform (IMDCT) to the full-band spectrum coefficients output from the envelope shaping unit 208, converts them into a time domain signal, and outputs an output signal. Here, in the case of a stationary frame, an inverse transform with a high frequency resolution is applied, and in the case of a transient frame, an inverse transform with a low frequency resolution is applied.

[0059] Next, details of the bit allocating unit 107 will be described with reference to Fig. 7. Since the bit allocating unit 107 of the speech and audio encoding device 100 and the bit allocating unit 204 of the speech and audio decoding device 200 have the same configuration, only the bit allocating unit 107 will be described here, and a description of the bit allocating unit 204 will be omitted.

[0060] Fig. 7 is a block diagram showing the internal configuration of the bit allocation unit 107 shown in Fig. 5. The dominant frequency band identification unit 301 detects the quantized spectral envelope output from the norm adjustment unit 106. Based on this, a dominant frequency band is identified as a subband in which the norm coefficient value in the spectrum has a maximum value, and each of the identified dominant frequency bands is output to the dominant group determining units 302-1 to 302N. As a method of determining a dominant frequency band, other than determining a frequency band in which the norm coefficient value has a maximum value, for example, a band in which the norm coefficient value is the maximum value among all the subbands is determined as a dominant frequency band, or a band in which the norm coefficient value exceeds a predetermined threshold value or a threshold value calculated from the norm of all the subbands is determined as a dominant frequency band.

[0061] The superior group determining units 302-1 to 302N adaptively determine a group width according to input signal characteristics, centering on the superior frequency band output from the superior frequency band identifying unit 301. Specifically, the group width is set to the point where the downward gradient of the norm coefficient values ​​on both sides of the superior frequency band stops. The superior group determining units 302-1 to 302N determine the frequency bands included in the group width as superior groups, and output the determined superior groups to the non-superior group determining unit 303. Note that when the superior frequency band is at an edge (end of the available frequency), only one side of the downward gradient is included in the group.

[0062] The non-superior group determining unit 303 determines the consecutive subbands other than the superior groups output from the superior group determining units 302-1 to 302N as non-superior groups having no superior frequency band. The non-superior group determining unit 303 outputs the superior groups and the non-superior groups to the group energy calculating unit 304 and the norm variance calculating unit 306.

[0063] The group energy calculation unit 304 calculates the energy for each of the superior and non-superior groups output from the non-superior group determination unit 303, and outputs the calculated energy to the total energy calculation unit 305 and the group bit allocation unit 308. The energy for each group is calculated by the following equation (1).

number

[0064] Here, k is the group index, Energy(G(k)) is the energy of group k, i is the subband index of group 2, M is the total number of subbands in group k, and Norm(i) is the norm coefficient value of subband i in group n.

[0065] The total energy calculation unit 305 adds up all the energies for each group output from the group energy calculation unit 304, and calculates the total energy of all groups. The calculated total energy is output to the group bit allocation unit 308. The total energy is calculated by the following equation (2).

number

[0066] where Energytotal is the total energy of all groups, N is the total number of groups in the spectrum, k is the group index, and Energy(G(k)) is the energy of group k.

[0067] The norm variance calculation unit 306 calculates the norm variance for each group for the superior groups and non-superior groups output from the non-superior group determination unit 303, and outputs the calculated norm variance to the total norm variance calculation unit 307 and the group bit allocation unit 308. The norm variance for each group is calculated by the following equation (3).

number

[0068] Here, k is the index of the group, Normvar(G(k)) is the norm variance of group k, Normmax(G(k)) is the maximum norm coefficient value of group k, and Normmin(G(k)) is the minimum norm coefficient value of group k.

[0069] The total norm variance calculation unit 307 calculates the total norm variance of all groups based on the norm variance for each group output from the norm variance calculation unit 306. The calculated total norm variance is output to the group bit allocation unit 308. The total norm variance is calculated by the following equation (4).

number

[0070] where Normvartotal is the total norm variance of all groups, N is the total number of groups in the spectrum, k is the group index, and Normvar(G(k)) denotes the norm variance of group k.

[0071] The group bit allocation unit 308 (corresponding to a first bit allocation means) allocates bits for each group based on the energy for each group output from the group energy calculation unit 304, the total energy of all groups output from the total energy calculation unit 305, the norm variance for each group output from the norm variance calculation unit 306, and the total norm variance of all groups output from the total norm variance calculation unit 307, and outputs the bits allocated for each group to the subband bit allocation unit 309. The bits allocated for each group are calculated by the following equation (5).

number

[0072] where k is the group index, Bits(G(k)) is the number of bits allocated to group k, Bitstotal is the total number of bits available, scale1 is the proportion of bits allocated by energy, Energy(G(k)) is the energy of group k, Energytotal is the total energy of all groups, and Normvar(G(k)) is the norm variance of group k.

[0073] In addition, in the above formula (5), scale1 takes a value in the range of [0,1] and adjusts the proportion of bits allocated by energy or norm variance. The larger the value of scale1, the more bits are allocated by energy, and in the extreme case, when its value is 1, all bits are allocated by energy. The smaller the value of scale1, the more bits are allocated by norm variance, and in the extreme case, when its value is 0, all bits are allocated by norm variance.

[0074] By the group bit allocation section 308 performing bit allocation for each group as described above, it is possible to allocate more bits to the superior group and fewer bits to the non-superior group.

[0075] In this way, the group bit allocation unit 308 determines the perceptual importance of the group based on the energy and norm variance, and can emphasize the superior group. Also, the norm variance is consistent with the masking theory, and by using this, the perceptual importance can be determined more accurately.

[0076] The subband bit allocation unit 309 (corresponding to a second bit allocation means) allocates bits to the subbands in each group based on the bits for each group output from the group bit allocation unit 308, and outputs the bits allocated to the subbands for each group as a bit allocation result to the lattice vector coding unit 108. Here, more bits are allocated to subbands that are perceptually more important, and fewer bits are allocated to subbands that are perceptually less important. The bits allocated to each subband in a group are calculated by the following equation (6).

number

[0077] Here, BitsG(k)sb(i) is the bits assigned to subband i in group k, i is the subband index of group k, Bits(G(k)) is the bits assigned to group k, Energy(G(k)) is the energy of group k, and Norm(i) is the norm coefficient value of subband i in group k.

[0078] Next, the grouping method will be described with reference to Fig. 8. It is assumed that a quantized spectral envelope as shown in Fig. 8(a) is input to the peak frequency band identification unit 301. The peak frequency band identification unit 301 identifies the dominant frequency bands 9 and 20 based on the input quantized spectral envelope (see Fig. 8(b)).

[0079] The prominent group generating units 302-1 to 302-N determine that the bands that are in the same prominent group are those up to the point where the downward gradient of the norm coefficient values ​​stops on both sides of the prominent frequency bands 9 and 20. In the example of Fig. 8, for the prominent frequency band 9, subbands 6 to 12 are determined as the prominent group (group 2), and for the prominent frequency band 20, subbands 17 to 22 are determined as the prominent group (group 4) (see Fig. 8(c)).

[0080] The non-prominent group determination unit 303 determines the consecutive frequency bands other than the prominent group as a non-prominent group having no prominent frequency band. In the example of Fig. 8, subbands 1 to 5 (group 1), subbands 13 to 16 (group 3), and subbands 23 to 25 (group 5) are each determined as a non-prominent group (see Fig. 8(c)).

[0081] As a result, the quantized spectral envelopes are grouped into five groups: two dominant groups (groups 2, 4) and three non-dominant groups (groups 1, 3, 5).

[0082] This grouping method allows the group width to be adaptively determined according to the input signal characteristics, and since the method uses quantized norm coefficients that can also be used by the speech / audio decoding device, there is no need to transmit additional information to the speech / audio decoding device.

[0083] The norm variance for each group is calculated in the norm variance calculation unit 306. For reference, the norm variance Energyvar(G(2)) in group 2 in the example of FIG.

[0084] Next, perceptual importance will be described. In general, there are multiple peaks and valleys in the spectrum of a speech audio signal. A peak is composed of spectral components (prominent sound components) located at the dominant frequencies of a speech audio signal. A peak is very perceptually important. The perceptual importance of a peak can be determined by the difference between the energy of the peak and the energy of the valley, i.e., the norm variance. Theoretically, if a peak has a sufficiently large energy compared to adjacent frequency bands, the peak should be coded with a sufficient number of bits, and if it is coded with an insufficient number of bits, the mixed coding noise becomes prominent and the sound quality deteriorates. On the other hand, a valley is not composed of a dominant sound component of a speech audio signal and is not perceptually important.

[0085] In the frequency band grouping method of this embodiment, the dominant frequency bands correspond to spectral peaks, and grouping the frequency bands separates peaks (dominant groups with dominant frequency bands) from valleys (non-dominant groups without dominant frequency bands).

[0086] The group bit allocation unit 308 determines the perceptual importance of the peaks. In a technique such as G.719, the perceptual importance is determined only by the energy, whereas in this embodiment, the perceptual importance is determined by both the energy and the norm (energy) variance, and the bits to be allocated to each group are determined based on the determined perceptual importance.

[0087] In addition, in the subband bit allocation unit 309, if the norm variance within a group is large, it means that this group is one of the peaks, and the peaks are more perceptually important, and the norm coefficient with the largest value should be coded accurately. Therefore, more bits are allocated to the subband of this peak. On the other hand, if the norm variance within a group is very small, it means that this group is one of the valleys, and the valleys are not perceptually important, and do not need to be coded as accurately. Therefore, fewer bits are allocated to each subband of this group.

[0088] In this manner, according to the present embodiment, a dominant frequency band in which the norm coefficient value in the spectrum of the input speech audio signal has a maximum value is identified, all subbands are grouped into a dominant group including the dominant frequency band and a non-dominant group not including the dominant frequency band, bits are allocated to each group based on the energy and norm variance of each group, and the bits allocated to each group are further allocated to each subband according to the ratio of the norm to the energy of the group. This allows more bits to be allocated to perceptually important groups and subbands, and allows efficient bit allocation. As a result, sound quality can be improved.

[0089] In addition, the norm coefficient in this embodiment represents subband energy, and is also called an energy envelope.

[0090] The disclosures of the specification, drawings and abstract contained in Japanese Patent Application No. 2012-272571, filed on December 13, 2012, are incorporated herein by reference in their entirety. [Industrial Applicability]

[0091] The speech and audio encoding device, speech and audio decoding device, speech and audio encoding method, and speech and audio decoding method of the present invention can be applied to wireless communication terminal devices, wireless communication base station devices, telephone conference terminal devices, video conference terminal devices, and Voice over Internet Protocol (VoIP) terminal devices, etc. [Explanation of symbols]

[0092] 101 Transient Detector 102 Conversion unit 103 Norm Estimation Unit 104 Norm quantization unit 105 Spectral Normalization Unit 106, 203 Norm Adjustment Unit 107, 204 bit allocation section 108 Lattice Vector Encoding Unit 109 Noise level adjustment section 110 Multiplexer 201 Demultiplexer 202 Norm Inverse Quantization Unit 205 Lattice Decoding Unit 206 Spectral Filter Generator 207 Adder 208 Envelope forming section 209 Inverse conversion unit 301 Dominant Frequency Band Identification Unit 302-1~302-N Excellent Group Decision Department 303 Non-Excellent Group Decision Section 304 Group Energy Calculation Department 305 Total Energy Calculation Unit 306 Norm Variance Calculation Unit 307 Total Norm Variance Calculation Unit 308 Group Bit Allocation Unit 309 Subband bit allocation unit

Claims

1. A transform means (102) for transforming an input signal from the time domain to the frequency domain; an estimation means (103) for estimating an energy envelope representing an energy level for each of a plurality of sub-bands into which the frequency spectrum of the input signal is divided; quantization means (104) for quantizing the energy envelope; A group determination means (302-1, 302-N, 303) for grouping the quantized energy envelope into a plurality of groups; a first bit allocation means (308) for allocating bits to said plurality of groups; A second bit allocation means (309) for allocating the bits allocated to the plurality of groups to a subband for each group; encoding means (108) for encoding the frequency spectrum using the bits allocated to the subbands; a dominant frequency band identifying means (301) for identifying a dominant frequency band, which is a subband in which the energy envelope of the frequency spectrum has a maximum value; A speech or audio coding device comprising: The group determination means (302-1, 302-N, 303) A speech or audio coding device that determines the dominant frequency band and subbands on either side of the dominant frequency band, each of which forms a downward gradient of the energy envelope, into a plurality of dominant groups, and determines consecutive subbands other than the dominant frequency band into a plurality of non-dominant groups.

2. An energy calculation means (304) for calculating energy for each group; A variance calculation means (306) for calculating an energy envelope variance for each group; Further comprising: The first bit allocation means (308) based on the calculated energy for each group and the calculated energy envelope variance for each group, the larger at least one of the energy and the energy envelope variance is, the more bits are allocated to the group, and the smaller at least one of the energy and the energy envelope variance is, the fewer bits are allocated to the group; 2. A speech or audio coding device according to claim 1.

3. The second bit allocation means (309) The larger the energy envelope of the subband, the more bits are allocated to the subband, and the smaller the energy envelope of the subband, the fewer bits are allocated to the subband.

2. A speech or audio coding device according to claim 1.

4. 2. A speech or audio coding device according to claim 1, wherein the second bit allocation means (309) is arranged to allocate more bits to subbands of greater perceptual importance and less bits to subbands of lesser perceptual importance.

5. 2. The speech or audio coding device of claim 1, wherein the second bit allocation means (309) is adapted to allocate more bits to subbands in groups with high energy variance and less bits to subbands in groups with low energy variance.

6. 2. The speech or audio coding device of claim 1, wherein the second bit allocation means (309) is configured to allocate more bits to subbands in groups having peaks in the frequency spectrum and to allocate fewer bits to subbands in groups having valleys in the frequency spectrum.

7. The second bit allocation means (309) is The calculation is configured based on Here, Bits G(k)sb(i)denotes the bits allocated to subband i in group k, i denotes the subband index of group k, Bits(G(k)) denotes the bits allocated to group k, Energy(G(k)) denotes the energy of group k, and Norm(i) denotes the subband energy value of subband i in group k.

2. A speech or audio coding device according to claim 1.

8. 2. The speech or audio coding device of claim 1, wherein the first bit allocation means (308) is arranged to allocate more bits to prominent groups of the plurality of prominent groups and to allocate less bits to non-prominent groups of the plurality of non-prominent groups.

9. 2. The speech or audio coding device of claim 1, wherein the first bit allocation means (308) is configured to allocate bits per group based on an energy per group, a total energy of all groups, an energy variance per group and a total energy variance of all groups.

10. The first bit allocation means (308) is The calculation is configured based on Here, k denotes the index of each group, Bits(G(k)) denotes the number of bits assigned to group k, and Bits total denotes the total number of available bits, scale1 denotes the ratio of bits allocated per energy, Energy(G(k)) denotes the energy of group k, and Energy total denotes the total energy of all groups, and Normvar(G(k)) denotes the energy variance of group k.

2. A speech or audio coding device according to claim 1.

11. 11. A speech or audio coding device as claimed in claim 10, wherein the value of scale1 is between 0 and 1.

12. 2. The speech or audio coding device of claim 1, wherein the first bit allocation means (308) is arranged to determine a perceptual importance of each group using the energy and energy variance of the group and to highlight a prominent group among the plurality of prominent groups.

13. 2. The speech or audio coding device of claim 1, wherein the first bit allocation means (308) is arranged to determine perceptual importance of groups based on their energies and energy variances and to determine bits allocated to each group based on the perceptual importance of the respective group.

14. 2. The speech or audio coding device according to claim 1, wherein said group determining means (302-1, 302-N, 303) is configured to adaptively determine a group width of said plurality of groups in accordance with a characteristic of said input signal.

15. 2. The speech or audio coding device of claim 1, wherein said group determining means (302-1, 302-N, 303) are arranged to use quantized subband energies.

16. 2. The speech or audio coding device of claim 1, wherein the group determination means (302-1, 302-N, 303) is configured to separate the frequency spectrum peaks from the frequency spectrum valleys, the frequency spectrum peaks being located within prominent groups of the plurality of prominent groups and the frequency spectrum valleys being located within non-prominent groups of the plurality of non-prominent groups.

17. A transform means (102) for transforming an input signal from the time domain to the frequency domain; an estimation means (103) for estimating an energy envelope representing an energy level for each of a plurality of sub-bands into which the frequency spectrum of the input signal is divided; quantization means (104) for quantizing the energy envelope; A group determination means (302-1, 302-N, 303) for grouping the quantized energy envelope into a plurality of groups; a first bit allocation means (308) for allocating bits to said plurality of groups; A second bit allocation means (309) for allocating the bits allocated to the plurality of groups to a subband for each group; encoding means (108) for encoding the frequency spectrum using the bits allocated to the subbands; A speech or audio coding device comprising: The group determination means (302-1, 302-N, 303) is configured to identify a dominant frequency band in which the energy value of the subband in the frequency spectrum of the input signal has a maximum value, and to group the subbands including the dominant frequency band into a plurality of dominant groups and group the other subbands into a plurality of non-dominant groups; the first bit allocation means (308) is configured to allocate bits to each of the groups based on an energy of each group and an energy variance of each of the groups; The second bit allocation means (309) is configured to allocate the bits allocated to each of the groups to each of the subbands in each of the groups in accordance with a ratio of the energy of each of the subbands to the energy of each of the groups.

18. an inverse quantization means (202) for inverse quantizing the quantized spectral envelope; group determining means (302-1, 302-N, 303) for grouping the quantized spectral envelope into a plurality of groups; a first bit allocation means (308) for allocating bits to said plurality of groups; A second bit allocation means (309) for allocating the bits allocated to the plurality of groups to a subband for each group; a decoding means (205) for decoding a frequency spectrum of a speech or audio signal using the bits allocated to the subbands; an envelope shaping means (208) for applying the dequantized spectral envelope to the decoded frequency spectrum to reproduce a decoded spectrum; inverse transform means (209) for inverse transforming the decoded spectrum from the frequency domain to the time domain; A dominant frequency band identifying means (301) for identifying a dominant frequency band, which is a subband in which the spectral envelope of the frequency spectrum has a maximum value; A speech or audio decoding device comprising: The group determination means (302-1, 302-N, 303) A speech or audio decoding device that determines the dominant frequency band and subbands on either side of the dominant frequency band, each of which forms a downward gradient of the spectral envelope, into a plurality of dominant groups, and determines consecutive subbands other than the dominant frequency band into a plurality of non-dominant groups.

19. An energy calculation means (304) for calculating energy for each group; A variance calculation means (306) for calculating a spectral envelope variance for each group; Further comprising: The first bit allocation means (308) Based on the calculated energy for each group and the spectral envelope variance for each group, the larger at least one of the energy and the spectral envelope variance is, the more bits are allocated to the group, and the smaller at least one of the energy and the spectral envelope variance is, the fewer bits are allocated to the group.

20. A speech or audio decoding device according to claim 18.

20. The second bit allocation means (309) Allocating more bits to subbands with larger spectral envelopes and less bits to subbands with smaller spectral envelopes.

20. A speech or audio decoding device according to claim 18.

21. Transform the input signal from the time domain to the frequency domain, estimating an energy envelope representing an energy level for each of a plurality of sub-bands into which the frequency spectrum of the input signal is divided; quantizing the energy envelope; grouping the quantized energy envelopes into a plurality of groups; assigning bits to said plurality of groups; allocating the bits allocated to the plurality of groups to subbands for each group; encoding the frequency spectrum using the bits allocated to the subbands; identifying dominant frequency bands, which are subbands in which the energy envelope of the frequency spectrum has a local maximum; A method for encoding speech or audio, comprising: The grouping comprises determining the dominant frequency band and subbands on either side of the dominant frequency band, each of which forms a downward gradient of the energy envelope, into a plurality of dominant groups, and determining consecutive subbands other than the dominant frequency band into a plurality of non-dominant groups.

22. Transform the input signal from the time domain to the frequency domain, estimating an energy envelope representing an energy level for each of a plurality of sub-bands into which the frequency spectrum of the input signal is divided; quantizing the energy envelope; grouping the quantized energy envelopes into a plurality of groups; performing a first allocation of bits to the plurality of groups; performing a second allocation of the bits allocated to the plurality of groups to a subband for each group; encoding the frequency spectrum using bits allocated to the subbands; A method for encoding speech or audio, comprising: The grouping includes identifying a dominant frequency band in the frequency spectrum of the input signal, the dominant frequency band having a maximum energy value, and grouping the subbands including the dominant frequency band into a plurality of dominant groups and grouping the other subbands into a plurality of non-dominant groups; the first allocation includes allocating bits to each group based on an energy of each group and an energy variance of each group; The speech or audio coding method, wherein the second allocation includes allocating the bits allocated to each group to each of the subbands in each of the groups in accordance with a ratio of an energy of each subband to an energy of each of the groups.

23. Dequantizing the quantized spectral envelope; grouping the quantized spectral envelopes into a plurality of groups; assigning bits to said plurality of groups; allocating the bits allocated to the plurality of groups to subbands for each group; Decoding a frequency spectrum of a speech or audio signal using the bits allocated to the subbands; applying the dequantized spectral envelope to the decoded frequency spectrum to reconstruct a decoded spectrum; Inverse transforming the decoded spectrum from the frequency domain to the time domain; identifying dominant frequency bands, which are subbands in which the spectral envelope of the frequency spectrum has a local maximum; A method for decoding speech or audio, comprising: The grouping includes determining the dominant frequency band and subbands on either side of the dominant frequency band, each subband forming a downward gradient of the spectral envelope, into a plurality of dominant groups, and determining consecutive subbands other than the dominant frequency band into a plurality of non-dominant groups.

24. A wireless communication terminal device, a wireless communication base station device, a telephone conference terminal device, a video conference terminal device, or a Voice over Internet Protocol (VoIP) terminal device, comprising the speech or audio encoding device according to claim 1 or claim 17, or the speech or audio decoding device according to claim 18.

Citation Information

Patent Citations

  • Bit allocation for sub band voice coder

    JP1988058500A

  • Systems, methods, apparatus, and computer-readable media for dynamic bit allocation

    WO2012016126A2