Audio encoding with distortion weight conversion

Abstract

Encoding an audio signal comprises: transforming the audio signal using a frame-based frequency transform; applying an analysis filterbank to the audio signal, the filterbank domain having B a number of frequency bands; defining a set B of weight functions, wherein each weight function is related to a perceptually-based distortion of the audio signal in one frequency band of the filterbank domain; converting the set B of weight functions to a set of bin weights in the frame-based transform domain; using the bin weights υ α to compute quantization steps, the quantization steps being configured to allocate an average coding bit rate over time and frequency; and quantizing a transform domain representation of the audio signal using the quantization steps. With this approach, a perceptually-based distortion function is defined for audio coding in the frame-based transform domain via the weighting defined in the analysis filterbank domain.

Description

Cross-reference to related applications

[0001] This application claims priority to U.S. Provisional Application No. 63 / 638,198, filed April 24, 2024, and European Patent Application No. 24183208.8, filed June 19, 2024, each of which is incorporated herein by reference in its entirety. Technical Field

[0002] This invention relates to audio coding, and more particularly to coding that scales the quantization of individual frames based on weighted distortion. Background Technology

[0003] Today, most audio coding involves, for example, using a modified discrete cosine transform (MDCT) to perform a frequency transformation to obtain a transform domain representation. This transform representation typically includes one (real or complex) coefficient for each time-frequency bin, and in the next stage, these coefficients are quantized to reduce the required bit rate.

[0004] It is known that perception-driven control is applied to the noise shape in time and frequency caused by the coding scheme (e.g., quantization). For example, the desired signal-to-noise ratio (SNR) for each frequency band of the codec's improved discrete cosine transform (MDCT) domain can be defined using the model discussed in WO 2021 / 113416.

[0005] For more advanced perceptual modeling, the gammatone filter bank domain can be used, and a coding scheme based on distortion analysis in the gammatone domain was proposed in the paper "A new perceptual model for audio coding based in spectro-temporal masking" published by van dePar et al. in the Audio Engineering Society in 2008. Summary of the Invention

[0006] The method developed by van de Par et al. may be difficult to implement, and therefore it is desirable to provide an improved method to perform perceptual modeling in a domain and apply such modeling to the codec transform domain to provide improved control over noise shape (e.g., quantization distortion).

[0007] The purpose of this disclosure is to provide such an improved method. The aspects of this method are defined by the appended claims.

[0008] As discussed above, the encoder quantizes the transform coefficients, thereby introducing distortion in the transform domain (e.g., the MDCT domain). According to various aspects of the invention, the encoder is configured to control this distortion in units of weighted distortion defined in the analysis filter bank domain (e.g., the gamma pass domain). This weighted distortion is converted back to a weighted average distortion in the transform domain.

[0009] According to one aspect of the invention, the above objective is achieved by using a method for a duration of I The method for encoding an audio signal s(t) into an encoded audio signal includes: transforming the audio signal using a frame-based frequency transform to obtain a transform domain representation of the audio signal; and applying an analytical filter bank to the audio signal to obtain a filter bank domain representation of the audio signal, wherein the filter bank domain has... B One frequency band; define a set B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b Perception-based distortion correlation in the group; B Weight functions w b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; the bin weight υ is used. α To calculate the quantization step size Δ J These quantization steps are configured to distribute the total number of bits in time and frequency. R ; and use these quantization steps to quantize the transform domain representation of the audio signal.

[0010] According to another aspect, the above objective is achieved through a method for setting the duration as I The audio signal s(t) is encoded into an encoder that encodes the audio signal. The encoder includes: a transform block for transforming the audio signal using a frame-based frequency transform to obtain a transform domain representation of the audio signal; and a filter bank application block for applying an analytical filter bank to the audio signal to obtain a filter bank domain representation of the audio signal, wherein the filter bank domain has... B One frequency band; weight determination block, used to define a set of B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b The perceptual-based distortion correlation in the data; weight transformation block, used to transform the group B Weight functionsw b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; scaling blocks are used to scale these bin weights. α To calculate the quantization step size Δ J These step sizes are configured to distribute the total number of bits in time and frequency. R ; and quantization blocks, used to utilize these quantization steps Δ J The transform domain representation of the audio signal is quantized.

[0011] Based on these aspects, a perceptually driven distortion function is defined for audio coding in the frame-based transform domain via weights defined in the analytical filter bank domain. In other words, a perceptually driven distortion function is defined by a weight function in the analytical filter bank domain for audio coding in the frame-based transform domain (“codec domain”). This is achieved by efficiently transforming the weights in the analytical domain to the transform domain of the audio encoder. The transformed weights can then be used to achieve a given bit budget (or average bit rate) on the considered signal segment.

[0012] Weights (i.e., the distortion function) are associated with the "perception-based" distortion of the audio signal. "Perception-based" refers to the distortion perceived by the (typical) listener. Frequency bands where distortion is strongly perceived will be associated with larger weights. Perception-based distortion can be determined using some model of the human auditory system, for example, by calibration based on the results of psychoacoustic experiments.

[0013] Compared to audio coding based on transform domain-aware noise shaping, the benefits observed in this disclosure are the smoothness of the resulting noise shaping, better transient signal processing, and improved speech quality.

[0014] Weighted distortion is defined on the complete signal, that is, on a signal time segment much longer than the frame length (block size, transform length) operated on by frame-based frequency transformation. This allows for an unequal distribution of contributions in time and frequency. By applying appropriate scaling according to this disclosure, rate optimization results in a more uniform distribution of weighted distortion, thereby achieving controllable noise shaping.

[0015] "Frame-based" means that the encoding transform is performed one sample "frame" (block, window) at a time. The number of (frequency) samples in the transform domain representation of a frame can be less than the number of (time) samples in that frame. For example, the MDCT transform reduces the number of samples by half (2N time samples produce N frequency samples). This can be compensated for by using a time step ("stride") shorter than the transform length. For example, the stride can be 50% of the frame. This is called an "overlapping" transform.

[0016] Encoding in this manner allows for satisfactory control of the trade-off between distortion and the total number of bits (or the average bit rate over the signal duration). For example, given a constraint on weighted distortion based on bin weights, the total number of bits can be controlled (e.g., minimized). Alternatively, given a maximum available average bit rate, weighted distortion can be controlled (e.g., minimized). Other satisfactory combinations of distortion and bit number are also conceivable.

[0017] The analytical filter bank domain does not introduce artificial framing into distortion metrics, thus benefiting distortion analysis. Furthermore, analytical filter banks typically have significantly higher temporal resolution, allowing for a physiologically plausible definition of distortion.

[0018] In some implementations, each weighting function is equal to a constant divided by a masking function, where the masking function represents a level at which distortion is masked by the audio signal so that the distortion is imperceptible to a human listener. The sum of all constants is preferably equal to the number of frequency bands.

[0019] In the simplest case, the constant equals one, but alternatively, the constant can be different for different frequency bands. Such a set of constants can provide calibration of the distortion weights to achieve the desired smooth noise shaping.

[0020] In some applications, based on the frequency resolution of the human ear, the transform domain is grouped into a set of "scaling factor" bands. Then, all bins belonging to the same scaling factor band can be assigned an equal quantization step size. In this case, it can be assumed that the variance of the quantization error of bins in a particular bin group with the same quantization step size is proportional to the inverse of the bin weights in that group, which is equal to the average of the bin weights in that group. This allows a simplified distortion-targeted transformation to be applied to the transform domain of the codec, where each specific step size is set to be proportional to the inverse of the group weights. Attached Figure Description

[0021] The invention will be described in more detail with reference to the accompanying drawings, which illustrate the presently preferred embodiments of the invention.

[0022] Figure 1 This is a schematic block diagram of an encoder system according to an embodiment of the present invention.

[0023] Figure 2 This is a flowchart of a method according to an embodiment of the present invention.

[0024] Figure 3 It is the spectral representation of the impulse response of the gamma-pass filter bank.

[0025] Figure 4 The time-frequency representation of the signal in the gamma-pass filter bank domain is shown.

[0026] Figure 5 The time-frequency representation of the signal in the MDCT domain is shown.

[0027] Figure 6 It shows the use of Figure 2 The graph shows the test results comparing the signal encoded using the method in this paper with the signal encoded using traditional coding.

[0028] Figure 7 The illustration shows a schematic block diagram of an example device or architecture that can be used to implement embodiments of the present invention. Detailed Implementation

[0029] Figure 1 A variable bit rate (VBR) encoder 10 according to an exemplary embodiment of the present invention is illustrated schematically.

[0030] The upper portion of the block diagram includes a transform block 11 that transforms the input audio signal 12 into a frame-based transform domain representation 13 of that signal. The duration of the input audio signal 12 is... I This duration can be on the order of seconds. Most importantly, this duration is significantly longer than (for example, by an order of magnitude) the frame length operated on by the transform block.

[0031] Transform block 11 operates on one frame of the input signal 12 at a time, and each frame is transformed into a set of frequency bins. Each bin (also called a time-frequency bin) has a coefficient (value) c. α And with specific waveforms in the transform domain filter bank. u α Related. Frames often overlap, so that the next frame includes a portion of the previous frame. The amount of advancement of a frame relative to the previous frame is sometimes called the "stride," and can be 50% of the frame length. The frame length (also called window length or block size) can be variable and depends on the dynamics of the signal. Figure 1 In the encoder, the frame length is controlled by the transformation control block 14.

[0032] In the example shown, transform block 11 implements a modified discrete cosine transform (MDCT). The frame length (block size, window length) of the MDCT transform is typically 768 or 1024 samples, and the stride is typically 50%, i.e., 384 or 512 samples. A frame with 768 time samples is transformed into 384 blocks, while a frame with 1024 time samples is transformed into 512 blocks. The reduced number of samples is compensated for by a 50% stride, ensuring that the total number of frequency samples is the same as the number of time samples. This is called an "overlapping" transform.

[0033] Transformer 11 is followed by quantization block 15, which is configured to match the binning coefficient c. α Quantification is performed. It is this quantification that introduces the distortion that this disclosure aims to control.

[0034] The lower part of the block diagram includes the analysis filter bank application block 16. Unlike the transform applied in transform block 11 (which operates on one signal frame at a time (as discussed above for the MDCT case)), the analysis filter bank is not frame-based; that is, it operates on the entire input signal, processing one sample at a time. The analysis filter bank includes a set of frequency transforms, each corresponding to a specific frequency band. Related. In the example shown, the analysis filter bank is a gamma-pass filter bank, which will be discussed in more detail below.

[0035] The output of the filter bank analysis is used in subsequent weight definition block 17 to determine a set of weight functions or curves. w b (t) Each frequency band of the analysis domain b Each corresponds to a weight.

[0036] Weights are provided to transformation block 18, which is configured to convert the weights w b (t) Convert to a set of warehouse weights v α Each cell in the transform domain corresponds to a cell weight. The transformation receives information from transform control block 14 about the transform length (frame length) applied in transform block 11.

[0037] Finally, the lower branch includes scaling block 19, which is configured to determine the scaling factor c for each bin. α The appropriate step size for quantization is determined. Scaling in block 19 is controlled by application-specific parameters such as target number of bits or maximum available number of bits (average bit rate), maximum acceptable distortion, etc. Scaling block 19 is concatenated to provide the calculated step size to quantization block 15.

[0038] In one embodiment, scaling block 19 is configured to compute the quantization step size by controlling (e.g., minimizing) the average coding bit rate, given a predefined constraint on a weighted distortion metric based on bin weights. In another embodiment, scaling block is configured to compute the quantization step size by controlling (e.g., minimizing) a weighted distortion metric based on bin weights, given a maximum number of available bits.

[0039] Figure 2 This illustrates the use of variable coding bit rate for duration. I audio signal s(t) Example encoding methods for performing encoding.

[0040] In step S11 (“Transforming Audio”), the audio signal is transformed using a frame-based frequency transform to obtain a transform domain representation of the audio signal. This transform includes a set of modules (also called time-frequency modules), where each module has a coefficient (value) c. α And with a specific waveform u in the transform domain filter bank α Related.

[0041] In step S12 ('Apply a filter bank to the audio'), the audio signal is applied with its respective frequency band. b Related B The filter bank is analyzed to obtain a filter bank domain representation of the audio signal. It should be noted that steps S11 and S12 are independent of each other and can be performed in any order.

[0042] In step S13 ('Define weight function'), a set of weight functions is defined. B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b Perception-based distortion is relevant in this context. Human auditory system models can be used to assess or estimate perception-based distortion. Specifically, as will be discussed below, weights can be determined based on masking levels, which represent a level at which distortion is masked by the original signal and therefore imperceptible.

[0043] In step S14 ('Transform the weights'), the set of weight functions is... w b (t) Transform into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin based on the frame-based transform domain.

[0044] In step S15 ('Calculate quantization step size'), the scaling block 15 uses bin weights to calculate the quantization step size Δ, which is configured to distribute the total number of available bits in time and frequency. R Or average bit rate. In many practical implementations, the frequency lines in the transform domain are grouped into bands (scaling factor bands) based on the frequency resolution of the human ear, and a single step size (quantization level) is applied to all bins belonging to the same scaling factor band group. In the following text, group... J The common step size shared by all warehouses is called Δ. J .

[0045] Finally, in step S16 ('quantize the coefficients'), the quantization step size Δ is used. J Quantize the transform domain representation.

[0046] The following sections will discuss additional details of the possible encoding processes performed in encoder 10.

[0047] Distortion definition Consider a set of forms of 4th-order gamma-pass filter with impulse response g b For each center frequency , for parameters and Adjustments are made to achieve an equivalent rectangular bandwidth (ERB) and unit amplitude response at that frequency. A set of center frequencies can be selected at 0.5 ERB intervals, thus covering a frequency range from 20 Hz to the coding bandwidth. Figure 3 The text provides the case where the coding bandwidth is 6750 Hz. An example of the resulting filter frequency response.

[0048] Encoded Distortion Signal It can be defined as the difference between the original audio signal and the encoded version of the same audio signal. Therefore, in the case of a length of... Duration of time Within the gamma-connected domain, the weighted distortion is defined by the following equation:

[0049] Each gamma passband Weights in It is a non-negative function of time. For ease of description, a continuous-time variable is used. It should be understood that, in implementation, integration must be replaced by summation over time samples of the discrete-time signal. Throughout this disclosure, the symbol '...' is used. ' is used to represent temporal convolution.

[0050] Weight Based on a perceptual model, distortion contributions that are more easily perceived by the listener should have higher weights. These weights can be determined using specific signal masking levels. m b (t) This determines that the masking level represents a level at which distortion will be masked by the original signal. In other words, if signal distortion contributes... Less than Therefore, it can be expected that the signal distortion contribution will be masked. The masking level will depend on the quiet hearing threshold, the original signal, and the human ear perception model. Thus, an appropriate distortion weight definition can be... Therefore, for the masked distortion, in equation (1) .

[0051] In one embodiment, the starting point can be the original input signal. Analysis was conducted to obtain the following incentive pattern.

[0052] in, It is the impulse response of a low-pass filter, and In the gamma passband The linear value of the quiet threshold at the center frequency. Then, a perceptually driven masking model (such as the model discussed in the international publication WO2021 / 113416 entitled "A Psychoacoustic Model for Audio Processing," which is incorporated herein by reference in its entirety) can be used to mask the excitation mode. Convert to a masking level with the above characteristics. .

[0053] Equation (2) uses The linear filtering performed can be replaced by... This involves a nonlinear smoothing operation designed to align the model's time-aftermasking characteristics with experimental data. An example of this is the use of an envelope generation tool based on two distinct first-order filters with different attack and decay time constants, where the decay time constant depends on the gamma-ton filter index. .

[0054] Alignment Correction By using equation (1) Replace with a smoothed version of that amount (the smoothing is consistent with the original). Generate incentive patterns By performing similar actions, improvements can be achieved in modeling temporal masking aspects.

[0055] In the smooth version by In the case of a defined linear envelope, a simple method taught in this disclosure for performing such alignment correction is time reversal via a smoothing filter. Distortion weights Perform filtering while keeping equation (1) unchanged.

[0056] In the case of a nonlinear envelope, the corresponding modification of the distortion weights involves the transpose of the envelope operator gradient, but a simpler alternative is to use the time reversal of a first-order filter defined by the attack time constant.

[0057] Weight conversion Since the transform control block 14 provides the applied frame length to the weight transform block 18, therefore within the considered time interval... The MDCT domain is given above. In other words, the allocation of the time-frequency compartment α is known.

[0058] Therefore, the distorted signal can be written as ,in, It is for a given warehouse The synthesized waveform of MDCT, and This refers to the quantization error of the warehouse. Based on the standard assumption that quantization errors are uncorrelated and have a zero average, a weighted distortion estimate can be derived.

[0059] in, It is the variance of the quantization error, and The MDCT domain weights are defined as follows:

[0060] In other words, the weight of each warehouse is υ α It is formed as the sum of frequency band terms across all filter bank domain frequency bands, where each frequency band term includes a duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor and a weighting function for that frequency band. w b (t) The pre-calculated factor includes the analysis filter associated with the frequency band and the time convolution between the transformed waveform associated with the bin weights.

[0061] By observation The computational complexity is reduced by depending only on the analysis filter bank (here, the gamma-pass filter bank) and the transform waveform (here, the MDCT waveform). Therefore, the convolution can be approximated with high numerical accuracy by time-shifting a pre-computed waveform table whose length depends on the transform size and the attenuation of the gamma-pass filter. Furthermore, the summation of the integrals that implement equation (4) can be downsampled when the weights change slowly enough that interpolated versions of these weights provide a good approximation.

[0062] Applications in coding The main free variable in MDCT field encoding is the quantization step size. Variance of quantization error It can be based on the commonly used methods in scalar quantization Estimating using rules. Adapting to the index. When using a companding quantizer, you can use

[0063] here, The original signal is in the warehouse. The MDCT (unquantized) coefficients at that location. Therefore, the coefficients from dimension [dimension number missing]. And share the same step length Cangzi Collection The weighted distortion estimate is

[0064] Estimates of the corresponding bit cost can be obtained in several ways. For example, it is observed that the Huffman codebook of an audio codec is essentially derived from a Laplace distribution model, and codebook search can be modeled by optimizing the Laplace scaling parameter, thus yielding the following approximation of the bit cost.

[0065] Rate-distortion optimization Total distortion obtained Total bit cost The estimate is obtained by considering all (step-size) subsets. of and This is obtained by summing. By ignoring details such as the difference between Huffman-coded segments and scaling factor bands, and ignoring the coding cost of the step size, gradient descent can be used to achieve a given upper limit on distortion. In situations where control (e.g., minimizing) the MDCT coding average bit rate estimate is desired. .

[0066] Figures 4 to 6The performance of this method is illustrated in the figure.

[0067] Figure 4 A conventional method is illustrated, in which, in box 42, the audio signal 41 is transformed into the MDCT domain by band-wise MDCT analysis. The MDCT signal is represented by a time-frequency plot 43. This representation is used by box 44 to determine the appropriate step size for quantizing the coefficients in representation 43.

[0068] Figure 5 It shows according to Figure 2 The method described in the section describes the implementation of the method. Here, gamma-pass analysis is performed on the audio signal 41 in box 52. The output of the gamma-pass analysis is represented as a time-frequency plot 53. This output is used in box 54 to define the weighted distortion, which is then converted to the MDCT domain in box 55 and used to determine the step size in box 56.

[0069] Figure 6 The results of MUSHRA tests (11 listeners) for different categories of a 48 kb / s mono signal are shown. Figure 6 In the middle, series 61 involves according to Figure 4 The traditional encoding, while Series 62 involves according to Figure 5 The encoding.

[0070] Corollaries of High Bit Rate Theory and Simplified Step Size Control According to rate-distortion theory, it is well known that at the high bitrate limit, the average value of the weighted distortion across all coding dimensions is constant. This result can also be obtained directly by explicit minimization using the high bitrate versions of equations (5) and (7), where the minimum and maximum value operations are omitted, and only the terms involving the step size are retained. In this case, there exists a constant. , making

[0071] in, It is to share the same step length Cangzi Collection J Dimensions.

[0072] In equation (5), let ,for A constant can be obtained. Therefore, by making a choice and To obtain a simplified interface for step size control, the following proportions are used:

[0073] in, It is defined as a subset J The "group weight" is the average weight of all positions in the portfolio.

[0074] It can be verified that equation (10) is consistent with the uniform distribution of average distortion defined by equation (8). Now, the step size for each group will be determined by the following equation. .

[0075] In other words, it is now possible to choose a scaling constant. This simplifies step size control and rate distortion adjustment. Then, scaling block 19 can be configured to set each specific step size Δ based on group weights and a constant E. j .

[0076] In this case, the computational complexity of weight transformation can be further reduced because the weight transformation in equation (10) can be optimized. The summation is then transferred into equation (4), therefore, only the summation needs to be considered. hour A pre-calculated table of average values.

[0077]

[0078] Calibration for desired smooth noise shaping if by Let it be the boundary, then according to equation (1) The choice of distortion weights will result in distortion. In this case, for those that satisfy... In the case of any redistribution of weights in the gamma passband , That also holds true. This provides an option. This provides an opportunity for rate-distortion optimization based on weighted distortion to approximate the desired noise shaping, assuming both noise and weights are time-stationary. Two methods can be used to obtain the desired noise shaping. Calibration.

[0079] Heuristic method based on gamma-ton filter bank bandwidth In order to directly apply the high code rate theoretical prediction with optimized distortion allocation to the weighted distortion of equation (1), a method for estimating the distortion of each frequency band is needed. A heuristic method for encoding dimensions. This invention proposes using a gamma-pass filter. bandwidth Therefore, the result is that there exists a constant. , making

[0080] For constant weight and signal distortion contribution Expected average time ,available In accordance with the requirements Thus, we obtained

[0081] Calibration under simplified step size control When the step size is chosen to be inversely proportional to the mean of the MDCT domain distortion weights, as in equation (9-10), the stationary weights... Contribution to signal distortion Time average It can be estimated as

[0082] Among them, the transformation matrix elements from MDCT frequency band to gamma passband Defined as

[0083] In equation (13), for any desired noise shape accomplish It is difficult, but when transforming matrix elements With the center frequency of the gamma passband and the bin group A good approximation can be achieved when the difference decays sufficiently quickly. Therefore, for calibration purposes, it is sought to satisfy... The existence of a single example is worthwhile. In the case described below, this is obtained for a spectrally flat (white) noise shape.

[0084] If the sum of the squared amplitude frequency responses of the gamma-pass is constant, and the synthesized MDCT waveform is normalized to unit energy and covers the entire spectrum, then it can be concluded that... as well as These identities can be substituted into equation (13) and let... It can be proven that, with the following calibration constants selected...

[0085] Therefore, for the relationship with any proportional They all .

[0086] Under the assumption that the sum of the squared amplitude and frequency response of the gamma-tone is constant (e.g.) Figure 3 (The situation shown is roughly like this), the heuristic rule of equation (12) has good consistency with the calibration of equation (15).

[0087] Implementation details Figure 7 A schematic block diagram is shown of an example electronic device or architecture 200 (e.g., device 200) suitable for implementing exemplary embodiments of this disclosure. Architecture 200 may be part of a mobile device such as a smartphone 2, but may also be a standalone device. Architecture 200 may embody a combination of Figure 1 The system described, but not limited to.

[0088] As shown, architecture 200 includes a central processing unit (CPU) 201 capable of executing various processes based on a program stored, for example, in read-only memory (ROM) 202 or loaded from, for example, storage unit 208 into random access memory (RAM) 203. The CPU 201 may be, for example, an electronic processor 201, which may include one or more processor cores, and in some examples, the processor 201 may be multiple processors. The RAM 203 also stores data required by the CPU 201 to execute various processes as needed. The CPU 201, ROM 202, and RAM 203 are interconnected via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0089] The following components are connected to I / O interface 205: input unit 206, which may include a keyboard, mouse, etc.; output unit 207, which may include a display such as a liquid crystal display (LCD) and one or more speakers; storage unit 208, which may include a hard disk or another suitable storage device; and communication unit 209, which may include a network interface card such as a network card (e.g., wired or wireless).

[0090] In some embodiments, communication unit 209 is configured to communicate with other devices (e.g., via a network). Drive 210 is also connected to I / O interface 205 as needed. Removable media 211 (such as a disk, optical disk, magneto-optical disk, flash drive, or other suitable removable media) is mounted on drive 210 such that computer programs read therefrom can be installed into storage unit 208 as needed. Those skilled in the art will understand that while apparatus 200 is described as including the components described above, in practice, some of these components may be added, removed, and / or replaced, and all such modifications or alterations fall within the scope of this disclosure.

[0091] According to exemplary embodiments of this disclosure, the processes described above can be implemented as computer software programs or on computer-readable storage media. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program including program code for performing methods. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 209, and / or installed from removable medium 211, such as... Figure 7 As shown.

[0092] Typically, the various exemplary embodiments of this disclosure can be implemented in hardware or dedicated circuitry (e.g., control circuitry), software, logic, or any combination thereof. For example, Figure 2 Each step can be controlled by a control circuit (e.g., CPU 201 combined with...) Figure 7 The control circuit can perform the actions described in this disclosure, as it is executed by other components.

[0093] Some aspects may be implemented in hardware, while others may be implemented in firmware or software (which may include control circuitry) that can be executed by a controller, processor, or other computing device(s). Although various aspects of the exemplary embodiments of this disclosure are illustrated and described as block diagrams, flowcharts, or using some other graphical representation, it should be understood that the blocks, apparatuses, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuitry or logic, general-purpose hardware or controllers, other computing devices, or some combination thereof, as non-limiting examples.

[0094] Furthermore, the various blocks shown in the flowchart can be viewed as method steps, and / or operations resulting from the operation of computer program code, and / or multiple coupled logic circuit elements configured to perform associated functions. For example, embodiments of this disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program containing program code configured to perform the methods described above.

[0095] Computer program code used to perform the methods of this disclosure may be written in any combination of one or more programming languages. This computer program code may be provided to one or more processors of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus having control circuitry, such that when executed by one or more processors of the computer or other programmable data processing apparatus, the program code implements the functions / operations specified in the flowcharts and / or block diagrams. The program code may be executed entirely on a computer, partially on a computer, as a standalone software package, partially on a computer and partially on a remote computer, or entirely on a remote computer or server, or distributed across one or more remote computers and / or servers.

[0096] The one or more processors can operate as independent devices or can be connected to other processors, such as a network connecting to other processors. Such a network can be built on a variety of different network protocols and can be the Internet, a wide area network (WAN), a local area network (LAN), or any combination thereof.

[0097] Software can be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is well known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, various forms of physical (non-transitory) storage media, such as ROM, PROM, EPROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital universal disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, as is well known to those skilled in the art, (transitory) communication media typically embody computer-readable instructions, data structures, program modules or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information transmission medium.

[0098] The embodiments of the technology disclosed in the accompanying drawings are merely illustrative examples, and the invention is not limited thereto. For example, the illustrated partitions (e.g.) Figure 1 The blocks in the text are merely illustrative logical partitions for ease of discussion, wherein, without departing from the spirit of the invention, such partitions may be split into additional partitions, combined into fewer partitions, supplemented with additional partitions, or reduced by eliminating partitions. For Figure 2The flowchart shown illustrates that, without departing from the spirit of the invention, the partitions of operation steps (which may also be referred to as functions, steps, operations, processes, or actions) can be combined into fewer steps or broken down into additional steps, wherein the steps can be completely or partially reordered or eliminated.

[0099] Unless otherwise specifically stated, it is obvious from the following discussion that, throughout this public discussion, terms such as “processing,” “computing,” “calculating,” “determining,” and “analyzing” are used to refer to the functions, actions, steps, and / or processes of computer hardware or computing systems or similar electronic computing devices that manipulate and / or transform data represented as physical (e.g., electronic) quantities into other data similarly represented as physical quantities.

[0100] It should be understood that in the foregoing description of exemplary embodiments of the invention, various features of the invention are sometimes combined together in a single embodiment, figure, or description thereof to simplify the disclosure and aid in understanding one or more of the inventive aspects. However, the approach of this disclosure should not be construed as reflecting an intention to require more features than expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspect lies in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into the detailed description, wherein each claim is independently a separate embodiment of the invention. Furthermore, while some embodiments described herein include some features included in other embodiments but not others included in other embodiments, as will be understood by those skilled in the art, combinations of features from different embodiments are intended to be within the scope of the invention and to form different embodiments. For example, any of the claimed embodiments in the appended claims can be used in any combination.

[0101] Furthermore, certain embodiments herein are described as methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing functions. Thus, a processor having instructions for performing such methods or method elements forms means for performing methods or method elements. It should be noted that when a method comprises multiple elements (e.g., several steps), no particular order of these elements is implied unless specifically stated otherwise. Furthermore, the elements of the apparatus embodiments described herein are examples of means for performing the functions performed by the elements to carry out embodiments of the invention. Numerous specific details are set forth in the description provided herein. However, it should be understood that embodiments of the invention can be practiced without these specific details. In other instances, well-known methods, structures, and techniques have not been shown in detail in order to avoid obscuring the understanding of this specification.

[0102] Those skilled in the art will recognize that the present invention is by no means limited to the preferred embodiments described above. Rather, various modifications and variations are possible within the scope of the appended claims. For example, other transforms besides MDCT may be used in the encoder, and other filter banks besides gamma-ton may be used in distortion analysis.

[0103] Various aspects of this disclosure can be understood from the following enumerated example embodiments (EEE): EEE 1. A method for specifying the duration as I audio signal s(t) An encoding method for encoding an audio signal, the method comprising: transforming the audio signal using a frame-based frequency transform to obtain a transform domain representation of the audio signal; applying an analytical filter bank to the audio signal to obtain a filter bank domain representation of the audio signal, the filter bank domain having... B One frequency band; define a set B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b Perception-based distortion correlation in; the set B Weight functions w b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; the bin weight υ is used α To calculate the quantization step size Δ J The quantization step size is configured to allocate a total number of bits in time and frequency. R ; and using the quantization step size Δ JThe transform domain representation of the audio signal is quantized.

[0104] EEE 2. The method according to EEE 1, wherein the quantization step size is calculated by controlling the total number of bits R, given a predefined constraint on a weighted distortion metric based on the bin weights.

[0105] EEE 3. The method according to EEE 1, wherein the quantization step size is calculated by controlling a weighted distortion metric based on bin weights, given a maximum number of available bits.

[0106] EEE 4. The method according to any one of the preceding EEEs, wherein the frame-based frequency transformation is configured to transform a first number of time samples in the frame into a second number of frequency samples, wherein the second number is less than the first number.

[0107] EEE 5. The method according to any one of the preceding EEE, wherein the frame-based transform domain has a first time resolution and the analysis filter bank domain has a second time resolution, wherein the second time resolution is higher than the first time resolution.

[0108] EEE 6. The method according to any one of the preceding EEE, wherein each weight function w b (t) Equal to a constant c b Divide by the masking function m b (t) The masking function m b (t) This represents a level at which distortion will be masked by the audio signal so that the human listener cannot perceive the distortion.

[0109] EEE 7. According to the method described in EEE 6, where all constants c b The sum equals the number of frequency bands B .

[0110] EEE 8. According to the method described in EEE 7, wherein each constant c b identified as Where B is the number of frequency bands. BW(g b ) It is a filter bank filter. g b The bandwidth, and It is the sum of the bandwidths of all filters in all filter banks.

[0111] EEE 9. According to the method described in EEE 8, wherein each constant c b identified as Where B is the number of frequency bands. It is a filter bank filter. g b The sample sum of squares of the impulse response, and It is the sum of the squares of all impulse responses of all filters in all filter banks.

[0112] EEE 10. The method according to any one of the preceding EEE, wherein each warehouse weight υ α It is formed as the sum of frequency band terms across all filter bank domain frequency bands, wherein each frequency band term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor and a weighting function for that frequency band. w b (t) .

[0113] EEE 11. The method according to EEE 9, wherein the pre-computed factor comprises a temporal convolution between an analysis filter associated with the frequency band and a transformed waveform associated with the bin weights.

[0114] EEE 12. The method according to any one of EEE 1 to 9, wherein the transform domain is divided into a set of scaling factor bands, and wherein bins belonging to the same scaling factor band are grouped together. J All warehouses share a common quantization step size Δ J .

[0115] EEE 13. The method according to EEE 12, wherein the quantization steps have the same Δ J warehouse group J Variance of quantization error in all warehouses Set as group weight Proportional to the reciprocal of, as follows: Where E is a proportionality constant, and the group weights Defined as with step size Δ J All associated warehouse weights υ α Average value: .

[0116] EEE 14. According to the method described in EEE 13, wherein each warehouse group has a weight υ jIt is formed as the sum of terms across all filter bank domain frequency bands, wherein each term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

[0117] EEE 15. The method according to EEE 14, wherein the pre-computed factor comprises the sum of bin entries on all bins in the bin group, wherein each bin entry comprises an analysis filter associated with the frequency band and a time convolution between a transformed waveform associated with the bin.

[0118] EEE 17. A method for setting the duration as I audio signal s(t) An encoder for encoding an audio signal includes: a transform block (11) for transforming the audio signal using a frame-based frequency transform to obtain a transform domain representation of the audio signal; and a filter bank application block (16) for applying an analysis filter bank to the audio signal to obtain a filter bank domain representation of the audio signal, wherein the filter bank domain has... B Each frequency band; weight determination block (17), used to define a set of frequency bands; B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b The perceptual-based distortion correlation in the middle; weight transformation block (18), used to transform the set of B Weight functions w b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; a scaling block (19) is used to scale the bin weights. α To calculate the quantization step size Δ J The step size is configured to allocate the total number of bits in time and frequency. R ; and a quantization block (15) for using the quantization step size Δ J The transform domain representation of the audio signal is quantized.

[0119] EEE 18. The encoder according to EEE 17, wherein the scaling block is configured to calculate the quantization step size by controlling the total number of bits R, given a predefined constraint on a weighted distortion metric based on the bin weights.

[0120] EEE 19. The encoder according to EEE 17, wherein the scaling block is configured to calculate the quantization step size by controlling a weighted distortion metric based on bin weights, given a maximum number of available bits.

[0121] EEE 20. The encoder according to any one of EEE 17 to 19, wherein the frame-based frequency transformation is configured to transform a first number of time samples in a frame into a second number of frequency samples, wherein the second number is less than the first number.

[0122] EEE 21. An encoder according to any one of EEE 17 to 20, wherein the frame-based transform domain has a first time resolution and the analysis filter bank domain has a second time resolution, wherein the second time resolution is higher than the first time resolution.

[0123] EEE 22. An encoder according to any one of EEE 17 to 21, wherein each weight function w b (t) Equal to a constant c b Divide by the masking function m b (t) The masking function m b (t) This represents a level at which distortion will be masked by the audio signal so that the human listener cannot perceive the distortion.

[0124] EEE 23. The encoder according to EEE 22, wherein all constants c b The sum equals the number of frequency bands B .

[0125] EEE 24. The encoder according to EEE 23, wherein each constant c b identified as Where B is the number of frequency bands. BW(g b ) It is a filter bank filter. g b The bandwidth, and It is the sum of the bandwidths of all filters in all filter banks.

[0126] EEE 25. The encoder according to EEE 23, wherein each constant c b identified as Where B is the number of frequency bands. It is a filter bank filter. g b The sample sum of squares of the impulse response, and It is the sum of the squares of all impulse responses of all filters in all filter banks.

[0127] EEE 26. The encoder according to any one of EEE 17 to 25, wherein the weight transformation block is configured to transfer the weight of each bin to a weight υ. α This is formed as the sum of frequency band terms across all filter bank domain bands, where each frequency band term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

[0128] EEE 27. The encoder according to EEE 26, wherein the pre-computed factor includes a temporal convolution between an analysis filter associated with the frequency band and a transformed waveform associated with the bin weights.

[0129] EEE 28. An encoder according to any one of EEE 17 to 25, wherein the transform domain is divided into a set of scaling factor bands, and wherein bins belonging to the same scaling factor band are grouped together. J All warehouses share a common quantization step size Δ J .

[0130] EEE 29. The encoder according to EEE 28, wherein the encoders have the same quantization step size Δ J warehouse group J Variance of quantization error in all warehouses Set as group weight Proportional to the reciprocal of, as follows: Where E is a proportionality constant, and the group weights Defined as with step size Δ J All associated warehouse weights υ α Average value: .

[0131] EEE 30. The encoder according to EEE 29, wherein the scaling block is configured to square the specific step size. It is set to be proportional to the reciprocal of the group weight.

[0132] EEE 31. The encoder according to EEE 29 or 30, wherein the weight transformation block is configured to transfer the weight υ of each bin group. j This is formed as the sum of terms across all filter bank domain bands, where each term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

[0133] EEE 32. The encoder according to EEE 31, wherein the pre-computed factor comprises the sum of bin entries on all bins in the bin group, wherein each bin entry comprises an analysis filter associated with the frequency band and a temporal convolution between a transform waveform associated with the bin.

[0134] EEE 33. An apparatus comprising a processor and a memory coupled to the processor, wherein the processor is adapted to cause the apparatus to perform the method according to any one of claims 1 to 16.

[0135] EE 34. A computer-readable storage medium storing a program including instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 16.

Claims

1. A method for setting a duration of I audio signal s(t) An encoding method for encoding an audio signal, the method comprising: The audio signal is transformed using frame-based frequency transformation to obtain a transform domain representation of the audio signal; An analytical filter bank is applied to the audio signal to obtain a filter bank domain representation of the audio signal, wherein the filter bank domain has B One frequency band; Define a set B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b Perception-based distortion correlation in [the context]; The group B Weight functions w b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; Using the aforementioned warehouse weight υ α To calculate the quantization step size Δ J The quantization step size is configured to allocate a total number of bits in time and frequency. R ;as well as Using the quantization step size Δ J The transform domain representation of the audio signal is quantized.

2. The method according to claim 1, wherein, The quantization step size is calculated by controlling the total number of bits R, given a predefined constraint on the weighted distortion metric based on the bin weights.

3. The method according to claim 1, wherein, The quantization step size is calculated by controlling a weighted distortion metric based on bin weights, given a maximum number of available bits.

4. The method according to any one of the preceding claims, wherein, The frame-based frequency transformation is configured to transform a first number of time samples in a frame into a second number of frequency samples, wherein the second number is less than the first number.

5. The method according to any one of the preceding claims, wherein, The frame-based transform domain has a first time resolution, and the analysis filter bank domain has a second time resolution, wherein the second time resolution is higher than the first time resolution.

6. The method according to any one of the preceding claims, wherein, Each weight function w b (t) Equal to a constant c b Divide by the masking function m b (t) The masking function m b (t) This represents a level at which distortion will be masked by the audio signal so that the human listener cannot perceive the distortion.

7. The method of claim 6, wherein, All constants c b The sum equals the number of frequency bands B .

8. The method according to claim 7, wherein, Each constant c b identified as ， Where B is the number of frequency bands. BW(g b ) It is a filter bank filter. g b The bandwidth, and It is the sum of the bandwidths of all filters in all filter banks.

9. The method according to claim 8, wherein, Each constant c b identified as Where B is the number of frequency bands. It is a filter bank filter. g b The sample sum of squares of the impulse response, and It is the sum of the squares of all impulse responses of all filters in all filter banks.

10. The method according to any one of the preceding claims, wherein, Weight of each warehouse υ α It is formed as the sum of frequency band terms across all filter bank domain frequency bands, wherein each frequency band term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

11. The method according to claim 10, wherein, The pre-calculated factors include the temporal convolution between the analysis filter associated with the frequency band and the transformed waveform associated with the bin weights.

12. The method according to any one of claims 1 to 9, wherein, The transform domain is divided into a set of scaling factor bands, wherein all bins in bin group J belonging to the same scaling factor band have a common quantization step size Δ. J .

13. The method according to claim 12, wherein, Having the same quantization step size Δ J The variance of the quantization error of all warehouses in warehouse group J Set as group weight Proportional to the reciprocal of, as follows: ， Where E is a proportionality constant, and the group weights Defined as with step size Δ J All associated warehouse weights υ α Average value: 。 14. The method according to claim 13, wherein, The square of a specific step It is selected as being proportional to the inverse of the group weight.

15. The method according to claim 13 or 14, wherein, Weight of each warehouse group υ j It is formed as the sum of terms across all filter bank domain bands, wherein each term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

16. The method according to claim 15, wherein, The pre-calculated factor includes the sum of bin entries across all bins in the bin group, wherein each bin entry includes an analysis filter associated with the frequency band and a temporal convolution between the transformed waveform associated with the bin.

17. A method for setting a duration of I audio signal s(t) Encoders that encode audio signals include: Transform block (11) is used to transform the audio signal using frame-based frequency transformation to obtain a transform domain representation of the audio signal; Filter bank application block (16) is used to apply an analytical filter bank to the audio signal to obtain a filter bank domain representation of the audio signal, wherein the filter bank domain has B One frequency band; Weight determination block (17) is used to define a set of weights B Weight functions w b (t) Each weighting function is associated with a frequency band of the audio signal in the filter bank domain. b Perception-based distortion correlation in [the context]; Weight transformation block (18) is used to transform the set of weights into weights. B Weight functions w b (t) Transformed into a set of bin weights υ in the frame-based transform domain. α Each bin weight is associated with a specific time-frequency bin in the frame-based transform domain; Scaling block (19), used to scale based on the warehouse weight υ α To calculate the quantization step size Δ J The step size is configured to allocate the total number of bits in time and frequency. R ;as well as Quantization block (15), used to quantize step size Δ J The transform domain representation of the audio signal is quantized.

18. The encoder according to claim 17, wherein, The scaling block is configured to calculate the quantization step size by controlling the total number of bits R, given a predefined constraint on the weighted distortion metric based on the bin weights.

19. The encoder according to claim 17, wherein, The scaling block is configured to calculate the quantization step size by controlling a weighted distortion metric based on bin weights, given a maximum number of available bits.

20. The encoder according to any one of claims 17 to 19, wherein, The frame-based frequency transformation is configured to transform a first number of time samples in a frame into a second number of frequency samples, wherein the second number is less than the first number.

21. The encoder according to any one of claims 17 to 20, wherein, The frame-based transform domain has a first time resolution, and the analysis filter bank domain has a second time resolution, wherein the second time resolution is higher than the first time resolution.

22. The encoder according to any one of claims 17 to 21, wherein, Each weight function w b (t) Equal to a constant c b Divide by the masking function m b (t) The masking function m b (t) This represents a level at which distortion will be masked by the audio signal so that the human listener cannot perceive the distortion.

23. The encoder according to claim 22, wherein, All constants c b The sum equals the number of frequency bands B .

24. The encoder according to claim 23, wherein, Each constant c b identified as ， Where B is the number of frequency bands. BW(g b ) It is a filter bank filter. g b The bandwidth, and It is the sum of the bandwidths of all filters in all filter banks.

25. The encoder according to claim 23, wherein, Each constant c b identified as Where B is the number of frequency bands. It is a filter bank filter. g b The sample sum of squares of the impulse response, and It is the sum of the squares of all impulse responses of all filters in all filter banks.

26. The encoder according to any one of claims 17 to 25, wherein, The weight conversion block is configured to assign weight υ to each warehouse. α This is formed as the sum of frequency band terms across all filter bank domain bands, where each frequency band term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

27. The encoder according to claim 26, wherein, The pre-calculated factors include the temporal convolution between the analysis filter associated with the frequency band and the transformed waveform associated with the bin weights.

28. The encoder according to any one of claims 17 to 25, wherein, The transform domain is divided into a set of scaling factor bands, wherein all bins in bin group J belonging to the same scaling factor band have a common quantization step size Δ. J .

29. The encoder according to claim 28, wherein, Having the same quantization step size Δ J The variance of the quantization error of all warehouses in warehouse group J Set as group weight Proportional to the reciprocal of, as follows: ， Where E is a proportionality constant, and the group weights Defined as with step size Δ J All associated warehouse weights υ α Average value: 。 30. The encoder according to claim 29, wherein, The scaling block is configured to square the specific step size. It is set to be proportional to the reciprocal of the group weight.

31. The encoder according to claim 29 or 30, wherein, The weighting conversion block is configured to weight each warehouse group. j This is formed as the sum of terms across all filter bank domain bands, where each term includes the duration. I The integral weighted contribution within the frequency band, wherein the weighted contribution for a specific frequency band includes a pre-calculated factor for that frequency band and the weighting function. w b (t) .

32. The encoder according to claim 31, wherein, The pre-calculated factor includes the sum of bin entries across all bins in the bin group, wherein each bin entry includes an analysis filter associated with the frequency band and a temporal convolution between the transformed waveform associated with the bin.

33. An apparatus comprising a processor and a memory coupled to the processor, wherein, The processor is adapted to cause the device to perform the method according to any one of claims 1 to 16.

34. A computer-readable storage medium storing a program comprising instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 16.

Images