Efficient coding of transients in transform domain

By performing window transformation and vector quantization processing on digital audio signals, the quantization noise and distortion problems caused by transient audio signals in digital audio encoding are solved, the encoding efficiency and applicability are improved, and the number of coded bits is reduced.

CN119068884APending Publication Date: 2024-12-03APPLE INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410649977.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-02
Filing Date
2024-05-24
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When existing digital audio encoding technologies deal with transient audio signals, the lack of time domain masking leads to quantization noise. Especially in encoding applications with limited bandwidth, frequency modeling can cause distortion perception of human perception, called "pre-echo" pseudo- Like, and commonly used techniques increase the number of coded bits, the applicability is poor.

Method used

By parsing the audio sample frame into smaller windows and transforming the time domain sample into frequency domain samples, the envelope of the frequency domain sample is organized using the alignment mode, selecting the cross-frame frequency domain sample to form a vector for quantization, and using vector quantization technology To minimize distortion, weighted processing based on human perception.

Benefits of technology

It effectively reduces quantization noise and distortion perception, improves encoding efficiency, is suitable for applications with limited bandwidth, reduces the number of coded bits, and reduces artifacts.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068884A_ABST
    Figure CN119068884A_ABST
Patent Text Reader

Abstract

The invention relates to efficient coding of transients in the transform domain. Aspects of the present disclosure provide improved techniques for encoding an audio signal having transient audio sounds. An improved technique includes parsing a frame of audio samples of a predetermined length into a series of windows of smaller size, and transforming a window of time domain samples into a series of windows of frequency domain samples. Frequency domain samples may be organized according to an alignment pattern, and frequency domain samples may be encoded relative to an envelope of the organized frequency domain samples.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Priority claim

[0002] The present disclosure benefits from the priority granted by application serial number 63 / 505,838, entitled “Efficient Coding Of Transients In Transform-Domain,” filed on June 2, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] This specification generally relates to digital audio coding. Background Art

[0004] Digital audio encoding typically involves transforming a frame of time-domain audio samples into blocks of frequency-domain samples, and then quantizing the frequency-domain samples.

[0005] In frequency domain coding, transients often result in perceptible quantization noise due to the lack of time domain masking. For example, an impulsive sound followed by silence or silence followed by the onset of speech results in transients that frequency domain coding cannot encode well. When frequency modeling is applied to such transients in bandwidth-limited coding applications, the frequency model often moves signal energy to parts of the audio signal that should be silent, which can result in distorted perception by human listeners. These artifacts are often characterized as "pre-echo" artifacts.

[0006] To mitigate such artifacts, two techniques are popular in audio coding. First, the audio encoder performs a frequency transform of the encoder on a frame of audio content, which may use a shorter transform window when transients occur than when transients are not present. Second, the audio encoder may use time-domain noise shaping (TNS). However, both of these techniques increase the number of bits of the encoded audio content, which may make the technique unsuitable for bandwidth-constrained coding applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Certain features of the disclosure are set forth in the appended claims. However, for purposes of illustration, several aspects of the disclosure are set forth in the following drawings.

[0008] Figure 1 An example encoding system is shown.

[0009] Figure 2 An example audio encoding system in accordance with aspects of the present disclosure is shown.

[0010] Figure 3 An example audio decoding system in accordance with aspects of the present disclosure is shown.

[0011] Figure 4An example process for audio encoding according to aspects of the present disclosure is shown.

[0012] Figure 5 An example process for audio decoding according to aspects of the present disclosure is shown.

[0013] Figure 6 An example frequency coefficient organization is shown in accordance with aspects of the present disclosure.

[0014] Figure 7 An example frequency coefficient organization is shown in accordance with aspects of the present disclosure.

[0015] Figure 8 An envelope alignment pattern in accordance with aspects of the present disclosure is shown.

[0016] Fig. 9 The operation of envelope normalization according to one aspect of the present disclosure is shown. Specifically, Fig. 9 (a) shows the relationship between the spectrum envelope and coefficients before normalization, and Fig. 9 (b) shows the coefficients after applying a normalization operation according to the spectral envelope.

[0017] Fig.10 An example quantizer according to aspects of the present disclosure is shown.

[0018] Fig.11 Example vector sampling patterns are shown in accordance with aspects of the present disclosure.

[0019] Fig.12 An example audio encoding system in accordance with aspects of the present disclosure is shown.

[0020] Fig.13 An example audio decoding system in accordance with aspects of the present disclosure is shown.

[0021] Fig.14 An example computing device is shown that can be used to implement aspects of the present disclosure. DETAILED DESCRIPTION

[0022] The specific description shown below is intended as a description of various configurations of the present disclosure and is not intended to represent the only configuration in which the present disclosure can be practiced. The accompanying drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details intended to provide a thorough understanding of the present disclosure. However, the present disclosure is not limited to the specific details described herein, and one or more other specific implementations may be used to practice. In one or more specific implementations, structures and components are shown in block diagram form to avoid obscuring the concepts of the present disclosure.

[0023] Improved techniques for encoding audio signals having transient audio sounds include parsing a frame of audio samples of a predetermined length into a series of windows of smaller sizes, and transforming the windows of time domain samples into a series of windows of frequency domain samples. In a first aspect, the frequency domain samples may be organized according to an alignment pattern, and the frequency domain samples may be encoded with respect to an envelope of the organized frequency domain samples. In a second aspect, the encoding of the frequency domain samples may include vector quantization of a vector formed by frequency domain samples selected across frames.

[0024] In some specific implementations of improved audio transient coding, an audio sample frame may be encoded relative to an envelope of frequency domain samples arranged according to an alignment pattern of multiple windows traversing a frame. The alignment pattern may include placing the lowest frequency coefficient of a window adjacent to the lowest frequency coefficient of its adjacent window, and also placing the highest frequency coefficient of another window adjacent to the highest frequency coefficient of its adjacent window. A first version of the alignment pattern may include sequentially connecting windows of frequency coefficients, wherein the frequency coefficients are ordered according to their corresponding frequencies, such as by sorting within each window, and the direction of the ordering is reversed between adjacent windows in the connection. For example, the first window may sort the frequencies from low to high, the next window may sort from high to low, then from low to high, and continue in an alternating sorting order. A second version of the alignment pattern may include sorting the frequency coefficients by frequency across the entire frame. For example, the lowest frequency coefficients from all windows in the frame may be grouped, followed by grouping the second lowest frequency coefficients from all windows. In one aspect, the first version of the alignment pattern may be used for frames with strong transients, while the second version of the alignment pattern may be used for frames with weaker transients. In another aspect, an estimate of the envelope of the frequency coefficients is organized according to this alignment pattern, which estimate can be improved, for example, by modeling the envelope with linear prediction.

[0025] In other specific implementations of improved audio transient coding, frequency coefficients can be encoded with vector quantization, wherein a vector is formed by selecting frequency coefficients from dispersed positions across frames. In a first example, the vector can be formed by non-adjacent or disjoint frequencies of a single window. In a second example, the vector can be formed by frequencies from multiple different windows in a frame. The formed vector can be quantized by selecting an entry that minimizes a distortion measure for each vector in a codebook, and in some aspects, the distortion can be weighted based on human perception weighting of the frequencies in the vector. In one aspect, vector quantization can be a conjugate vector quantization according to a conjugate vector codebook.

[0026] Figure 1An example encoding system 100 is shown. The system 100 includes an audio encoder 102 in communication with an audio decoder 104 via a channel 110. In operation, the audio encoder 102 may receive a source audio signal of audio sampled over time, which may be a time-domain representation of an audio source. The audio encoder may encode the signal from the audio source into an encoded audio signal, which may be sent to the audio decoder via the communication channel 110. The audio decoder 104 may decode the received encoded audio, thereby reconstructing the encoded audio into a decoded time-domain audio signal.

[0027] In one aspect, the communication channel 110 may include a transmitter and a receiver and a transmission medium between the transmitter and the receiver over which the encoded audio is transmitted. In other aspects, the communication channel may include a computer storage device on which the encoded audio is stored for transmission between the audio encoder 102 and the audio decoder 104. In some implementations, both the audio encoder and the decoder will be implemented on the same side of the communication channel 110, for example to achieve two-way (duplex) audio transmission.

[0028] Figure 2 is a functional block diagram of an audio coding system 200 according to one aspect of the present disclosure. The system 200 is suitable for Figure 1 The system 200 may include a transient detector 210, a transform unit 220, an envelope processor 230, a quantizer 240, and a syntax unit 250. The system 200 may operate on source audio, which is presented to the system in the form of frames of a predetermined size (e.g., 1024 audio samples). The system 200 may process the source audio frame by frame.

[0029] The transient detector 210 may determine whether the content of a frame indicates the presence of a transient by analyzing the content of the frame. Based on this determination, the transient detector 210 may issue control commands to other units of the system 200. In a first aspect, the transient detector 210 may issue a control signal to the transform unit 220 that determines the size of the window used by the transform unit 220 when applying its transform to the input frame of the source video. In another aspect, the transient detector 210 may issue a control signal to the envelope processor 230 that determines the alignment mode used by the envelope processor 230 when processing the transform coefficients output from the transform unit 220. In yet another aspect, the transient detector 210 may issue a control signal to the quantizer 240 that defines a window from which the quantizer 240 extracts processed coefficients for quantization. The transient detector 210 may also output its control signal to the syntax unit 250, which may provide for output to the audio decoder 102 ( Figure 1 ) in the form of encoded audio data.

[0030] The transform unit 220 may process audio samples within an input frame and transform them into an alternative domain for processing. Typically, the input samples represent source audio on a time domain basis. The transform unit 220 may convert the time domain samples of the input frame into a frequency domain representation, for example, into a set of frequency coefficients. As part of this operation, the transform unit 220 may perform frequency analysis on the samples in the input frame and derive a frequency-based representation of those samples. For example, an overlapped or modified discrete cosine transform ("MDCT") may be applied to the input frame, or a pseudo quadrature mirror filter ("PQMF") may be used. In some examples, the transform unit 220 may perform frequency domain processing based on a basis function derived based on: a non-uniform frequency scale, for example, a warped frequency transform such as a warped MDCT or a warped DCT; or a custom non-uniform frequency scale derived from a machine learning (ML) system that minimizes a multi-resolution short-time Fourier transform (STFT) loss function. The multi-resolution STFT loss function in the ML system may estimate a series of error values ​​in a test system relative to a reference system at multiple time-frequency resolutions.

[0031] The transform unit 220 may apply the transform of the transform unit at a window size as determined by the control signal output by the transient detector 210. When the transient detector 210 determines, for example, that the input frame does not have a transient, the transform unit 220 may perform its transform operation on the entire frame. When the transient detector 210 determines that the input frame has a transient, the transform unit 220 may divide the input frame into a plurality of smaller units (referred to as "windows" for convenience) and apply the transform of the transform unit to each window separately. The number of windows may be determined by a binary decision output from the transient detector 210 indicating whether a transient is determined to exist, which causes the transform unit 220 to perform the transform of the transform unit on the entire frame or on a predetermined number of windows (e.g., 8 windows). Alternatively, the control signal from the transient detector 210 may identify the number of windows into which the source frame may be divided, such as 1, 2, 4, or 8 windows.

[0032] The envelope processor 230 may process the transform domain coefficients from the transform unit 220 to decorrelate them. In general, envelope processing involves processing for normalizing the transform domain coefficient values ​​and for reducing (or eliminating) structure that may be present in the transform domain coefficients input to the envelope processor 230. The envelope processor 230 may output the processed coefficients to the quantizer 240 and output envelope representation data to the syntax unit 250. The envelope representation data may identify the envelope processing parameters applied by the envelope processor 230, the audio decoder 102 ( Figure 1) may invert the envelope processing parameters during audio decoding. For example, the envelope representation may describe the spectral envelope of the transform domain coefficients input to the envelope processor 230; the envelope processor 230 may derive the spectral envelope and then apply the spectral envelope to the coefficients during the envelope normalization process.

[0033] The operation of the envelope processor 230 may be controlled at least in part by a control signal output from the transient detector 210. When the transient detector 210 determines, for example, that the input frame does not have transients, the envelope processor 230 may represent the spectral envelope according to an alignment pattern that extends across the entire frame. However, when the transient detector 210 determines that the input frame has transients, the envelope processor 230 may represent the spectral envelope according to an alignment pattern that arranges the transform coefficients from each window in an effective representation.

[0034] The quantizer 240, as its name implies, may apply a quantization operation to the normalized coefficients output from the envelope processor 230. The quantizer 240 may include, for example, a scalar quantizer, and / or may include a vector quantizer operating according to a vector quantization codebook. In the case of vector quantization, a vector may be derived from the normalized coefficients (which may be selected on a predetermined basis) received from the envelope processor 230, and the vector may be normalized and then applied to one or more predetermined codebooks to identify the best matching codebook entry for the normalized vector. The codebook entry may be output from the quantizer 240 as a representation of the selected coefficient. The quantizer 240 may repeat this operation for a predetermined number of frame coefficients. In one aspect, the vector quantization may be a conjugate vector quantization according to a conjugate vector codebook.

[0035] The operation of quantizer 240 may be controlled at least in part by a control signal output from transient detector 210. In one aspect, quantizer 240 may alter its selection of coefficients used to form a vector in response to a determination from transient detector 210 that a transient is detected. For example, quantizer 240 may select coefficients for each vector according to a selection pattern that ensures that coefficients will be selected from two or more windows generated by transform unit 220. Alternatively, quantizer 240 may select coefficients for each vector according to a decentralized selection pattern.

[0036] The syntax unit 250 may generate an encoded audio signal based on data provided to the syntax unit from the transient detector 210, the envelope processor 230, and the quantizer 240. For example, the syntax unit 250 may receive data from the transient detector 210 representing a determination of whether a transient is present in a source audio frame. The syntax unit 250 may receive data representing a spectral envelope derived by the envelope processor 230. The syntax unit 250 may also receive a vector generated by the quantizer 240 using normalized coefficients. The syntax unit 250 may integrate the received data into a signal to be sent to the audio decoder 104 ( Figure 1 ) in an encoded audio signal of the audio coder 102. Typically, the audio encoder 102 and the audio decoder 104 will operate according to a predetermined encoding protocol; the syntax unit 250 may generate an encoded audio signal that conforms to the encoding protocol. In one aspect, the syntax unit 250 may include lossless entropy encoding output by the quantizer 240 and / or encoding parameters such as an indication of a frame envelope, detected transients, a window size, an envelope alignment mode, and a vector selection mode. In one aspect, the indication of a detected transient may refer to one or more of a frame envelope, a window size, an envelope alignment mode, and a vector selection mode.

[0037] Figure 3 is a functional block diagram of an audio decoding system 300 according to one aspect of the present disclosure. The system 300 may be applicable to Figure 1 The system 300 may include a syntax unit 310, an inverse quantizer 320, an envelope processor 330, an inverse transform unit 340, and a controller 350. The inverse quantizer 320, the envelope processor 330, and the inverse transform unit 340 may reverse the encoding operation, which is performed by Figure 2 The encoding operation is performed by the corresponding devices (quantizer 240, envelope processor 230 and transform unit 220) in the audio encoding system 200.

[0038] The syntax unit 310 can parse the audio codec from the audio coding system 200 ( Figure 2 ) and distributes those data elements to other components within system 300. The encoded audio data may include an indication of decoding control data that controls the decoding process at the decoder, for example, according to a predetermined encoding protocol. For example, transient indicators may be parsed from the encoded audio and provided to controller 350, which may output control signals to inverse quantizer 320, envelope processor 330, and inverse transform unit 340 to govern their operation.

[0039] The inverse quantizer 320 may invert the quantizer 240 ( Figure 2 ) is applied by the encoding operation. For example, the inverse quantizer 320 may refer the codebook index generated by the quantizer 240 to the codebook (not shown) of the inverse quantizer and generate vector data through the codebook. The inverse quantizer 320 may apply the vector parameter to the frequency coefficients, which are as determined by the controller 350 and ultimately by the transient processor 210 ( Figure 2 ) is determined by the selection mode control signal provided by the inverse quantizer 320. The inverse quantizer 320 may output the coefficient set to the envelope processor 330. In one aspect, the vector dequantization may be a conjugate vector dequantization with the aid of a conjugate vector codebook.

[0040] The envelope processor 330 may invert the envelope processor 230 ( Figure 2) encoding operation applied. For example, the envelope processor 330 may be provided in the encoded audio signal and by the syntax unit 310 and ultimately by the transient processor 210 ( Figure 2 ) to invert the normalization process performed by the envelope processor 230. Therefore, the normalization coefficients input to the envelope processor 330 may be output to the inverse transform unit 340 as restored transform coefficients. The normalization inversion may be applied according to the envelope alignment mode control signal provided by the controller 350.

[0041] The inverse transform unit 340 may reverse the transform process performed by the transform unit 220. The recovered transform coefficients received from the envelope processor 330 may be transformed from the transform domain into time domain samples of the recovered audio. For example, frequency coefficients that may be generated by an MDCT process may be converted from the frequency domain to the time domain. The inverse transform conversion process may be based on, for example, the information from the controller 350 and ultimately from the transient processor 210 ( Figure 2 )'s control signal determines the window size.

[0042] In another aspect, frames determined by transient detector 210 to not include transients may be encoded and decoded by an alternative process (not shown). Thus, frames not including transients may bypass the encoding and decoding process. Figure 2 and Figure 3 The encoding and decoding elements shown in .

[0043] Figure 2 and Figure 3 The systems 200 and 300 shown in the drawings represent elements that are provided to encode and decode frames from a single audio source. Many audio coding applications require encoding and decoding of audio from multiple audio channels simultaneously. For example, a stereo audio application involves a pair of audio channels representing left channel audio content and right channel audio content. Other applications such as spatial audio applications require a greater number of channels, such as five or more. The principles of the present disclosure are also applicable to such audio applications. In this case, several instances of systems 200 and 300 may be provided, if necessary, to support these independent audio sources. The grammar units 250 and 310 of each system 200, 300 may be configured in accordance with the audio encoder 102 and the audio decoder 104 ( Figure 1 ) encoding protocol operating under it to multiplex and demultiplex encoded audio from different instances of systems 200, 300.

[0044] Figure 4 4 shows an example process 400 for audio encoding according to aspects of the present disclosure. The process 400 may be performed by, for example, the audio encoder 102 ( Figure 1 ) or audio coding system 200 ( Figure 2). Process 400 includes parsing a frame of time-domain audio samples into windows smaller than the frame (block 406). Transforming the audio samples of each window of the frame into the frequency domain separately from other windows of the frame (block 408). Organizing the resulting frequency-domain coefficients of the frame according to an alignment pattern (block 410). A frequency envelope may be estimated for the organized coefficients (block 416). The frequency-domain coefficients may be encoded into an encoded bitstream relative to the estimated frequency envelope (block 418).

[0045] In an optional aspect of method 400, a transient may be detected in a frame (block 402). When a transient is not detected, the frame may be encoded using an alternative technique (block 404). In one aspect, for example, using the first alignment pattern described above, the frequency domain coefficients of the frame may be organized by an alternating sorting order of frequencies within a window (block 412). In another aspect, for example, using the second alignment pattern described above, the frequency domain coefficients of the frame may be organized by sorting frequencies across a window of the frame (block 414).

[0046] In some implementations, encoding the frequency domain coefficients relative to the envelope (block 418) may include normalizing the coefficients based on the estimated envelope (block 410), removing residual structure from the normalized coefficients (block 422), and vector quantizing the vector of normalized coefficients (block 424). In an aspect, the vector may be formed from disjoint frequencies within a window of the frame (block 426) or may be formed from coefficients across windows of the frame (block 428).

[0047] Figure 5 5 shows an example process 500 for audio decoding according to aspects of the present disclosure. The process 500 may be performed by, for example, the audio decoder 104 ( Figure 1 ) or audio decoding system 300 ( Figure 3 ) is implemented. Process 500 includes decoding frequency domain coefficients and an indication of a frequency envelope from an encoded bitstream (block 502). The coefficients are organized according to an alignment pattern (e.g., the first alignment pattern or the second alignment pattern described above) for the indicated frequency envelope (block 506). The organized coefficients are then denormalized relative to the indicated frequency envelope (block 508). The denormalized frequency domain coefficients are then reorganized into windows (block 512), and the windows of frequency domain coefficients are inverse frequency transformed into reconstructed time domain audio samples.

[0048] In some optional aspects, the decoder may parse an indication of decoding control data from the encoded bitstream, and subsequent decoding operations may be controlled by the decoding control data. For example, the decoder may parse an indication of a transient in an audio frame (516). When a transient is indicated for the frame (518), decoding proceeds to block 502; otherwise, when a transient is not indicated for the frame, an alternative technique is used to decode the frame (520). In another example, the decoder may parse an indication of a residual structure in a frequency coefficient frame. The decoder may apply a residual reorganization structure to the frequency coefficients prior to denormalization in block 508 (526).

[0049] In other optional aspects, the frequency coefficients are quantized using a vector quantizer, and decoding the coefficients (504) can include decoding an index of a vector (504), assigning coefficients from the vector to disjoint frequencies within a window (522), and / or assigning coefficients from the vector to frequencies of different windows across a frame (524). In another optional aspect, denormalizing the coefficients (508) can include scaling the coefficients of a frame with a corresponding envelope value for each coefficient (510).

[0050] Figure 6 6 depicts an exemplary organization of frequency domain coefficients of a frame 600 that may be processed by the transform unit 220 ( Figure 2 ) output. In this example, the determination that the frame contains a transient causes the frame 600 to be divided into eight windows 610.1-610.8. Applying the transform may cause each window 610.1, 610.2, ..., 610.8 to be occupied by respective frequency coefficients 615.1, 615.2, ..., 615.8 representing the audio content of the frame 600 within the respective window 610.1, 610.2, ..., 610.8. Within each window 610.1, 610.2, ..., 610.8, the transform process typically arranges the coefficients in order from the lowest frequency coefficient to the highest frequency (shown as "low-high").

[0051] Figure 6 An example of relates to frame 600 where the transient occurs in time around windows 610.3 and 610.4. The frequency domain coefficients of windows 610.1 and 610.2 have relatively low magnitudes, which increase in window 610.3, and which are at their maximum values ​​in windows 610.4-610.8. This may occur in a frame that has low amplitude time domain samples (e.g., silence) in windows 610.1 and 610.2 followed by some element audio content. In bandwidth-limited coding applications, bits may be allocated to represent the spectral envelope 630 of the window coefficients, and where the audio encoder is limited by the number of bits, the encoder may form a representation of the spectral envelope 630 that is insufficient to represent all of the frequency domain coefficients in the frame. For example, in Figure 6In the example of , the spectral envelope 630 does not include all frequency domain coefficients from all windows 610.1-610.8. In particular, the spectral envelope 630 does not cover the frequency coefficient values ​​within windows 610.1, 610.2, and 610.3 at the lowest frequency coefficient positions (labeled as maximum 620, 622, 624). The frequency coefficients that do not fall under the spectral envelope 630 are not well modeled by the spectral envelope 630, and therefore may appear distorted when decoded. In addition, because the frequency coefficient values ​​at the lowest frequency coefficient positions 620, 622, 624 of windows 610.1, 610.2, and 610.3 are likely to be encoded with distortion, this may result in pre-echo artifacts and possible perceptual distortion when encoding and decoding the frame 600 with reference to the spectral envelope 630.

[0052] Figure 7 700 when generating a spectral envelope 710 according to an envelope alignment mode. Frame 700 may be a frame 600 ( Figure 6 ) is a reorganization of the coefficients in . Figure 6 As shown, in frame 700, frame 700 is divided into a plurality of windows 720.1-720.8. The frequency coefficients 725.1, 725.2, ..., 725.8 of windows 720.1, 720.2, ..., 720.8 are Figure 6 Rather than being rearranged, the rearrangement is done in such a way as to change the order in which the frequency coefficients appear in the representation of the spectral envelope 710. Figure 7 As shown, the envelope alignment pattern positions the highest frequency coefficient in one window (e.g., 720.2) adjacent to the highest frequency coefficient in an adjacent window 720.1. Similarly, the envelope alignment pattern shown positions the lowest frequency coefficient in one window (again 720.2) adjacent to the lowest frequency coefficient in the next window 720.3. This alignment pattern may be repeated across all windows 720.1-720.8 in the frame 700, placing the highest frequency coefficients of adjacent windows adjacent to each other and placing the lowest frequency coefficients of adjacent windows adjacent to each other. Organizing the coefficients according to this envelope alignment pattern may allow for a more simplified representation of the spectral envelope 710 of the frame, which may result in conservation of the number of bits required to represent the spectral envelope 710. As shown by comparing Figure 6 and Figure 7 As can be seen, the spectral envelope 710 may represent a spectrum having a single local maximum 730 (rather than Figure 6 The frequency coefficients of the windows 720.1-720.3 of the three local maxima 620, 622, 624 shown in FIG. 1 are shown in FIG. 1 . In this way, compared to Figure 6 Spectral envelope 630, spectral envelope 710 may be more efficiently represented in the encoded bitstream.

[0053] Figure 8A second envelope alignment mode for an exemplary frame 800 is shown in accordance with one aspect of the present disclosure. Figure 8 As shown, the envelope alignment mode 810 may consider the frequency domain coefficients from multiple windows (e.g., Figure 6 For example, Figure 8 As shown, the next lowest frequency component (f1 1-8 ), the lowest frequency coefficient (usually referred to as the DC coefficient) in the eight windows can be sequentially scanned (shown as fDC 1-8 The envelope alignment pattern 810 may advance to higher and higher frequency positions, thereby sequentially scanning frequency coefficients from all windows until the envelope alignment pattern reaches and scans the highest frequency coefficient (fMAX 1-8 ). It is expected that when Figure 8 The envelope alignment pattern 810 shown in FIG. 8 is obtained from a plurality of windows 610.1-610.8 ( Figure 6 ) when deriving a spectral envelope (not shown) from the frequency components of Figure 6 The spectral envelope 600 of the sweep mode shown in FIG. 6 —can be represented more efficiently.

[0054] In one aspect, the system 200 can be configured to detect a transient condition in accordance with a transient detector 210 ( Figure 1 ) to change the selection of the envelope alignment mode. For example, when the transient detector 210 determines that a transient with a probability exceeding a predetermined threshold is identified, the envelope alignment mode may be applied. Figure 7 The envelope alignment mode shown, however, may be applied when the transient detector 210 determines that a transient with a probability below a threshold has been identified. Figure 8 Of course, when the transient detector 210 determines that there is no transient in the frame (identification with high probability), the envelope processor 230 ( Figure 2 ) can represent the spectral envelope according to a scan direction that progresses across the frequency coefficients of a frame in a default direction, for example, from an isolated DC coefficient to the highest frequency coefficient of the frame.

[0055] In one aspect, the envelope processor 230 ( Figure 2 ) can provide a representation of the spectral envelope of all frequency coefficients traversing a frame, regardless of whether the frame is Figure 7 and Figure 8 In this aspect, the envelope processor 230 ( Figure 2 ) and 330( Figure 3) determines the envelope alignment mode for a frame with reference to window partitions that may exist, but it may encode a representation of the spectral envelope at the granularity of a frame. It is expected that by defining the spectral envelope at a frame granularity rather than at a window granularity (which would require multiple definitions of the spectral envelope due to the presence of multiple windows), the proposed technique results in improved coding efficiency for audio coding.

[0056] As discussed, envelope processor 230 may use the spectral envelope derived for a frame to perform coefficient normalization. Fig. 9 The operation of envelope normalization of an exemplary frame 900 as applied to frequency coefficients according to one aspect of the present disclosure is shown. The envelope processor 230 may scale the frequency coefficients 920 of the frame 900 by dividing the coefficient value at each coefficient position by the value of the spectral envelope 910 at the same position. After the normalization operation, each normalized coefficient will take a value between -1 and 1. Fig. 9 (a) shows the log-magnitude MDCT spectrum and an example spectral envelope used to perform the normalization operation. Fig. 9 (b) shows the Fig. 9 (a) and the normalized coefficient values ​​930 obtained by analyzing the coefficient values ​​920 of spectral envelope 910. In one aspect, the normalized coefficient values ​​930 may be further processed to remove any remaining residual structure present in the spectrum after normalization. For example, a second stage decorrelator such as residual structure remover 1214 may remove Fig. 9 (b) depicts the periodic fine structure. Fig.12 And spectral modeler 1250 provides additional details, including spectral reorganizer 1206, envelope normalizer 1210, envelope estimator 1212, and residual structure remover 1214.

[0057] Fig.10 1 is a functional block diagram of a quantizer 1000 according to one aspect of the present disclosure. Fig.10 As shown, the quantizer 1000 is based on a vector quantizer. The quantizer 1000 may include a vector combination processor 1010, a codebook 1020, a comparator 1030 and a selector 1040. The vector combination processor 1010 may be selected from the envelope processor 230 ( Figure 2 ) and form a vector by extracting coefficients from the normalized coefficients of the frame generated by the codebook 1020. For each vector, the comparator 1030 can compare the vector with the candidate vectors stored in the codebook 1020 to determine the encoding error that will occur if the candidate vector is selected to represent the input vector. The selector 1040 can select the candidate vector that minimizes the encoding distortion of the input vector and output a codebook index representing the selected candidate vector.

[0058] In one aspect, the quantizer 100 may select an index to represent the vector based on the perceptual weighting of the frequencies of the coefficients forming the vector. For example, the comparator 1030 may determine the distortion of the candidate vector by subtracting the element values ​​in the candidate vector from the corresponding element values ​​in the vector to be quantized from the vector combination processor 1010. The resulting difference between the vector elements may be weighted based on the perceptual values ​​of the corresponding frequencies represented by the coefficient vector elements in the vector to be quantized. Subsequently, the weighted differences between the vector elements may be combined into a distortion metric for the candidate vector using a measure such as a mean square error (MSE) or a mean absolute error. Then, the distortion metrics for each corresponding candidate vector may be compared to select an index to represent the vector being quantized.

[0059] The quantizer 10000 may repeat the operation of the quantizer on a plurality of input vectors selected from the frame coefficients until a predetermined number of vectors have been generated from the frame or until the frame coefficients are exhausted.

[0060] As discussed, the vector combination processor 1010 may extract coefficients from a frame of normalized coefficients as determined by a sampling pattern. In one aspect, the sampling pattern may be selected from the transient detector 210 ( Figure 2 ) is provided to the quantizer 1000. In one aspect, as Fig.11 As shown in the exemplary form in , a sampling pattern may be defined to ensure that a candidate vector is formed by coefficients of a plurality of windows as generated by the transform. Fig.11 An exemplary frame 1100 of normalized coefficients shown as being divided into eight windows 1110.1-1110.8 is shown. In this example, the sampling pattern forms candidate vectors from windows 1110.1, 1110.3, 1110.4, and 1110.5. For an n-dimensional candidate vector, the quantizer 1000 ( Fig.10 ) will select n coefficients from the input frame 1100. By selecting coefficients from a plurality of windows 1110.1, 1110.2, ..., 1110.8, it is expected that any coding error caused by quantization of the vectors generated therefrom will be distributed over the relevant windows. Likewise, the quantizer 1000 may repeat its operation on a plurality of candidate vectors, each of which may be selected according to a sampling pattern.

[0061] In a second aspect, the quantizer 1000 input vector may be formed by collecting frequency domain coefficients from a dispersed set of frequencies within a single window, such as 1110.1. In this second example, the sampling pattern may select non-adjacent frequency coefficient positions from the coefficients within the single window 1110.1.

[0062] Fig.12 1 shows an example audio encoding system 1200 according to aspects of the present disclosure. The system 1200 may be an audio encoder 102 ( Figure 1). System 1200 includes a windowed transformer 1202, a transient detector 1204, a spectrum modeler 1250 (including a spectrum reorganizer 1206, an envelope normalizer 1210, an envelope estimator 1212, and a residual structure remover 1214), a residual structure estimator 1216, a quantizer codebook 1218, a quantizer 1220, and a syntax generator 1224.

[0063] In operation, source audio is provided as a time domain signal of audio samples. The source audio is typically organized into "frames," which are units of a predetermined number of samples (such as 1024 samples). When the audio is represented at a fixed sampling rate (e.g., 48kHz or 48,000 samples / second), each frame represents the content of the source audio for a predetermined duration.

[0064] The transient detector 1204 may detect from the content of the frame whether a transient sound occurs during the frame. In one aspect, the transient detector 1204 may determine the strength of the transient within the frame or the probability that a transient exists in the frame. The transient detector 1204 may provide an indication of the transient as, for example, a Boolean value (a transient is detected or not detected in the frame), the strength of the transient detected in the frame, or the probability that a transient exists in the frame. When the transient detector 1204 determines that a transient has occurred, it may generate a control signal to the windowed converter.

[0065] The windowed transformer 1202 may divide the input frame into a plurality of windows when it receives a control signal indicating the presence of transients in the frame from the transient detector 1204. The windowed transformer 1202 may also transform the time-domain audio samples within each window into a set of frequency-domain coefficients, for example using MDCT.

[0066] In one aspect, the number of windows generated by the windowed transformer 1202 may vary based on the content provided by the transient detector 1204 control signal. For example, if no transient is detected, the windowed transformer 1202 may avoid partitioning; the windowed transformer 1202 may transform a frame of source audio using a single MDCT as a unit. Alternatively, if a transient is detected by the transient detector 1204, the windowed transformer 1202 may separately transform windows of frames whose window width is less than the frame length. The MDCT may generate a set of frequency domain coefficients, one set for each partitioned window, which represents the audio content contained within the corresponding partitioned window.

[0067] The spectrum organizer 1206 may order the frequency domain coefficients of each frame according to an alignment pattern. The reorganization in the alignment pattern may improve the efficiency of the envelope representation performed by later stages of the system 100. The alignment pattern may order the frequency coefficients according to their corresponding frequencies.

[0068] As described above, the first alignment pattern may include sequentially connecting windows of frequency coefficients, wherein the frequency coefficients are ordered within each window according to their corresponding frequencies, and the order of the ordering is reversed between adjacent windows in the connection. The frequency coefficients within each window will contain coefficient values ​​for each of a plurality of frequencies between the DC frequency and the maximum frequency generated by the windowed transformer 1202; when considered along the scanning direction represented by the envelope, the first alignment pattern may reposition like coefficients adjacent to each other at the boundary between adjacent windows (e.g., the DC coefficient of one window will be placed adjacent to the DC coefficient of another window, and the highest frequency coefficient of the window will be placed adjacent to the highest frequency coefficient of the adjacent window). Reference is made below to Figure 6 and Figure 7 An example of organizing according to a first alignment mode is described.

[0069] The envelope estimator 1212 may estimate an envelope for an entire frame of frequency domain coefficients organized according to an alignment pattern. The envelope estimator 1212 may generate output data shown as a frequency envelope indication, which is placed into an encoded audio bitstream and sent to the audio decoder 104 ( Figure 1 ); In one aspect, the envelope can be modeled using a linear prediction analysis of the organized coefficients. The envelope estimator 1212 can estimate parameters of the linear prediction model and provide an indication of the frequency envelope as the estimated parameters of the linear model.

[0070] The envelope normalizer 1210 may generate a normalized representation of the frequency domain coefficients based on the frequency envelope indication generated by the envelope estimator 1212. For example, the envelope normalizer 1210 may divide each frequency coefficient by the corresponding value of the coefficient in the frequency envelope indication.

[0071] The residual structure estimator 1216 can identify any residual structure remaining in the frequency domain coefficient sequence after normalizing the frequency domain coefficient sequence based on the envelope, and the residual structure remover can remove the identified residual structure from the normalized frequency domain coefficients. For example, the residual structure can be modeled as a periodic characteristic remaining in the values ​​along the normalized coefficients. The residual structure estimator 1216 estimates parameters of the periodic characteristic model and provides an indication of the residual structure as the estimated parameters of the periodic characteristic model.

[0072] In one aspect, the normalizer 1210 may act as a first stage to decorrelate the sequence of frequency domain coefficients, and then the structure remover 1214 may act as a second stage to decorrelate the sequence of frequency domain coefficients. Some quantizers, including vector quantizers, may operate more efficiently when the sequential inputs to the quantizer are decorrelated from each other.

[0073] Quantizer 1220 may quantize decorrelated frequency domain coefficients. In some implementations, quantizer 1220 may include a vector quantizer 1222 that quantizes the vector of decorrelated frequency coefficients according to vector codebook 1218 to generate an index of a codeword in the codebook for each vector input to quantizer 1220. In one aspect, quantizer 1220 may combine multiple types of quantizers. For example, quantizer 1220 may use a (uniform or non-uniform) scalar quantizer to quantize lower frequency coefficients and also use a vector quantizer to quantize higher frequency coefficients. In another aspect, codebook 1218 may include multiple codebooks, such as a conjugate vector codebook.

[0074] In various aspects, the vectors may be formed by collecting disjoint frequency coefficients from a sequence of frequency samples of a frame. In a first example, the vectors may be formed by collecting frequency domain coefficients from multiple windows into each vector. In a second example, the vectors may be formed by collecting frequency domain coefficients from a dispersed set of frequencies within a single window. In this second example, the vectors may include only non-adjacent frequencies.

[0075] The grammar generator 1224 may integrate input data received from other processing elements in the system 1200 into an encoded bitstream to be sent to the audio decoder 100. For example, the grammar generator 1224 may receive a frame codebook index from the quantizer 1220, an indication of a detected transient from the transient detector 1204, a frequency envelope indication from the envelope estimator 1212, and a residual structure estimate from the residual structure estimator 1216. The grammar generator 1224 may generate a grammar based on the information generated between the audio encoder 102 and the audio decoder 104 ( Figure 1 ) integrates these data elements into the encoded representation of the frame using the syntax of the encoding protocol utilized between the 1224 and 1225. In some implementations, the syntax generator 1224 may provide a representation of the codebook 1218 in the encoded bitstream. However, the codebook 1218 may not be replicated in each encoded frame.

[0076] Fig.13 1 shows an example audio decoding system 1300 according to aspects of the present disclosure. The system 1300 may be an audio decoder 104 ( Figure 1 ). System 1300 may include a syntax unit 1302, a codebook 1304, an inverse quantizer 1306, a spectrum modeler 1350 (including a residual structure generator 1308, an envelope generator 1310, and a spectrum recombiner 1312), and an inverse transformer 1316.

[0077] In operation, the decoding system 1300 may reverse the encoding system 1200 ( Fig.12). Syntax unit 1302 may parse the encoded bitstream of a frame to generate a codebook index and an indication of the residual structure, frequency envelope, and any detected transients of the frame. Inverse quantizer 1306 may invert quantization (as may be performed by quantizer 1220 ( Fig.12 ) is performed). Codebook 1304 may be based on or equal to encoding codebook 1218 ( Fig.12 ). In one aspect, the resulting frequency domain coefficients may be organized according to the indicated frequency envelope, such as according to the organization of the first alignment pattern or the second alignment pattern described above. The residual structure generator 1308 may generate the indicated residual structure and apply or add the residual structure to the normalized frequency domain coefficients. For example, the residual structure may be generated based on a periodic characteristic model, such as the residual structure estimator 1216 ( Fig.12 ). Envelope generator 1310 may generate the indicated envelope and may denormalize the normalized frequency domain coefficients by applying the envelope to the normalized coefficients. For example, envelope generator 1310 may multiply each normalized coefficient by its corresponding envelope value to produce denormalized frequency domain coefficients. The denormalized coefficients may be reorganized by spectrum reorganizer 1312 for inverse frequency transform. Inverse windowed transformer 1316 may apply a windowed inverse frequency transform, such as an inverse MDCT. In one aspect, the size of the window used by transformer 1316, the organization used in reorganizer 1312 may be based on transient indications.

[0078] Fig.14 An example computing device 400 is shown, by which aspects of the present disclosure may be implemented according to one or more specific implementations, including, for example, systems 200, 300 ( Figure 2 , Figure 3 ) and processes 400, 500( Figure 5 , Figure 6 ). Computing device 1400 can be any computing device or server and / or a portion thereof for generating the features and processes described above, including but not limited to a laptop computer, a smart phone, a tablet device, a wearable device such as goggles or glasses, earbuds or other audio devices, a housing for an audio device, etc. Computing device 1400 may include various types of computer-readable media and interfaces for various other types of computer-readable media. Computing device 1400 includes permanent storage device 1402, system memory 1404 (and / or buffer), input device interface 1406, output device interface 1408, bus 1410, ROM 1412, one or more processing units 1414, one or more network interfaces 1416, and / or subsets and variations thereof.

[0079] The bus 1410 generally represents all system, peripheral, and chipset buses that communicatively connect the many internal devices of the computing device 1400. In one or more implementations, the bus 1410 communicatively connects the one or more processing units 1414 with the ROM 1412, the system memory 1404, and the permanent storage device 1402. The one or more processing units 1414 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes disclosed herein. In different implementations, the one or more processing units 1414 can be a single processor or a multi-core processor.

[0080] ROM 1412 stores static data and instructions needed by one or more processing units 1414 and other modules of computing device 1400. In other aspects, permanent storage 1402 may be a read-write memory device. Permanent storage 1402 may be a non-volatile memory unit that stores instructions and data even when computing device 1400 is turned off. In one or more specific implementations, a mass storage device (such as a magnetic or optical disk and its corresponding disk drive) may be used as permanent storage 1402.

[0081] In one or more implementations, a removable storage device (such as a floppy disk, a flash drive, and its corresponding disk drive) may be used as the permanent storage device 1402. Like the permanent storage device 1402, the system memory 1404 may be a read-write memory device. However, unlike the permanent storage device 1402, the system memory 1404 may be a volatile read-write memory, such as a random access memory. The system memory 1404 may store any of the instructions and data that one or more processing units 1414 may need at runtime. In one or more implementations, the processes disclosed in the subject matter are stored in the system memory 1404, the permanent storage device 1402, and / or the ROM 1412. The one or more processing units 1414 retrieve instructions to be executed and data to be processed from these various memory units in order to perform the processes of one or more implementations.

[0082] The bus 1410 is also connected to an input device interface 1406 and an output device interface 1408. The input device interface 1406 enables a user to transmit information and select commands to the computing device 1400. Input devices that can be used with the input device interface 1406 can include, for example, an alphanumeric keyboard and a pointing device (also referred to as a "cursor control device"). The output device interface 1408 can, for example, enable the display of images generated by the computing device 1400. Output devices that can be used with the output device interface 1408 can include, for example, a printer and a display device, such as a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a flexible display, a flat panel display, a solid state display, a projector, or any other device for outputting information.

[0083] One or more implementations may include a device that acts as both an input device and an output device, such as a touch screen. In these implementations, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and the input from the user may be received in any form, including acoustic, voice, or tactile input.

[0084] Finally, if Fig.14 As shown, bus 1410 also couples computing device 1400 to one or more networks and / or to one or more network nodes via one or more network interfaces 1416. In this manner, computing device 1400 may be part of a computer network, such as a LAN, a wide area network ("WAN"), or an intranet, or may be part of a network of networks, such as the Internet. Any or all components of computing device 1400 may be used with the subject disclosure.

[0085] Implementations within the scope of the present disclosure may be implemented in part or in whole using a tangible computer-readable storage medium (or multiple tangible computer-readable storage media of one or more types) programmed with one or more instructions. Tangible computer-readable storage media may also be non-transitory in nature.

[0086] Computer-readable storage media can be any storage media that can be read, written, or otherwise accessed by a general or special computing device, including any processing electronics and / or processing circuitry capable of executing instructions. For example, without limitation, computer-readable media can include any volatile semiconductor memory, such as RAM, DRAM, SRAM, T-RAM, Z-RAM, and TTRAM. Computer-readable media can also include any non-volatile semiconductor memory, such as ROM, PROM, EPROM, EEPROM, NVRAM, flash memory, nvSRAM, FeRAM, FeTRAM, MRAM, PRAM, CBRAM, SONOS, RRAM, NRAM, racetrack memory, FJG, and Millipede memory.

[0087] In addition, the computer-readable storage medium may include any non-semiconductor memory, such as optical disk storage, magnetic disk storage, magnetic tape, other magnetic storage devices, or any other medium capable of storing one or more instructions. In one or more specific implementations, the tangible computer-readable storage medium may be directly coupled to the computing device, while in other specific implementations, the tangible computer-readable storage medium may be indirectly coupled to the computing device, for example, via one or more wired connections, one or more wireless connections, or any combination thereof.

[0088] Instructions can be directly executable, or can be used to develop executable instructions. For example, instructions can be implemented as executable or non-executable machine code, or can be implemented as high-level language instructions that can be compiled to produce executable or non-executable machine code. In addition, instructions can also be implemented as data, or can include data. Computer executable instructions can also be organized in any format, including routines, subroutines, programs, data structures, objects, modules, applications, applets, functions, etc. As recognized by those skilled in the art, the details including but not limited to the number, structure, sequence and organization of instructions can be significantly different without changing the underlying logic, function, processing and output.

[0089] Although the above discussion mainly involves microprocessors or multi-core processors that execute software, one or more implementations are performed by one or more integrated circuits such as ASICs or FPGAs. In one or more implementations, such integrated circuits execute instructions stored on the circuits themselves.

[0090] Those skilled in the art will recognize that the various illustrative frames, modules, elements, parts, methods and algorithms described herein can be implemented as electronic hardware, computer software or a combination of the two. In order to illustrate this interchangeability of hardware and software, various illustrative frames, modules, elements, parts, methods and algorithms have been generally described above according to functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. The technician can implement the described functionality in different ways for each specific application. Various parts and frames can be arranged differently (e.g., arranged in different orders, or divided in different ways) without departing from the scope of the present disclosure.

[0091] It should be understood that any specific order or the hierarchy of the frames in the disclosed process are illustrations of the example method. Based on the design preferred requirements, it should be understood that the specific order or the hierarchy of the frames in the process can be rearranged or all the frames shown are executed. Any frame in these frames can be executed simultaneously. In one or more specific implementations, multitasking and parallel processing may be advantageous. In addition, the division of each system component in the specific implementation described above should not be understood as requiring such division in all specific implementations, and it should be understood that the program component (e.g., computer program product) and the system can be generally integrated together in a single software product or encapsulated in multiple software products.

[0092] As used in this specification and any claims of this patent application, the terms "base station", "receiver", "computer", "server", "processor" and "memory" all refer to electronic devices or other technical devices. These terms exclude people or groups of people. For the purpose of this specification, the term "display" or "displaying" means displaying on an electronic device.

[0093] As used herein, the phrase "at least one of" following a list of items, any of which is separated by the terms "and" or "or", modifies the list as a whole, rather than each member (i.e., each item) of the list. The phrase "at least one of" does not require selection of at least one of each item listed; rather, the phrase allows for a meaning that includes at least one of any one item and / or at least one of any combination of items and / or at least one of each item. For example, the phrases "at least one of A, B, and C" or "at least one of A, B, or C" each refer to only A, only B, or only C; any combination of A, B, and C; and / or at least one of each of A, B, and C.

[0094] The predicate words "configured to," "operable to," and "programmed to" do not imply any specific tangible or intangible modification of a subject matter but are intended to be used interchangeably. In one or more specific implementations, a processor configured to monitor and control an operation or component may also mean that the processor is programmed to monitor and control the operation or that the processor is operable to monitor and control the operation. Likewise, a processor configured to execute code may be interpreted as a processor that is programmed to execute code or is operable to execute code.

[0095] Phrases such as aspect, this aspect, another aspect, some aspects, one or more aspects, a specific implementation, this specific implementation, another specific implementation, some specific implementations, one or more specific implementations, an embodiment, this embodiment, another embodiment, some embodiments, one or more embodiments, configuration, this configuration, other configurations, some configurations, one or more configurations, disclosure, disclosure, the present disclosure, other variations thereof, and the like are for convenience and do not imply that disclosure involving such one or more phrases is essential to the disclosure, nor does it imply that such disclosure applies to all configurations disclosed. Disclosure involving such phrases may apply to all configurations or one or more configurations. Disclosure involving such phrases may provide one or more examples. Phrases such as aspect or some aspects may refer to one or more aspects and vice versa, and this applies similarly to the other aforementioned phrases.

[0096] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" or as an "example" is not necessarily to be construed as preferred or advantageous over other implementations. In addition, to the extent that the terms "including," "having," and the like are used in the specification or claims, such terms are intended to be inclusive in a manner similar to how the term "comprising" is interpreted when used as a transitional word in a claim.

[0097] All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be covered by the claims. In addition, nothing disclosed herein is intended to be made available to the public regardless of whether the disclosure is explicitly stated in the claims. No claim element shall be interpreted under the provisions of 35 U.S.C. §112(f) unless the element is explicitly stated using the phrase “means for…” or, in the case of a method claim, the element is stated using the phrase “step for…”

[0098] The previous description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications of these aspects are apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein, but are intended to make the full scope consistent with the language claims, wherein reference to elements in singular values ​​is not intended to mean "only one", but refers to "one or more", unless specifically noted. Unless otherwise specifically stated, the term "some" refers to one or more. Male pronouns (e.g., his) include female and neutral (e.g., her and its), and vice versa. Titles and subtitles (if any) are used only for convenience and do not limit the subject disclosure.

Claims

1. An audio encoding method, comprising: Parsing a sequence of audio samples contained in a frame of a predetermined size into a plurality of windows of smaller sizes, Transform the audio samples in the window into a corresponding set of frequency domain coefficients, The spectral envelope of the frame is estimated by: arranging the frequency domain coefficients of the windows according to an alignment pattern in which the lowest frequency coefficient of one window is placed adjacent to the lowest frequency coefficient of its neighboring window and the highest frequency coefficient of another window is placed adjacent to the highest frequency coefficient of its neighboring window, and estimating the spectral envelope of the frame by arranging the frequency domain coefficients according to the alignment pattern, and The audio samples of the frame are encoded with reference to the spectral envelope of the frame.

2. The audio encoding method according to claim 1, further comprising, before parsing into a plurality of windows: detect the presence of audio transients in input audio frames, for frames determined to contain audio transients, repeating the audio encoding method according to claim 1, and For frames determined to not contain audio transients, the frames are encoded according to an alternative encoding technique.

3. The audio encoding method according to claim 1, wherein: The arrangement of the frequency domain coefficients is according to a first alignment pattern, and the first alignment pattern includes connecting the windows of the frequency domain coefficients into a series of windows for the frame, wherein the frequency domain coefficients within each window are ordered by frequency, and the frequency ordering is reversed between adjacent windows in the series.

4. The audio encoding method according to claim 1, wherein: The arrangement of the frequency domain coefficients is according to a second alignment pattern, and the second alignment pattern includes ordering the frequency domain coefficients of all windows according to frequency such that frequency coefficients for the lowest frequencies from all windows are adjacent to each other in the second alignment pattern.

5. The audio encoding method according to claim 1, wherein: The estimation of the spectral envelope comprises estimating linear prediction (LP) parameters corresponding to the frequency domain coefficients arranged according to the alignment pattern.

6. The audio encoding method according to claim 1, wherein: The encoding includes: Normalizing the frequency domain coefficients with reference to the spectral envelope comprises dividing the frequency domain coefficients by corresponding magnitudes of the spectral envelope, and Encode normalized value.

7. The audio encoding method according to claim 6, further comprising: Check whether the normalized frequency domain coefficients have periodic characteristics. encoding a representation of the periodic property, and removing the periodic characteristic from the normalized frequency domain coefficients to produce a reduced correlation value, Wherein the encoding of the audio samples encodes the reduced correlation value.

8. The audio encoding method according to claim 1, wherein: The encoding of the audio samples comprises vector quantization of a vector of frequency domain coefficients extracted from a plurality of the windows.

9. A system for audio encoding, comprising: processor; and a memory storing instructions that, when executed by the processor, cause the system to: Parsing a sequence of audio samples contained in a frame of a predetermined size into a plurality of windows of smaller sizes, Transform the audio samples in the window into a corresponding set of frequency domain coefficients, The spectral envelope of the frame is estimated by: arranging the frequency domain coefficients of the windows according to an alignment pattern in which the lowest frequency coefficient of one window is placed adjacent to the lowest frequency coefficient of its neighboring window and the highest frequency coefficient of another window is placed adjacent to the highest frequency coefficient of its neighboring window, and estimating the spectral envelope of the frame by arranging the frequency domain coefficients according to the alignment pattern, and The audio samples of the frame are encoded with reference to the spectral envelope of the frame.

10. A non-transitory computer readable memory storing instructions for encoding audio, the instructions when executed by a processor causing the processor to: Parsing a sequence of audio samples contained in a frame of a predetermined size into a plurality of windows of smaller sizes, Transform the audio samples in the window into a corresponding set of frequency domain coefficients, The spectral envelope of the frame is estimated by: arranging the frequency domain coefficients of the windows according to an alignment pattern in which the lowest frequency coefficient of one window is placed adjacent to the lowest frequency coefficient of its neighboring window and the highest frequency coefficient of another window is placed adjacent to the highest frequency coefficient of its neighboring window, and estimating the spectral envelope of the frame by arranging the frequency domain coefficients according to the alignment pattern, and The audio samples of the frame are encoded with reference to the spectral envelope of the frame.

11. An audio encoding method, comprising: detecting, from a frame of audio samples, whether an audio transient occurs within the frame and a confidence level of the detection of the audio transient, When an audio transient is detected: Parsing a sequence of audio samples contained in a frame of a predetermined size into a plurality of windows of smaller sizes, Transform the audio samples in the window into a corresponding set of frequency domain coefficients, and The spectral envelope of the frame is estimated by: When an audio transient is detected with a first confidence level, arranging the frequency domain coefficients according to a first alignment pattern, the first alignment pattern comprising concatenating windows of the frequency coefficients into a series of windows for the frame, wherein the frequency coefficients within each window are ordered by frequency and the ordering by frequency is reversed between adjacent windows in the series, and when an audio transient is detected at a second confidence level lower than the first confidence level, arranging the frequency domain coefficients according to a second alignment pattern, the second alignment pattern comprising ordering the frequency coefficients of all windows according to frequency such that the frequency coefficients for the lowest frequencies from all windows are adjacent to each other in the second alignment pattern; as well as The audio samples of the frame are encoded with reference to the spectral envelope of the frame.

12. An audio decoding method, comprising: decoding a frame of normalization coefficients with reference to a spectral envelope of the frame, the spectral envelope being defined by an envelope representation provided in the encoded audio data, the decoding comprising denormalizing the normalization coefficients according to the spectral envelope; as well as transforming the scaled coefficients of the frame from a transform coefficient domain to a temporal sample domain, wherein the transformation occurs at a window granularity at a size smaller than the size of the frame, and the spectral envelope is defined according to an alignment pattern in which the lowest frequency coefficient of a first window is placed adjacent to the lowest frequency coefficient of its neighboring window, and the highest frequency coefficient of the first window is placed adjacent to the highest frequency coefficient of another neighboring window.

13. The audio decoding method according to claim 12, further comprising: Before denormalization of the normalization coefficients: decoding from the encoded audio signal an indication of the presence of an audio transient in a frame of the encoded audio signal, for frames indicated as containing audio transients, repeating the audio decoding method according to claim 12, and For frames indicated as not containing audio transients, the frames are decoded according to an alternative decoding technique.

14. The audio decoding method according to claim 12, wherein: The alignment mode is a first alignment mode, and the first alignment mode includes concatenating the windows of frequency domain coefficients into a series of windows for the frame, wherein the frequency domain coefficients within each window are ordered by frequency, and the frequency ordering is reversed between adjacent windows in the series.

15. The audio decoding method according to claim 12, wherein: The alignment pattern is a second alignment pattern, and the second alignment pattern includes ordering frequency domain coefficients of all windows according to frequency such that frequency domain coefficients for lowest frequencies from all windows are adjacent to each other in the second alignment pattern.

16. The audio decoding method according to claim 12, wherein: The indication of the spectral envelope comprises a linear prediction (LP) corresponding to the frequency domain coefficients arranged according to the alignment pattern.

17. The audio decoding method according to claim 12, wherein: Denormalization includes: The frequency domain coefficients are divided by corresponding amplitudes of the spectral envelope to produce denormalized frequency coefficients.

18. The audio decoding method according to claim 17, further comprising: decoding an indication of a representation of a periodic characteristic of normalized frequency domain coefficients from the encoded audio signal to produce a reduced correlation value; as well as The periodic characteristic is applied to the normalized frequency domain coefficients prior to denormalization.

19. The audio decoding method according to claim 12, wherein: Decoding of the encoded audio signal to determine normalized frequency domain coefficients comprises inverse vector quantization of a vector of frequency domain coefficients from a plurality of the windows.

20. A system for audio decoding, comprising: processor; and a memory storing instructions that, when executed by the processor, cause the system to: for a frame of a predetermined size comprising a plurality of time domain windows of smaller sizes, decoding the encoded audio signal to determine normalized frequency domain coefficients for the frame and an indication of a spectral envelope for the frame arranged according to an alignment pattern, wherein the alignment pattern places the lowest frequency coefficient of one window adjacent to the lowest frequency coefficient of its neighboring window and places the highest frequency coefficient of another window adjacent to the highest frequency coefficient of its neighboring window, denormalizing the normalized frequency domain coefficients for the frame with reference to the spectral envelope for the frame, The denormalized frequency domain coefficients corresponding to the time domain window of the frame are inversely transformed into decoded audio samples for the time domain window.

21. A non-transitory computer readable memory storing instructions for decoding audio, the instructions when executed by a processor causing the processor to: for a frame of a predetermined size comprising a plurality of time domain windows of smaller sizes, decoding the encoded audio signal to determine normalized frequency domain coefficients for the frame and an indication of a spectral envelope for the frame arranged according to an alignment pattern, wherein the alignment pattern places the lowest frequency coefficient of one window adjacent to the lowest frequency coefficient of its neighboring window and places the highest frequency coefficient of another window adjacent to the highest frequency coefficient of its neighboring window, denormalizing the normalized frequency domain coefficients for the frame with reference to the spectral envelope for the frame, The denormalized frequency domain coefficients corresponding to the time domain window of the frame are inversely transformed into decoded audio samples for the time domain window.