Phase quantization in voice encoder

Abstract

Innovations in phase quantization during speech encoding and phase reconstruction during speech decoding are described. For example, in order to encode a set of phase values, a speech encoder may ignore high frequency phase values and / or represent at least some phase values as a weighted sum of primary functions. Alternatively, as another example, to decode a set of phase values, the speech decoder uses a weighted sum of the basis functions to reconstruct at least some of the phase values and / or reconstruct low-frequency phase values, and then uses at least some of the low-frequency phase values to synthesize high-frequency phase values. In many cases, these innovations can improve the performance of voice codecs in low bit rate scenarios even when encoded data is transmitted over networks with insufficient bandwidth or poor transmission quality.

Description

[0001] This application is a divisional application of the invention patent application "Phase Quantization in Speech Encoder" with application date of December 10, 2019 and application number 201980082842.7. Background Art

[0002] With the advent of digital wireless phone networks, voice streaming over the Internet, and Internet telephony, digital processing of speech has become commonplace. Engineers use compression to process speech efficiently while still maintaining quality. One goal of speech compression is to represent the speech signal in a way that provides maximum signal quality for a given number of bits. In other words, the goal is to represent the speech signal with the fewest bits for a given quality level. In some scenarios, other goals may also apply, such as resilience to transmission errors and limiting the overall delay due to encoding / transmission / decoding.

[0003] One type of conventional speech encoder / decoder ("codec") uses linear prediction ("LP") to achieve compression. The speech encoder finds and quantizes the LP coefficients for the prediction filter, which are used to predict the sample value as a linear combination of the previous sample values. The residual signal (also called the "excitation" signal) indicates the portion of the original signal that was not accurately predicted by filtering. Because different kinds of speech have different characteristics, the speech encoder typically compresses the residual signal using different compression techniques for voiced segments (characterized by vocal cord vibrations), unvoiced segments, and silent segments. The corresponding speech decoder reconstructs the residual signal, recovers the LP coefficients used in the synthesis filter, and processes the residual signal with the synthesis filter.

[0004] Given the importance of compression for representing speech in computer systems, speech compression has attracted a great deal of research and development activity. Although previous speech codecs provide good performance in many cases, they also have some disadvantages. In particular, problems may arise when using previous speech codecs in very low bit rate situations. In such cases, wireless telephone networks or other networks may not have enough bandwidth (e.g., due to congestion or packet loss) or may have transmission quality problems (e.g., due to transmission noise or intermittent delays), which prevents the delivery of coded speech under quality constraints and time constraints suitable for real-time communications. Summary of the invention

[0005] In summary, the detailed description presents innovations in speech encoding and speech decoding. Some of the innovations involve phase quantization during speech encoding. Other innovations involve phase reconstruction during speech decoding. In many cases, these innovations can improve the performance of speech codecs in low bit rate scenarios, even when the encoded data is transmitted over networks with insufficient bandwidth or transmission quality issues.

[0006] According to a first set of innovations described herein, a speech encoder receives a speech input (e.g., in an input buffer), encodes the speech input to produce encoded data, and stores the encoded data (e.g., in an output buffer) to output as part of a bitstream. As part of the encoding, the speech encoder filters input values ​​based on the speech input according to linear prediction ("LP") coefficients to produce residual values. The speech encoder encodes the residual values. In particular, the speech encoder determines a set of phase values ​​and encodes the set of phase values. For example, the phase value can be determined by applying a frequency transform to a subframe of a current frame, which produces a complex amplitude value for the subframe, and calculating the phase value (and the corresponding amplitude value) based on the complex amplitude value. To improve performance, the speech encoder can perform various operations when encoding the set of phase values.

[0007] For example, when a speech encoder encodes a set of phase values, the speech encoder uses a weighted sum of a basis function (e.g., a sine function) and a linear component to represent at least some of the phase values ​​in the set of phase values. The speech encoder can use a delayed decision method or other method to determine a set of coefficients for weighting the basis function. The count of the coefficients can be different, depending on the target bit rate and / or other criteria of the encoded data. When a suitable coefficient is found, the speech encoder can use a cost function based on a linear phase metric or other cost function, so that the weighted sum of the basis function together with the linear component is similar to the represented phase value. The speech encoder can parameterize the linear component using an offset value and a slope value, and then combine it with the weighted sum. Using the weighted sum of the basis function and the linear component, the speech encoder can accurately represent the phase value in a compact and flexible manner, which can improve the rate-distortion performance in low bit rate scenarios (i.e., providing better quality for a given bit rate, or equivalently, providing a lower bit rate for a given quality level).

[0008] As another example, when a speech encoder encodes a set of phase values, it omits any phase values ​​in the set of phase values ​​that have a frequency above a cutoff frequency. The speech encoder may select the cutoff frequency based at least in part on a target bit rate for the encoded data, pitch cycle information, and / or other criteria. The omitted high-frequency phase values ​​may be synthesized during decoding based on low-frequency phase values ​​signaled as part of the encoded data. By omitting high-frequency phase values ​​(and synthesizing them during decoding based on low-frequency phase values), the speech encoder may efficiently represent a full range of phase values, which may improve rate-distortion performance in low bitrate scenarios.

[0009] According to a second group of innovations described herein, a speech decoder receives encoded data (e.g., in an input buffer) as part of a bitstream, decodes the encoded data to reconstruct speech, and stores the reconstructed speech (e.g., in an output buffer) for output. As part of decoding, the speech decoder decodes residual values ​​and filters the residual values ​​based on LP coefficients. In particular, the speech decoder decodes a set of phase values ​​and reconstructs the residual values ​​based at least in part on the set of phase values. To improve performance, the speech decoder can perform various operations when decoding the set of phase values.

[0010] For example, when a speech decoder decodes a set of phase values, it uses a weighted sum of basis functions (e.g., sinusoidal functions) and a linear component to reconstruct at least some of the phase values ​​in the set of phase values. The linear component can be parameterized by an offset value and a slope value. The speech decoder can decode a set of coefficients (which weight the basis functions), an offset value, and a slope value, and then use the set of coefficients, the offset value, and the slope value as part of the reconstructed phase value. The counting of the coefficients weighted by the basis functions can be different depending on the target bit rate and / or other criteria of the encoded data. Using the weighted sum of the basis functions and the linear component, the phase values ​​can be accurately represented in a compact and flexible manner, which can improve the rate-distortion performance in low bit rate scenarios.

[0011] As another example, when a speech decoder decodes a set of phase values, the speech decoder reconstructs a first subset of the set of phase values, and then uses at least some of the phase values ​​in the first subset to synthesize a second subset of the set of phase values, wherein each phase value in the second subset has a frequency above a cutoff frequency. The speech decoder may determine the cutoff frequency based at least in part on a target bit rate of the encoded data, pitch cycle information, and / or other criteria. To synthesize the phase values ​​of the second subset, the speech decoder may identify a range of the first subset, determine (as a pattern) the difference between adjacent phase values ​​within the range of the first subset, repeat the pattern above the cutoff frequency, and then integrate the differences between adjacent phase values ​​to determine the second subset. By synthesizing omitted high-frequency phase values ​​based on low-frequency phase values ​​signaled in the bitstream, the speech decoder can efficiently reconstruct the entire range of phase values, which can improve rate-distortion performance in low bitrate scenarios.

[0012] The innovations described herein include, but are not limited to, those covered by the claims. These innovations may be implemented as part of a method, as part of a computer system configured to perform the method, or as part of a computer-readable medium storing computer-executable instructions for causing one or more processors in a computer system to perform the method. Various innovations may be used in combination or alone. The present disclosure is provided to introduce some concepts in a simplified form, which will be further described in the detailed description below. The present disclosure is neither intended to identify the key features or essential features of the claimed subject matter nor to limit the scope of the claimed subject matter. The foregoing and other objects, features and advantages of the present invention will become more apparent through the following detailed description, which is provided with reference to the accompanying drawings and illustrates many examples. The examples may also be capable of other and different applications, and some details may be modified in various aspects without departing from the spirit and scope of the disclosed innovations. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The following drawings illustrate some features of the disclosed innovation.

[0014] Figure 1 is a diagram illustrating an example computer system in which some described examples may be implemented.

[0015] Figure 2a and 2b is a diagram of an example network environment in which some described embodiments may be implemented.

[0016] Figure 3 is a diagram illustrating an example speech encoder system.

[0017] Figure 4 It is shown Figure 3 FIG. 1 is a diagram of the encoding stage of residual values ​​in an example speech encoder system.

[0018] Figure 5 is a diagram illustrating an example delay decision method for finding coefficients that represent a phase value as a weighted sum of basis functions.

[0019] Figure 6a-6d is a flow chart illustrating a technique for speech encoding that includes representing phase values ​​as a weighted sum of basis functions and / or omitting phase values ​​having frequencies above a cutoff frequency.

[0020] Figure 7 is a diagram illustrating an example speech decoder system.

[0021] Figure 8 It is shown Figure 7 FIG. 1 is a diagram of the decoding stage of residual values ​​in an example speech decoder system.

[0022] Figure 9a-9c is a diagram illustrating an example method of synthesizing phase values ​​having frequencies higher than a cutoff frequency.

[0023] Figures 10a-10d is a flow chart illustrating a technique for speech decoding that includes reconstructing a phase value represented as a weighted sum of basis functions and / or synthesizing a phase value having a frequency above a cutoff frequency. DETAILED DESCRIPTION

[0024] The detailed description presents innovations in speech encoding and speech decoding. Some of the innovations involve phase quantization during speech encoding. Other innovations involve phase reconstruction during speech decoding. In many cases, these innovations can improve the performance of speech codecs in low bit rate scenarios, even when the encoded data is delivered over networks with insufficient bandwidth or transmission quality issues.

[0025] In the examples described herein, the same reference numerals in different figures represent the same components, modules or operations. More generally, various alternatives to the examples described herein are possible. For example, some methods described herein can be changed by changing the order of the described method actions, by segmenting, repeating or omitting certain method actions, etc. Various aspects of the disclosed technology can be used in combination or alone. Some innovations described herein solve one or more problems mentioned in the background technology. Generally, a given technology / tool ​​does not solve all these problems. It should be understood that other examples can be utilized without departing from the scope of the present disclosure, and structural, logical, software, hardware and electrical changes can be made. Therefore, the following description should not be considered restrictive. On the contrary, the scope of the present invention is defined by the appended claims.

[0026] I. Example Computer System.

[0027] Figure 1 A general example of a suitable computer system (100) is shown in which some of the described innovations may be implemented. The innovations described herein relate to speech encoding and / or speech decoding. Apart from its use in speech encoding and / or speech decoding, the computer system (100) is not intended to imply any limitation on scope of use or functionality, as the innovations may be implemented in a variety of computer systems including special purpose computer systems adapted for operations in speech encoding and / or speech decoding.

[0028] refer to Figure 1, the computer system (100) includes one or more processing cores (110...11x) of a central processing unit ("CPU") and local on-chip memory (118). The processing cores (110...1xx) execute computer executable instructions. The number of processing cores (110...11x) depends on the implementation and can be, for example, 4 or 8. The local memory (118) can be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two, which can be accessed by the corresponding processing core (110...11x).

[0029] The local memory (118) may store software (180) implementing one or more innovative tools for phase quantization in a speech encoder and / or phase reconstruction in a speech decoder in the form of computer executable instructions for operations performed by the corresponding processing core (110 ... 11x). Figure 1 In the present invention, the local memory (118) is an on-chip memory, such as one or more caches, to which access operations, transfer operations, etc. by the processing cores (110...11x) are fast.

[0030] The computer system (100) may include a processing core (not shown) of a graphics processing unit ("GPU") and local memory (not shown). Alternatively, the computer system (100) includes one or more processing cores (not shown) of a system on a chip ("SoC"), an application specific integrated circuit ("ASIC"), or other integrated circuit, and associated memory (not shown). The processing cores may execute computer executable instructions of one or more innovations for phase quantization in a speech encoder and / or phase reconstruction in a speech decoder.

[0031] More generally, the term "processor" may refer broadly to any device that can process computer-executable instructions, and may include microprocessors, microcontrollers, programmable logic devices, digital signal processors, and / or other computing devices. The processor may be a CPU or other general purpose unit, however, it is also known to provide a dedicated processor using, for example, an ASIC or a field programmable gate array ("FPGA").

[0032] The term "control logic" may refer to a controller, or more generally, one or more processors, that are operable to process computer-executable instructions, determine results, and generate outputs. Depending on the implementation, control logic may be implemented by software executable on a CPU, by software controlling dedicated hardware (e.g., a GPU or other graphics hardware), or by dedicated hardware (e.g., in an ASIC).

[0033] The computer system (100) includes a shared memory (120), which may be a volatile memory (e.g., RAM), a non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or some combination of the two, accessible to the processing core. The memory (120) stores software (180) in the form of computer executable instructions that implements one or more innovative tools for phase quantization in a speech encoder and / or phase reconstruction in a speech decoder for operations performed. Figure 1 In the embodiment, the shared memory (120) is an off-chip memory, and the access operation and transfer operation of the processing core to the shared memory (120) are relatively slow.

[0034] The computer system (100) includes one or more network adapters (140). As used herein, the term network adapter refers to any network interface card ("NIC"), network interface, network interface controller, or network interface device. The network adapter (140) enables communication with another computing entity (e.g., a server, other computer system) over a network. The network can be a telephone network, a wide area network, a local area network, a storage area network, or other network. The network adapter (140) can support wired connections and / or wireless connections for telephone networks, wide area networks, local area networks, storage area networks, or other networks. The network adapter (140) transmits data (e.g., computer executable instructions, voice / audio or video input or output, or other data) via a network connection using a modulated data signal. A modulated data signal is a signal that sets or changes one or more of its characteristics in a manner that encodes information in the signal. By way of example and not limitation, a network connection can use electrical, optical, RF, or other carrier waves.

[0035] The computer system (100) also includes one or more input devices (150). The input device can be: a touch input device such as a keyboard, a mouse, a pen or a trackball, a scanning device, or another device that provides input to the computer system (100). For voice / audio input, the input devices (150) of the computer system (100) include one or more microphones. The computer system (100) may also include a video input, another audio input, a motion sensor / tracker input, and / or a game controller input.

[0036] The computer system (100) includes one or more output devices (160), such as a display. For voice / audio output, the output devices (160) of the computer system (100) include one or more speakers. The output devices (160) may also include a printer, a CD burner, a video output, another audio output, or another device that provides output from the computer system (100).

[0037] The storage device (170) may be removable or non-removable and includes magnetic media (e.g., disks, tapes, or cassettes), optical media, and / or any other media that can be used to store information and can be accessed within the computer system (100). The storage device (170) stores instructions for software (180) that implements one or more innovative tools for phase quantization in a speech encoder and / or phase reconstruction in a speech decoder.

[0038] An interconnection mechanism (not shown) such as a bus, controller, or network interconnects the components of the computer system (100). Typically, operating system software (not shown) provides an operating environment for other software executed in the computer system (100) and coordinates the activities of the components of the computer system (100).

[0039] Figure 1 The computer system (100) is a physical computer system. A virtual machine may include Figure 1 Components of the organization shown.

[0040] The term "application" or "program" may refer to software such as any user mode instructions for providing functionality. The software of an application (or program) may also include instructions for an operating system and / or device drivers. The software may be stored in an associated memory. The software may be, for example, firmware. Although it is contemplated that such software may be executed using a properly programmed general-purpose computer or computing device, it is also contemplated that hard-wired circuits or custom hardware (e.g., ASICs) may be used in place of or in combination with software instructions. Thus, the examples are not limited to any particular combination of hardware and software.

[0041] The term "computer-readable medium" refers to any medium that participates in providing data (e.g., instructions) that can be read by a processor and accessed within a computing environment. Computer-readable media can take many forms, including but not limited to non-volatile media and volatile media. Non-volatile media include, for example, optical or magnetic disks and other permanent memories. Volatile media include dynamic random access memory ("DRAM"). Common forms of computer-readable media include, for example, solid-state drives, flash drives, hard disks, any other magnetic media, CD-ROMs, digital versatile disks ("DVDs"), any other optical media, RAM, programmable read-only memory ("PROM"), erasable programmable read-only memory ("EPROM"), USB memory sticks, any other memory chip or cassette tape, or any other medium that a computer can read. The term "computer-readable memory" specifically excludes temporary propagation signals, carrier waves and waveforms or other intangible or temporary media, although they can also be read by a computer. The term "carrier wave" can refer to an electromagnetic wave modulated in amplitude or frequency to transmit a signal.

[0042] The present invention is described in the general context of computer executable instructions that can be executed in a computer system on a target real or virtual processor. The computer executable instructions may include: instructions that can be executed on the processing core of a general-purpose processor to provide the functions described herein, instructions that can be executed to control a GPU or dedicated hardware to provide the functions described herein, instructions that can be executed on the processing core of a GPU to provide the functions described herein, and / or instructions that can be executed on the processing core of a dedicated processor to provide the functions described herein. In some implementations, computer executable instructions can be organized in program modules. Typically, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The functions of program modules can be combined or split between program modules as desired in various embodiments. Computer executable instructions for program modules can be executed in a local or distributed computer system.

[0043] Many examples are described in this disclosure, and many examples are given for illustrative purposes only. The examples described are not restrictive in any sense, and are not intended to be restrictive. As is apparent from this disclosure, the innovation currently disclosed can be widely applied in a variety of contexts. Those of ordinary skill in the art will recognize that the disclosed innovation can be practiced with various modifications and changes (e.g., structural, logical, software, and electrical modifications). Although the specific features of the disclosed innovations can be described with reference to one or more specific examples, it should be understood that, unless otherwise explicitly stated, these features are not limited to usage in one or more specific examples described with reference to them. This disclosure is neither a literal description of all examples, nor a list of features of the present invention that must be present in all examples.

[0044] When an ordinal number (e.g., "first," "second," "third," etc.) is used before a term as an adjective, the ordinal number is used only to indicate a specific feature (unless otherwise expressly stated), for example, to distinguish the specific feature from another feature described by the same or similar terms. The mere use of the ordinal numbers "first," "second," "third," etc. does not indicate any physical order or position, any temporal order, or ranking in importance, quality, or otherwise. In addition, the mere use of an ordinal number does not define a numerical limitation on the feature identified by the ordinal number.

[0045] When introducing elements, the words “a,” “an,” “the,” and “said” are intended to mean that there are one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that there may be additional elements other than the listed elements.

[0046] When describing a single device, component, module or structure, multiple devices, components, modules or structures (whether they collaborate or not) may be used instead to replace a single device, component, module or structure. Functions described as being owned by a single device may be owned by multiple devices instead, whether they collaborate or not. Similarly, where multiple devices, components, modules or structures are described in this article (whether they collaborate or not), single devices, components, modules or structures may be used instead to replace multiple devices, components, modules or structures. Functions described as being owned by multiple devices may be owned by a single device instead. Typically, a computer system or device may be local or distributed, and may include any combination of dedicated hardware and / or hardware with software that implements the above functions.

[0047] In addition, the techniques and tools described herein are not limited to the specific examples described herein. Instead, the corresponding techniques and tools can be used independently and separately from other techniques and tools described herein.

[0048] Devices, components, modules, or structures that communicate with each other do not need to communicate with each other continuously unless otherwise explicitly stated. Instead, such devices, components, modules, or structures only need to send to each other when necessary or desired, and can actually avoid exchanging data most of the time. For example, a device that communicates with another device via the Internet may not be able to send data to another device once every few weeks. In addition, devices, components, modules, or structures that communicate with each other can communicate directly or indirectly through one or more intermediaries.

[0049] As used herein, the term "send" means any way of transferring information from one device, component, module or structure to another device, component, module or structure. The term "receive" means any way of obtaining information from another device, component, module or structure at a device, component, module or structure. Devices, components, modules or structures can be part of the same computer system or different computer systems. Information can be passed by value (e.g., as a parameter of a message or function call) or by reference (e.g., in a buffer). Depending on the context, information can be directly transmitted or transmitted through one or more intermediate devices, components, modules or structures. As used herein, the term "connection" means an operable communication link between devices, components, modules or structures (which can be part of the same computer system or different computer systems). An operable communication link can be a wired or wireless network connection, which can be direct or through one or more intermediaries (e.g., network).

[0050] The description of an example with multiple features does not imply that all or even any such features are required. Rather, various optional features are described to illustrate various possible examples of the innovations described herein. No feature is essential or required unless expressly stated otherwise.

[0051] In addition, although the processing steps and stages can be described in order, such processing can be configured to work in a different order. The description of a specific sequence or order does not necessarily mean that the steps / stages are required to be performed in that order. The steps or stages can be performed in any practical order. In addition, although described or implied as non-simultaneous, some steps or stages can be performed simultaneously. Describing a process as including multiple steps or stages does not imply that all or even any steps or stages are essential or necessary. Various other examples can omit some or all of the steps or stages described. Unless otherwise explicitly stated, any step or stage is not essential or necessary. Similarly, although a product can be described as including multiple aspects, qualities or characteristics, this does not mean that all aspects are essential or necessary. Various other examples can omit some or all of the aspects, qualities or characteristics.

[0052] Many of the techniques and tools described herein are described with reference to a speech codec. Alternatively, the techniques and tools described herein may be implemented in an audio codec, a video codec, a still image codec, or other media codecs for which encoders and decoders use a set of phase values ​​to represent residual values.

[0053] An enumerated listing of items does not imply that any or all of the items are mutually exclusive unless expressly stated otherwise. Likewise, an enumerated listing of items does not imply that any or all of the items are comprehensive in any category unless expressly stated otherwise.

[0054] For presentation purposes, the detailed description uses terms such as "determine" and "select" to describe computer operations in a computer system. These terms represent operations performed by one or more processors or other components in a computer system and should not be confused with actions performed by humans. The actual computer operations corresponding to these terms vary depending on the implementation.

[0055] II. Example Network Environment.

[0056] Figure 2a and 2b An example network environment (201, 202) is shown, which includes a speech encoder (220) and a speech decoder (270). The encoder (220) and the decoder (270) are connected via a network (250) using an appropriate communication protocol. The network (250) may include a telephone network, the Internet, or another computer network.

[0057] exist Figure 2a In the illustrated network environment (201), each real-time communication ("RTC") tool (210) includes both an encoder (220) and a decoder (270) for two-way communication. A given encoder (220) may produce output compatible with a speech codec format or an extension of a speech codec format, wherein a corresponding decoder (270) accepts encoded data from the encoder (220). The two-way communication may be part of an audio conference, a telephone call, or other two-party or multi-party communication scenario. Although in Figure 2a The network environment (201) includes two real-time communication tools (210), but the network environment (201) may alternatively include three or more real-time communication tools (210) that participate in multi-party communication.

[0058] The real-time communication tool (210) manages the encoding performed by the encoder (220). Figure 3 An exemplary encoder system (300) is shown that may be included in a real-time communication tool (210). Alternatively, the real-time communication tool (210) uses another encoder system. The real-time communication tool (210) also manages decoding by a decoder (270). Figure 7 An example decoder system (700) is shown, which may be included in the real-time communication tool (210). Alternatively, the real-time communication tool (210) uses another decoder system.

[0059] exist Figure 2b In the illustrated network environment (202), an encoding tool (212) includes an encoder (220) that encodes speech for delivery to a plurality of playback tools (214) including decoders (270). One-way communication can be provided for surveillance systems, web surveillance systems, remote desktop conference presentations, game play broadcasts, or other scenarios where speech is encoded and sent from one location to one or more other locations for playback. Although in Figure 2b The network environment (202) in FIG. 1 includes two playback tools (214), but the network environment (202) may include more or fewer playback tools (214). Typically, the playback tool (214) communicates with the encoding tool (212) to determine an encoded speech stream for the playback tool (214) to receive. The playback tool (214) receives the stream, buffers the received encoded data for an appropriate period of time, and begins decoding and playback.

[0060] Figure 3An exemplary encoder system (300) is shown that may be included in an encoding tool (212). Alternatively, the encoding tool (212) uses another encoder system. The encoding tool (212) may also include server-side controller logic for managing connections with one or more playback tools (214). Figure 7 An example decoder system (700) is shown, which may be included in a playback tool (214). Alternatively, the playback tool (214) uses another decoder system. The playback tool (214) may also include client-side controller logic for managing the connection with the encoding tool (212). III. Example speech encoder system.

[0061] Figure 3 An example speech encoder system (300) is shown, with which some of the described embodiments may be implemented. The encoder system (300) can be a general-purpose speech encoding tool capable of operating in any of a variety of modes, such as a low-latency mode for real-time communications, a transcoding mode, and a higher-latency mode for generating media for playback from a file or stream, or the encoder system (300) can be a dedicated encoding tool suitable for one such mode. In some example implementations, the encoder system (300) can provide high-quality speech and audio over various types of connections, including connections over networks with insufficient bandwidth (e.g., low bitrates due to congestion or high packet loss rates) or transmission quality issues (e.g., due to transmission noise or high jitter). In particular, in some example implementations, the encoder system (300) operates in one of two low-latency modes, namely, a low bitrate mode or a high bitrate mode. The low bitrate mode uses the same bitrate as described in reference to FIG. Figure 3 and Figure 4 Describes the components.

[0062] The encoder system (300) can be implemented as part of an operating system module, as part of an application library, or as part of a standalone application using GPU hardware or using dedicated hardware. In general, the encoder system (300) is configured to receive a speech input (305), encode the speech input (305) to produce encoded data, and store the encoded data as part of a bitstream (395). The encoder system (300) includes various components that are implemented using one or more processors and are configured to encode the speech input (305) to produce encoded data.

[0063] The encoder system (300) is configured to receive speech input (305) from a source such as a microphone. In some example implementations, the encoder system (300) can accept ultra-wideband speech input (for input signals sampled at 32kHz) or wideband speech input (for input signals sampled at 16kHz). The encoder system (300) temporarily stores the speech input (305) in an input buffer that is implemented in a memory of the encoder system (300) and is configured to receive the speech input (305). From the input buffer, components of the encoder system (300) read sample values ​​of the speech input (305). The encoder system (300) uses variable length frames. Periodically, sample values ​​in a current batch (input frame) of the speech input (305) are added to the input buffer. The length of each batch (input frame) is, for example, 20 milliseconds. When a frame is encoded, the sample values ​​of the frame are deleted from the input buffer. Any unused sample values ​​are retained in the input buffer for encoding as part of the next frame. Therefore, the encoder system (300) is configured to buffer any unused sample values ​​in the current batch (input frame) and place them in the input buffer before the next batch (input frame).Alternatively, the encoder system (300) may use frames of uniform length.

[0064] A filter bank (310) is configured to separate a speech input (305) into a plurality of frequency bands. The plurality of frequency bands provide input values ​​that are filtered by prediction filters (360, 362) to produce residual values ​​in corresponding frequency bands. Figure 3 In the embodiment of the present invention, the filter bank (310) is configured to divide the speech input (305) into two equal frequency bands - a low frequency band (311) and a high frequency band (312). For example, if the speech input (305) is from an ultra-wideband input signal, the low frequency band (311) may include speech in the range of 0-8kHz, and the high frequency band (312) may include speech in the range of 8-16kHz. Alternatively, the filter bank (310) divides the speech input (305) into more frequency bands and / or unequal frequency bands. Depending on the implementation, the filter bank (310) may use any of various types of infinite impulse response ("HR") or other filters.

[0065] The filter bank (310) can be selectively bypassed. Figure 3In the encoder system (300), if the speech input (305) is from a broadband input signal, the filter bank (310) can be bypassed. In this case, the subsequent processing of the high frequency band (312) by the high frequency band LPC analysis module (322), the high frequency band prediction filter (362), the framer (370), the residual encoder (380), etc. can be skipped, and the speech input (300) directly provides the input value filtered by the prediction filter (360).

[0066] Figure 3 The encoder system (300) includes two linear predictive coding ("LPC") analysis modules (320, 322) configured to determine LP coefficients for respective frequency bands (311, 312). In some example implementations, each LPC analysis module (320, 322) uses a five millisecond look-ahead window to calculate whitening coefficients. Alternatively, the LPC analysis modules (320, 322) are configured to determine the LP coefficients in some other manner. If the filter bank (310) divides the speech input (305) into more frequency bands (or is omitted), the encoder system (300) may include more LPC analysis modules for the respective frequency bands. If the filter bank (310) is bypassed (or omitted), the encoder system (300) may include a single LPC analysis module (360) for a single frequency band, i.e., all speech inputs (305).

[0067] The LP coefficient quantization module (325) is configured to quantize the LP coefficients to produce quantized LP coefficients (327, 328) for the corresponding frequency bands (or for all speech inputs (305) if the filter bank (310) is bypassed or ignored). Depending on the implementation, the LP coefficient quantization module (325) can use any of various combinations of quantization operations (e.g., vector quantization, scalar quantization), prediction operations, and domain conversion operations (e.g., conversion to a line spectral frequency ("LSF") domain) to quantize the LP coefficients.

[0068] Figure 3 The encoder system (300) includes two prediction filters (360, 362), such as whitening filters A(z). The prediction filters (360, 362) are configured to filter input values ​​based on speech input according to quantized LP coefficients (327, 328). The filtering produces residual values ​​(367, 368). Figure 3In the embodiment of the present invention, the low-band prediction filter (360) is configured to filter input values ​​in the low-band (311) according to the quantized LP coefficients (327) of the low-band (311), or if the filter bank (310) is bypassed or omitted, the input values ​​are filtered directly from the speech input (305) according to the quantized LP coefficients (327) to produce (low-band) residual values ​​(367). The high-band prediction filter (362) is configured to filter input values ​​in the high-band (312) according to the quantized LP coefficients (328) of the high-band (312) to produce high-band residual values ​​(368). If the filter bank (310) is configured to divide the speech input (305) into more frequency bands, the encoder system (300) may include more prediction filters for the corresponding frequency bands. If the filter bank (310) is omitted, the encoder system (300) may include a single prediction filter for the entire range of the speech input (305).

[0069] The pitch analysis module (330) is configured to perform pitch analysis to generate pitch cycle information (336). Figure 3 In the embodiment of the present invention, the pitch analysis module (330) is configured to process the low frequency band (311) of the speech input (305) in parallel with the LPC analysis. Alternatively, the pitch analysis module (330) can be configured to process other information, such as the speech input (305). In essence, the pitch analysis module (330) determines a sequence of pitch cycles so as to maximize the correlation between pairs of adjacent cycles. The pitch cycle information (336) can be, for example, a set of subframe lengths corresponding to the pitch cycles, or some other type of information about the pitch cycles in the input of the pitch analysis module (330). The pitch analysis module (330) can also be configured to generate a correlation value. The pitch quantization module (335) is configured to quantize the pitch cycle information (336).

[0070] The voicing decision module (340) is configured to perform a voicing analysis to generate voicing decision information (346). The residual values ​​(367, 368) are encoded using a model suitable for voiced speech content or a model suitable for unvoiced content. The voicing decision module (340) is configured to determine which model to use. Depending on the implementation, the voicing decision module (340) may use any of a variety of criteria to determine which model to use. Figure 3 In an encoder system (300) of the present invention, voicing decision information (346) indicates to a residual encoder (380) whether a frame of residual values ​​(367, 368) should be encoded as voiced speech content or unvoiced speech content on a frame-by-frame basis. Alternatively, the voicing decision module (340) generates the voicing decision information (346) according to other timings.

[0071] The framer (370) is configured to organize the residual values ​​(367, 368) into variable length frames. In particular, the framer (370) is configured to set a framing strategy (voiced or unvoiced) based at least in part on the voicing decision information (346), then set a frame length (367, 368) for a current frame of residual values ​​and set a subframe length for a subframe of the current frame based at least in part on the pitch cycle information (336) and the residual values ​​(367, 368). Some parameters are signaled per subframe in the bitstream (395), while other parameters are signaled per frame. In some example implementations, the framer (370) looks in an input buffer for residual values ​​(367, 368) for a current batch of speech input (305) (and any remainder from a previous batch).

[0072] If the framing strategy is voiced, the framer (370) is configured to set the subframe length based at least in part on the pitch cycle information so that each subframe includes multiple sets of residual values ​​(367, 368) for one pitch period. This facilitates encoding in a pitch-synchronous manner. (Using pitch-synchronous subframes can facilitate packet loss concealment because such operations typically generate an integer count of pitch cycles. Similarly, using pitch-synchronous subframes can facilitate time compression stretching operations because such operations typically remove an integer count of pitch cycles)

[0073] The framer (370) is also configured to set the frame length of the current frame to an integer count of subframes from 1 to w, where w depends on the implementation (e.g., it corresponds to a minimum subframe length of two milliseconds or some other count of milliseconds). In some example implementations, the framer (370) is configured to set the subframe length to encode an integer count of pitch cycles per frame, packing as many subframes as possible into the current frame while having a single pitch cycle per subframe. For example, for a frame length of 20 milliseconds, if the pitch cycle is 4 milliseconds, then the current frame includes residual values ​​(367, 368) for five pitch cycles. As another example, for a frame length of 18 milliseconds, if the pitch cycle is six milliseconds, then the current frame includes residual values ​​(367, 368) for three pitch cycles. In practice, the frame length is limited by the look-ahead window of the framer (370) (e.g., 20 milliseconds of the new batch of residual values ​​plus the remainder of the previous batch).

[0074] The subframe lengths are quantized. In some example implementations, for voiced frames, the subframe lengths are quantized to have an integer length for a signal sampled at 32kHz, and the sum of the subframe lengths has an integer length for a signal sampled at 8kHz. Thus, the length of a subframe is a multiple of 1 / 32 milliseconds, and the length of a frame is a multiple of 1 / 8 milliseconds. Alternatively, subframes and frames of voiced content may have other lengths.

[0075] If the framing strategy is unvoiced, the framer (370) is configured to set the frame length of a frame and the subframe length of a subframe of a frame according to different methods, which may be suitable for unvoiced content. For example, the frame length may have a uniform or dynamic size, and the subframe length of a subframe may be equal or variable for each subframe.

[0076] In some example implementations, although the length of each frame can vary, the average frame length is approximately 20 milliseconds. Using variable-sized frames can improve coding efficiency, simplify codec design, and facilitate encoding each frame independently, which can help speech decoders with packet loss concealment and time-stamping.

[0077] Any residual values ​​not included in a subframe of a frame are left for encoding in the next frame. Therefore, the framer (370) is configured to buffer any unused residual values ​​and place them before the next frame of residual values. The framer (370) can receive new pitch cycle information (336) and voicing decision information (346) and then make decisions about the frame / subframe length and framing strategy for the next frame.

[0078] Alternatively, the framer (370) is configured to organize the residual values ​​(367, 368) into variable length frames using some other method.

[0079] The residual encoder (380) is configured to encode the residual values ​​(367, 368). Figure 4 The residual encoder (380) is shown with respect to the encoding stages of residual values ​​(367, 368), which include an encoding stage in a path for voiced speech and an encoding stage in a path for unvoiced speech. The residual encoder (380) is configured to select one of the paths based on voicing decision information (346) provided to the residual encoder (380).

[0080] If the residual values ​​(377, 378) are for voiced speech, the residual encoder (380) includes separate processing paths for the residual values ​​in different frequency bands. Figure 4 In the embodiment of the present invention, the low-frequency band residual values ​​(377) and the high-frequency band residual values ​​(378) are mostly encoded in different processing paths. If the filter bank (310) is bypassed or omitted, the residual values ​​(377) for the entire range of the speech input (305) are encoded. In any case, for the low-frequency band (or, if the filter bank (310) is bypassed or omitted, the speech input (305)), the residual values ​​(377) are encoded in a pitch-synchronous manner because a frame is divided into subframes each containing a pitch cycle.

[0081] The frequency converter (410) is configured to apply a one-dimensional ("1D") frequency transform to one or more subframes of the residual values ​​(377), thereby generating complex amplitude values ​​for the corresponding subframes. In some exemplary implementations, the 1D frequency transform is a variation of a Fourier transform (e.g., a discrete Fourier transform ("DFT"), a fast Fourier transform ("FFT")) that has no overlap, or alternatively, has overlap. Alternatively, the 1D frequency transform is some other frequency transform that produces frequency domain values ​​from the residual values ​​(377) of the corresponding subframes. Typically, the complex amplitude values ​​for the subframe include: for each frequency within a certain frequency range, (1) a real value representing the amplitude of the cosine at that frequency, and (2) an imaginary value representing the amplitude of the sine at that frequency). Thus, each frequency grouping (bin) contains complex amplitude values ​​for one harmonic. For a completely periodic signal, the complex amplitude values ​​in each grouping remain constant across subframes. If the subframes are stretched or compressed versions of each other, the complex amplitude values ​​also remain constant. The lowest packet (at 0 Hz) can be ignored and set to zero in the corresponding residual decoder.

[0082] The frequency converter (410) is further configured to determine a set of amplitude values ​​(414) and one or more sets of phase values ​​(412) for the corresponding subframes based at least in part on the complex amplitude values ​​of the corresponding subframes. For a frequency, the amplitude value represents the amplitude of the combined cosine and sine at the frequency, and the phase value represents the relative proportion of the cosine and sine at the frequency. In the residual encoder (380), the amplitude value (414) and the phase value (412) are further encoded separately.

[0083] The phase encoder (420) is configured to encode one or more phase value sets (412) to generate a quantization parameter (384) for the phase value set (412). The phase value set can be for a low frequency band (311) or for the entire range (305) of the speech input. The phase encoder (420) can encode one phase value set (412) per subframe or one phase value set (412) for a frame. In this case, the complex amplitude values ​​for the subframes of the frame can be averaged or otherwise aggregated, and the phase value set (412) for the frame can be determined from the aggregated complex amplitude values. Section IV explains the operation of the phase encoder (420) in detail. In particular, the phase encoder (420) can be configured to perform an operation to omit any phase value in the phase value set (412) having a frequency higher than a cutoff frequency. The cutoff frequency can be selected based at least in part on a target bit rate for the encoded data, pitch cycle information (336) from the pitch analysis module (330), and / or other criteria. In addition, the phase encoder (420) can be configured to perform operations to represent at least some of the phase values ​​in the set of phase values ​​(412) using a linear component in conjunction with a weighted sum of a basis function. In this case, the phase encoder (420) can be configured to perform operations to: determine a set of coefficients for weighting the basis function using a delayed decision method, set a count of the coefficients for weighting the basis function (based at least in part on a target bit rate for the encoded data), and / or determine a score for a candidate set of coefficients for weighting the basis function using a cost function based at least in part on a linear phase metric.

[0084] The amplitude encoder (430) is configured to encode the set of amplitude values ​​(414) of the corresponding subframe, thereby generating a quantization parameter (385) for the set of amplitude values ​​(414). Depending on the implementation, the amplitude encoder (430) can encode the set of amplitude values ​​(414) of the corresponding subframe using any one of various combinations of quantization operations (e.g., vector quantization, scalar quantization), prediction operations, and domain conversion operations (e.g., conversion to the frequency domain).

[0085] The frequency converter (410) may also be configured to generate a correlation value (416) for the residual value (377). The correlation value (416) provides a measure of a general characteristic of the residual value (377). Typically, the correlation value (416) measures the correlation of complex amplitude values ​​across subframes. In some example implementations, the correlation value (416) is a cross-correlation measured at the following three frequency bands: 0-1.2 kHz, 1.2-2.6 kHz, and 2.6-5 kHz. Alternatively, the correlation value (416) may be measured in more or fewer frequency bands.

[0086] The sparsity evaluator (440) is configured to generate a sparsity value (442) for the residual values ​​(377), which provides another measure of the general characteristics of the residual values ​​(377). Typically, the sparsity value (442) quantifies the degree of diffusion of energy in the residual values ​​(377) in the time domain. In other words, the sparsity value (442) quantifies the proportion of the energy distribution in the residual values ​​(377). If there are few non-zero residual values, the sparsity value is high. If there are many non-zero residual values, the sparsity value is low. In some example implementations, the sparsity value (442) is the ratio of the average absolute value of the residual values ​​(377) to the root mean square value. The sparsity value (442) can be calculated in the time domain for each subframe of the residual values ​​(377) and then averaged or otherwise aggregated over the subframes of the frame. Alternatively, the sparsity value (442) may be calculated in some other manner (eg, as a percentage of non-zero values).

[0087] The correlation / sparseness encoder (450) is configured to encode the sparsity value (442) and the correlation value (416), thereby generating one or more quantization parameters (386) for the sparsity value (442) and the correlation value (416). In some example implementations, the correlation value (416) and the sparsity value (442) are vector quantized jointly per frame. The correlation value (416) and the sparsity value (442) can be used at the speech decoder when reconstructing high frequency information.

[0088] For the high-frequency band residual values ​​(377) of voiced speech, the encoder system (300) relies on decoder reconstruction through bandwidth extension, as described below. The high-frequency band residual values ​​(378) are processed in a separate path in the residual encoder (380). The energy estimator (460) is configured to measure the energy level of the high-frequency band residual values ​​(378), for example per frame or per subframe. The energy level encoder (470) is configured to quantize the high-frequency band energy level (462), thereby generating a quantized energy level (387).

[0089] If the residual values ​​(377, 378) are for unvoiced speech, the residual encoder (380) includes one or more separate processing paths (not shown) for the residual values. Depending on the implementation, the unvoiced path in the residual encoder (380) can encode the residual values ​​(377, 378) for unvoiced speech using any of a variety of combinations of filtering operations, quantization operations (e.g., vector quantization, scalar quantization), and energy / noise estimation operations.

[0090] exist Figure 3 and Figure 4, the residual encoder (380) is shown processing low-band residual values ​​(377) and high-band residual values ​​(378). Alternatively, the residual encoder (380) can process residual values ​​in more bands or a single band (e.g., if the filter bank (310) is bypassed or omitted).

[0091] Return to Figure 3 In an encoder system (300), one or more entropy encoders (390) are configured to entropy encode parameters (327, 328, 336, 346, 384-389) generated by other components of the encoder system (300). For example, quantization parameters generated by other components of the encoder system (300) can be entropy encoded using a range encoder that uses a cumulative quality function that represents the probability that a value of the quantization parameter is encoded. The cumulative quality function can be trained using a database of speech signals with different background noise levels. Alternatively, the parameters (327, 328, 336, 346, 384-389) generated by other components of the encoder system (300) are entropy encoded in some other manner.

[0092] In conjunction with the entropy encoder, a multiplexer ("MUX") (391) multiplexes the entropy encoded parameters into a bitstream (395). An output buffer implemented in a memory is configured to store the encoded data for output as part of the bitstream (395). In some example implementations, each packet of encoded data for the bitstream (395) is independently encoded, which helps avoid error propagation (loss of one packet affects the reconstructed speech and voice quality of subsequent packets), but may contain encoded data for multiple frames (e.g., three frames or some other frame count). When a single packet contains multiple frames, the entropy encoder (390) may use conditional encoding to improve the encoding efficiency of the second and subsequent frames in the packet.

[0093] The bit rate of the encoded data produced by the encoder system (300) depends on the speech input (305) and the target bit rate. In order to adjust the average bit rate of the encoded data to match the target bit rate, a rate controller (not shown) can compare the most recent average bit rate with the target bit rate and then select among multiple coding profiles. The selected coding profile can be indicated in the bitstream (395). The coding profile can define the bits allocated to different parameters set by the encoder system (300). For example, the coding profile can define: a phase quantization cutoff frequency, a coefficient count for representing a set of phase values ​​as a weighted sum of basis functions (as part of a complex amplitude value), and / or another parameter.

[0094] Depending on the implementation and the desired compression type, modules of the encoder system (300) can be added, omitted, separated into multiple modules, combined with other modules, and / or replaced with similar modules. In alternative embodiments, encoders with different modules and / or other configurations of modules perform one or more of the described techniques. Specific encoder embodiments typically use variants or supplementary versions of the encoder system (300). The relationships shown between modules within the encoder system (300) indicate the general information flow in the encoder system (300); for simplicity, other relationships are not shown.

[0095] IV. Example of phase quantization in a speech coder.

[0096] This section describes innovations in phase quantization during speech coding. In many cases, the innovations can improve the performance of speech codecs in low bit rate scenarios, even when delivering the encoded data over networks with insufficient bandwidth or transmission quality issues. The innovations described in this section fall into two major groups of innovations, which can be used individually or in combination.

[0097] According to the first group of innovations, when the speech encoder encodes the phase value set, the speech encoder only quantizes and encodes the low-frequency phase values ​​below the cut-off frequency. Based on at least some of the low-frequency phase values, high-frequency phase values ​​(higher than the cut-off frequency) are synthesized at the speech decoder. By ignoring the high-frequency phase values ​​(and synthesizing them during decoding based on the low-frequency phase values), the speech encoder can efficiently represent the entire phase value range, which can improve the rate-distortion performance in low bit rate scenarios. The cut-off frequency can be predefined and unchanged. Alternatively, in order to provide flexibility for encoding speech with different target bit rates or encoding speech with different characteristics, the speech encoder can select the cut-off frequency based at least in part on the target bit rate of the encoded data, pitch cycle information and / or other criteria.

[0098] According to the second group of innovations, when the speech encoder encodes a set of phase values, the speech encoder uses a linear component in combination with a weighted sum of basis functions to represent at least some of the phase values. Using the weighted sum of basis functions and the linear component, the speech encoder can accurately represent the phase values ​​in a compact and flexible manner, which can improve the rate-distortion performance in low bit rate scenarios. Although the speech encoder can be implemented to use any of a variety of cost functions when determining the coefficients of the weighted sum, the cost function based on the linear phase metric usually obtains a weighted sum of basis functions that is very similar to the represented phase value. Although the speech encoder can be implemented to use any of a variety of methods when determining the coefficients of the weighted sum, the delayed decision method usually finds suitable coefficients in a computationally efficient manner. The count of the coefficients weighted by the basis function can be predefined and kept unchanged. Alternatively, in order to provide flexibility for encoding speech at different target bit rates, the count of the coefficients can depend on the target bit rate.

[0099] A. Ignore the high frequency phase value and set the cutoff frequency.

[0100] When encoding the set of phase values, the speech encoder can quantize and encode low-frequency phase values ​​below a cutoff frequency and ignore high-frequency phase values ​​above the cutoff frequency. The ignored high-frequency phase values ​​can be synthesized at the speech decoder based on at least some of the lower-frequency phase values.

[0101] The encoded phase value set may be a phase value set for a frame or a phase value set for a subframe of a frame. If the phase value set is for a frame, the phase value set may be calculated directly from the complex amplitude values ​​of the frame. Alternatively, the phase value set may be calculated by aggregating (e.g., averaging) the complex amplitude values ​​of the subframes of the frame and then calculating the phase value of the frame from the aggregated complex amplitude values. For example, to quantize the phase value set for a frame, the speech encoder determines the complex amplitude values ​​of the subframes of the frame, averages the complex amplitude values ​​of the subframes, and then calculates the phase value of the frame from the average complex amplitude value for the frame.

[0102] When omitting high frequency phase values, the speech encoder discards phase values ​​above the cutoff frequency. After determining the phase values, the high frequency phase values ​​can be discarded. Alternatively, the high frequency phase values ​​can be discarded by discarding complex amplitude values ​​(e.g., average complex amplitude values) above the cutoff frequency, and the corresponding high frequency phase values ​​are never determined. Either way, phase values ​​above the cutoff frequency are discarded and therefore omitted from the encoded data in the bitstream.

[0103] Although the cutoff frequency can be predetermined and not changed, there are advantages to adaptively changing the cutoff frequency. For example, to provide flexibility for encoding speech at different target bit rates or encoding speech with different characteristics, the speech encoder can select the cutoff frequency based at least in part on the target bit rate of the encoded data and / or pitch cycle information, which can represent the average pitch frequency.

[0104] Typically, information in a speech signal is transmitted at a fundamental frequency and some of its multiples (harmonics). The speech encoder can set a cutoff frequency in order to retain important information. For example, if a frame contains high-frequency speech content, the speech encoder sets a higher cutoff frequency in order to retain more phase values ​​for the frame. On the other hand, if the frame only includes low-frequency speech content, the speech encoder sets a lower cutoff frequency to save bits. Thus, in some example implementations, the cutoff frequency can fluctuate in a manner that compensates for information loss due to the average of the complex amplitude values ​​of the subframes. If the frame contains high-frequency speech content, the pitch period is short, and the complex amplitude values ​​of many subframes are averaged. The average value may not represent the value in a specific subframe in the subframe. Since information may have been lost due to averaging, the cutoff frequency is higher in order to retain the remaining information. On the other hand, if the frame contains low-frequency speech content, the pitch period is longer, and the complex amplitude values ​​of fewer subframes are averaged. Because less information is lost due to averaging, the cutoff frequency can be lower while still having sufficient quality.

[0105] With respect to the target bit rate, if the target bit rate is lower, the cutoff frequency is lower. If the target bit rate is higher, the cutoff frequency is higher. In this way, the bits allocated to represent the high frequency phase value can be varied in direct proportion to the available bit rate.

[0106] In some example implementations, the cutoff frequency falls within the range of 962 Hz (for low target bitrate and low average pitch frequency) to 4160 Hz (for high target bitrate and high average pitch frequency). Alternatively, the cutoff frequency may vary within some other range.

[0107] The speech encoder may set the cutoff frequency on a frame-by-frame basis. For example, the speech encoder may set the cutoff frequency for a frame to the average pitch frequency variation from frame to frame, even if the target bit rate (e.g., set in response to network conditions reported to the speech encoder by some component external to the speech encoder) changes less frequently. Alternatively, the cutoff frequency may change on some other basis.

[0108] The speech encoder may set the cutoff frequency using a lookup table that associates different cutoff frequencies with different target bit rates and average pitch frequencies. Alternatively, the speech encoder may set the cutoff frequency in some other manner based on a rule, logic, etc. Similarly, the cutoff frequency may be similarly derived at the speech decoder based on the information the speech decoder has about the target bit rate and pitch cycle.

[0109] Depending on the implementation, the phase value exactly at the cutoff frequency may be considered as one of the high-frequency phase values ​​(omitted) or as one of the low-frequency phase values ​​(quantized and encoded).

[0110] B. Use the weighted sum of basis functions to represent the phase value.

[0111] When encoding the phase value set, the speech encoder may represent the phase value set as a weighted sum of basis functions. For example, when the basis function is a sine function, the quantized phase value set P i Defined as:

[0112] Where 0≤i≤I-1,

[0113] Where N is the count of the quantized coefficients (hereinafter referred to as “coefficients”) weighted by the basis function, and K n is one of the coefficients, and I is a count of complex amplitude values ​​(and thus a frequency grouping with phase values). In some example implementations, the basis functions are sine functions, but the basis functions may alternatively be cosine functions or some other type of basis function. The set of phase values ​​may be low frequency phase values ​​(after discarding high frequency phase values, as described in the previous section), a full range of phase values ​​(if high frequency phase values ​​are not discarded), or some other range of phase values. The set of phase values ​​encoded may be a set of phase values ​​for a frame or a set of phase values ​​for a subframe of a frame, as described in the previous section.

[0114] Using the quantized phase value set P i (weighted sum of basis functions) and linear components to define the final quantized phase value set P final_i The linear component can be defined as a×i+b, where a represents the slope value and b represents the offset value. For example, P final_i =P i +a×i+b. Alternatively, other and / or additional parameters may be used to define the linear component.

[0115] To encode a set of phase values, the speech encoder finds the coefficients K n The coefficients are similar to the weighted sum of the basis functions of the phase value set. n To limit the computational complexity, the speech encoder can limit the coefficient Kn For example, the coefficient K n is an integer value constrained in magnitude as follows.

[0116] If n = 1, then |K n |≤5;

[0117] If n = 2, then |K n |≤3;

[0118] If n = 3, then |K n |≤2;

[0119] If n≥4; then |K n |≤1.

[0120] K n The value of is quantized to an integer value. Alternatively, the coefficient K can be constrained according to other constraints n The value of .

[0121] Although the coefficient K n The count N can be predetermined and constant, but the coefficient K can be adaptively changed. n To provide flexibility for encoding speech at different target bit rates, the speech encoder may select the coefficient K based at least in part on the target bit rate for the encoded data. n For example, depending on the target bit rate, the speech encoder can set the coefficient K n The count N of K is set to a fraction of the count I of complex amplitude values ​​(and hence frequency groupings with phase values). In some example implementations, the fraction is in the range of 0.29 to 0.51. Alternatively, the fraction may have some other range. If the target bit rate is high, the coefficient K n The count N is high (there are more coefficients K n ). If the target bit rate is low, the coefficient K n The count N is low (the coefficient K n The speech encoder can use a lookup table that associates different coefficient counts with different target bit rates to set the coefficients K. n Alternatively, the speech encoder may set the coefficient K in some other way according to rules, logic, etc. n Similarly, the coefficient K can be derived at the speech decoder based on the information the speech decoder has about the target bit rate. n The count of N. The coefficient K n The count N of may also depend on the average pitch frequency. The speech encoder may set the coefficient K on a frame-by-frame basis (eg, set to the average pitch frequency variation) or on some other basis nThe count N.

[0122] In the evaluation coefficient K n When the speech encoder uses the cost function (fitness function) option, the speech encoder uses a cost function (fitness function). The cost function depends on the implementation. Using the cost function, the speech encoder determines a set of candidate coefficients K that weight the basis function. n The cost function can also take into account the values ​​of other parameters. For example, for a cost function, the speech encoder calculates the value of the candidate coefficient set K according to the n The weighted basis functions are used to reconstruct a version of the phase value set, and then the linear phase measure is calculated when the inverse of the reconstructed version of the phase value set is applied to the complex amplitude value. In other words, the coefficient K n The cost function is defined as the quantized phase signal P i Applying the inverse of P to the (original) averaged complex spectrum yields a spectrum with maximum linear phase. This linear phase measure is the peak amplitude value of the inverse Fourier transform. If the result is perfect linear phase, then the quantized phase signal exactly matches that of the averaged complex spectrum. For example, when P is applied to the (original) averaged complex spectrum, the quantized phase signal exactly matches that of the averaged complex spectrum. final_i Defined as P i +a×i+b, maximizing the linear phase means maximizing the residual of how well the linear component a×i+b represents the phase value. Alternatively, the cost function may be defined in some other way.

[0123] In theory, the speech encoder can adjust the coefficient K in the entire parameter space. n In practice, a complete search is computationally too complex for most scenarios. To reduce computational complexity, the speech encoder finds the coefficients K that weight the basis functions. n A delayed decision method (eg, Viterbi algorithm) may be used when the set is used to represent a set of phase values.

[0124] Typically, for the delayed decision method, the speech encoder iteratively performs operations to find the coefficients K in multiple stages. n For a given stage, the speech encoder evaluates the coefficient K n The speech encoder evaluates the candidate values ​​according to the cost function to evaluate each candidate value of the given coefficient in combination with each of the candidate solution sets (if any) from the previous stage. The speech encoder retains a certain count of the evaluated combinations as the candidate solution set from the given stage based at least in part on the scores according to the cost function. For example, for a given stage n, the speech encoder retains the coefficients K that pass through the given stage. n Thus, using the delayed decision method, the speech encoder tracks the coefficient Kn The most promising sequence.

[0125] Figure 5 An example of a speech encoder (500) is shown that uses a delayed decision method to find coefficients to represent a set of phase values ​​as a weighted sum of basis functions. To determine the set of coefficients K n , the speech encoder iterates on n=1…N. At each stage (for each value of n), the speech encoder tests K according to the cost function n For example, for the linear phase metric cost function, the speech encoder calculates the coefficient K according to n The combination generates a new phase signal P i , and measure the linear phase of the result. Instead of evaluating the coefficient K n For all possible permutations of the values ​​of (i.e., every possible value of stage 1 × every possible value of stage 2 × ... × every possible value of stage n), the speech encoder evaluates a subset of the possible permutations. Specifically, the speech encoder checks the coefficients K of stage n when linking to each retained combination from stage n-1. n The remaining combinations from stage n-1 include coefficients K0, K1, K n-1 The most promising combination to stage n-1. The count of the retained combinations depends on the implementation. For example, the count is 2, 3, 5, or some other count. The count of the retained combinations can be the same at each stage, or different at different stages.

[0126] exist Figure 5 In the example shown, for the first stage, the speech encoder evaluates each possible value of K1 from -j to j (2j+1 possible integer values), and retains the top three combinations (the best K1 values ​​in the first stage) based on the cost function. For the second stage, the speech encoder evaluates each possible value of K2 from -2 to 2 (five possible integer values) linked to each retained combination (the best K1 values ​​from the first stage), and retains the top three combinations (the best K1+K2 combinations in the second stage) based on the cost function. For the third stage, the speech encoder evaluates each possible value of K3 from -1 to 1 (three possible integer values) linked to each retained combination (the best K1+K2 combinations from the second stage), and retains the top three combinations (the best K1+K2+K3 combinations in the third stage) based on the cost function. This process continues for n stages. In the final stage, the speech encoder evaluates each possible value of K3 linked to each retained combination (the best K1+K2+K3+…+K ... n-1 K from -1 to 1 nEach possible value of (three possible integer values), and then select the best combination (best K1+K2+K3+…+K n-1 +K n ). Even when N is 50, 60, or even higher, the delayed decision method makes the process of finding coefficient values ​​tractable;

[0127] In addition to finding the coefficient K n In addition to the set of basis functions, the speech encoder also determines the parameters of the linear component. For example, the speech decoder determines the slope value a and the offset value b. The offset value b represents the linear phase (offset) relative to the start of the weighted sum of the basis functions, so that the result P final_i The slope value a represents the total slope of the linear component, which is used as a multiplier or scaling factor so that the result P final_i Closer to the original phase signal. The speech encoder can quantize the offset value and the slope value uniformly. Alternatively, the speech encoder can jointly quantize the offset value and the slope value, or encode the offset value and the slope value in some other way. Alternatively, the speech encoder can determine other and / or additional parameters of the weighted sum of the linear components or basis functions.

[0128] Finally, the speech encoder has quantized coefficients K n The speech decoder can use the coefficient K n A set of phase values, offset values, slope values, and / or other values ​​are used to generate an approximation of a set of phase values.

[0129] C. Example techniques for phase quantization in speech coding.

[0130] Figure 6a A general technique for speech coding is shown (601), which may include, for example, Figure 6b , Figure 6c or Figure 6d Additional operations shown. Figure 6b A general technique for speech coding is shown (602) which includes omitting phase values ​​having frequencies above a cutoff frequency. Figure 6c A general technique for speech coding is shown (603), which includes using a weighted sum of basis functions and a linear component to represent a phase value. Figure 6d A more specific example technique (604) for speech coding is shown, which includes omitting high-frequency phase values ​​(which are above the cutoff frequency) and representing low-frequency phase values ​​(which are below the cutoff frequency) as a weighted sum of basis functions. Figure 3 and Figure 4 The described speech encoder or another speech encoder performs the techniques (601-604).

[0131] refer to Figure 6a, the speech encoder receives (610) speech input. For example, an input buffer implemented in a memory of a computer system is configured to receive and store the speech input.

[0132] The speech encoder encodes (620) the speech input to produce encoded data. As part of encoding (620), the speech encoder filters the input values ​​based on the speech input according to the LP coefficients. The input values ​​can be, for example, frequency bands of the speech input generated by the filter bank. Alternatively, the input values ​​can be speech input received by the speech encoder. In any case, filtering produces residual values, which the speech encoder encodes. Figure 6b-6d An example of operations that may be performed as part of the encoding (620) stage of residual values ​​is shown.

[0133] The speech encoder stores (640) the encoded data for output as part of a bitstream. For example, an output buffer implemented in a memory of a computer system stores the encoded data for output.

[0134] refer to Figure 6b , the speech encoder determines (621) a set of phase values ​​for the residual value. The set of phase values ​​may be for a subframe of the residual value or a frame of the residual value. For example, to determine the set of phase values ​​for a frame, the speech encoder applies a frequency transform to one or more subframes of the current frame, which produces complex amplitude values ​​for the corresponding subframes. The frequency transform may be a variant of a Fourier transform (e.g., DFT, FFT), or some other frequency transform that produces complex amplitude values. The speech encoder then averages or otherwise aggregates the complex amplitude values ​​for the corresponding subframes. Alternatively, the speech encoder may aggregate the complex amplitude values ​​for the subframes in other ways. Finally, the speech encoder calculates the set of phase values ​​based at least in part on the aggregated complex amplitude values. Alternatively, the speech encoder determines the set of phase values ​​in some other way, for example, by applying a frequency transform to the entire frame without splitting the current frame into subframes and calculating the set of phase values ​​from the complex amplitude values ​​for the frame.

[0135] The speech encoder encodes the set of phase values ​​(635). In doing so, the speech encoder ignores any set of phase values ​​having a frequency above the cutoff frequency. The speech encoder may select the cutoff frequency based at least in part on a target bit rate for encoding the data, pitch cycle information, and / or other criteria. Phase values ​​at frequencies above the cutoff frequency are discarded. Phase values ​​at frequencies below the cutoff frequency are encoded, for example, as described in reference 6c. Depending on the implementation, the phase value exactly at the cutoff frequency may be treated as one of the high frequency phase values ​​(omitted) or as one of the low frequency phase values ​​(quantized and encoded).

[0136] refer to Figure 6c, the speech encoder determines (621) a set of phase values ​​for the residual value. The set of phase values ​​may be for a subframe of the residual value or a frame of the residual value. For example, the speech encoder determines the set of phase values, such as reference Figure 6b described.

[0137] The speech encoder encodes the set of phase values ​​(636). In doing so, the speech encoder represents at least some of the phase values ​​in the set of phase values ​​using a weighted sum of basis functions and a linear component. For example, the basis functions are sine functions. Alternatively, the basis functions are cosine functions or some other type of basis functions. The phase value represented as a weighted sum of basis functions can be a low frequency phase value (if the high frequency phase values ​​are discarded), an entire range of phase values, or some other range of phase values.

[0138] To encode the phase value set, the speech encoder can determine a set of coefficients that weight the basis function, and also determine an offset value and a slope value that parameterize the linear component. The speech encoder can then entropy encode the coefficient set, the offset value, and the slope value. Alternatively, the speech encoder can encode the phase value set using a set of coefficients that weight the basis function together with some other combination of parameters that define the linear component (e.g., no offset value, or no slope value, or using other parameters). Alternatively, in combination with the set of coefficients that weight the basis function and the linear component, the speech encoder can still use other parameters to represent the phase value set.

[0139] To determine the set of coefficients to weight the basis functions, the speech encoder may use a delayed decision method (described above) or another method (e.g., a complete search of the parameter space for the set of coefficients). When determining the set of coefficients to weight the basis functions, the speech encoder may use a cost function based on a linear phase metric (described above) or another cost function. The speech encoder may set the count of coefficients to weight the basis functions based at least in part on a target bit rate for the encoded data (described above) and / or other criteria.

[0140] exist Figure 6d In the exemplary technique (604), when encoding a set of phase values ​​for a residual value, the speech encoder ignores high-frequency phase values ​​having frequencies above a cutoff frequency and represents low-frequency phase values ​​as a weighted sum of basis functions.

[0141] The speech encoder applies (622) a frequency transform to one or more subframes of a frame, which produces complex amplitude values ​​for the corresponding subframes. The frequency transform can be a variation of a Fourier transform (e.g., DFT, FFT), or some other frequency transform that produces complex amplitude values. The speech encoder then averages (623) the complex amplitude values ​​of the subframes of the frame. Next, the speech encoder calculates (624) a set of phase values ​​for the frame based at least in part on the averaged complex amplitude values.

[0142] The speech encoder selects (628) a cutoff frequency based at least in part on a target bit rate for encoding the data and / or pitch cycle information. The speech encoder then discards (629) any set of phase values ​​for frequencies above the cutoff frequency. Thus, phase values ​​at frequencies above the cutoff frequency are discarded, but phase values ​​at frequencies below the cutoff frequency are further encoded. Depending on the implementation, the phase value exactly at the cutoff frequency may be treated as one of the high frequency phase values ​​(discarded) or as one of the low frequency phase values ​​(quantized and encoded).

[0143] To encode low frequency phase values ​​(i.e., phase values ​​below the cutoff frequency), the speech encoder represents the low frequency phase values ​​using a weighted sum of basis functions and a linear component. The speech encoder sets (630) a count of coefficients weighted for the basis functions based at least in part on a target bit rate for the encoded data. The speech encoder uses (631) a delayed decision method to determine a set of coefficients for weighting the basis functions. The speech encoder also determines (632) an offset value and a slope value for parameterizing the linear component. The speech encoder then encodes (633) the set of coefficients, the offset value, and the slope value.

[0144] The speech encoder can repeat on a frame-by-frame basis Figure 6d The speech encoder may repeat the process on a frame-by-frame basis or on some other basis. Figure 6a-6c Any of the techniques (601-603) shown.

[0145] V. Example Speech Decoder System.

[0146] Figure 7 An example speech decoder system (700) is shown in conjunction with which some of the described embodiments may be implemented. The decoder system (700) may be a general purpose speech decoding tool capable of operating in any of a variety of modes, such as a low latency mode for real-time communications, a transcoding mode, and a higher latency mode for playing back media from a file or stream, or the decoder system (700) may be a dedicated decoding tool adapted for one such mode. In some example implementations, the decoder system (700) may play back high-quality speech and audio over various types of connections, including connections over networks with insufficient bandwidth (e.g., low bitrates due to congestion or high packet loss) or transmission quality issues (e.g., due to transmission noise or high jitter). In particular, in some example implementations, the decoder system (700) operates in one of two low latency modes, a low bitrate mode or a high bitrate mode. The low bitrate mode uses reference Figure 7 and Figure 8 The components described.

[0147] The decoder system (700) can be implemented as part of an operating system module, as part of an application library, or as part of a standalone application using GPU hardware or dedicated hardware. In general, the decoder system (700) is configured to receive encoded data as part of a bitstream (705), decode the encoded data to reconstruct speech, and store the reconstructed speech (775) for output. The decoder system (700) includes various components that are implemented using one or more processors and are configured to decode the encoded data to reconstruct speech.

[0148] The decoder system (700) temporarily stores the encoded data in an input buffer implemented in a memory of the decoder system (700) and configured to receive the encoded data as part of a bitstream (705). From time to time, the encoded data is read from the output buffer by a demultiplexer ("DEMUX") (711) and one or more entropy decoders (710). The decoder system (700) temporarily stores reconstructed speech (775) in an output buffer implemented in a memory of the decoder system (300) and configured to store the reconstructed speech (775) for output. Periodically, sample values ​​(775) in an output frame of the reconstructed speech are read from the output buffer. In some example implementations, for each packet of encoded data that arrives as part of the bitstream (705), the decoder system (700) decodes and buffers (e.g., performs entropy decoding operations, recovers parameter values) the subframe parameters upon arrival of the packet. When an output frame is requested from the decoder system (700), the decoder system (700) decodes one subframe at a time until sufficient output sample values ​​for reconstructing speech (775) have been generated and stored in an output buffer to satisfy the request. The timing of the decoding operation has several advantages. By decoding the subframe parameters as packets arrive, the processor load for the decoding operation is reduced when an output frame is requested. This can reduce the risk of output buffer underflow (data not being available in time for playback due to processing constraints) and allow for tighter scheduling of operations. On the other hand, decoding subframes "on demand" in response to requests increases the likelihood that packets containing encoded data for those subframes have already been received. Alternatively, the decoding operations of the decoder system (700) can follow different timing.

[0149] exist Figure 7 In the embodiment of the present invention, the decoder system (700) uses frames of variable length. Alternatively, the decoder system (700) can use frames of uniform length.

[0150] In some example implementations, the decoder system (700) can reconstruct ultra-wideband speech (based on an input signal sampled at 32kHz) or wideband speech (based on an input signal sampled at 16kHz). In the decoder system (700), if the reconstructed speech (775) is for a wideband signal, processing of the high frequency band by the residual decoder (720), the high frequency band synthesis filter (752), etc. can be skipped, and the filter bank (760) can be bypassed.

[0151] In a decoder system (700), a DEMUX (711) is configured to read encoded data from a bitstream (705) and parse parameters from the encoded data. In conjunction with the DEMUX (711), one or more entropy decoders (710) are configured to entropy decode the decoded parameters to produce quantization parameters (712, 714-719, 737, 738) used by other components of the decoder system (700). For example, the parameters decoded by the entropy decoder (710) can be entropy decoded using a range decoder that uses a cumulative mass function that represents the probability that the value of the parameter is decoded. Alternatively, the quantization parameters (712, 714-719, 737, 738) decoded by the entropy decoder (710) are entropy decoded in some other manner.

[0152] The residual decoder (720) is configured to decode the residual values ​​(727, 728) on a subframe-by-subframe basis, or alternatively, to decode the residual values ​​(727, 728) on a frame-by-frame basis or on some other basis. In particular, the residual decoder (720) is configured to decode a set of phase values ​​and reconstruct the residual values ​​(727, 728) based at least in part on the set of phase values. Figure 8 The decoding stages of the residual values ​​(727, 728) in the residual decoder (720) are shown.

[0153] In some places, the residual decoder (720) includes different processing paths for residual values ​​in different frequency bands. Figure 8 In the embodiment of the present invention, at least after the parameters of the corresponding frequency bands have been reconstructed or generated, the low-band residual values ​​(727) and the high-band residual values ​​(728) are decoded in different paths. In some exemplary implementations, for ultra-wideband speech, the residual decoder (720) produces the low-band residual values ​​(727) and the high-band residual values ​​(728). However, for wideband speech, the residual decoding (720) produces the residual values ​​(727) for one frequency band. Alternatively (for example, if the filter bank (760) combines more than two frequency bands), the residual decoder (720) can decode the residual values ​​for more frequency bands.

[0154] In the decoder system (700), residual values ​​(727, 728) are reconstructed using a model suitable for voiced speech content or a model suitable for unvoiced speech content. The residual decoder (720) includes a decoding stage in the path for voiced speech and a decoding stage in the path for unvoiced speech (not shown). The residual decoder (720) is configured to select one of the paths based on voicing decision information (712) provided to the residual decoder (720).

[0155] If the residual values ​​(727, 728) are for voiced speech, the complex amplitude values ​​are reconstructed using an amplitude decoder (810), a phase decoder (820), and a restoration / smoothing module (840). The complex amplitude values ​​are then transformed by an inverse frequency converter (850) to produce time domain residual values, which are processed by a noise addition module (855).

[0156] The amplitude decoder (810) is configured to reconstruct the set of amplitude values ​​(812) for one or more subframes of the frame using the quantization parameters (715) for the set of amplitude values ​​(812). Depending on the implementation and the inversion operation (with some loss due to quantization) typically performed during encoding, the amplitude decoder (810) can use any of a variety of combinations of inverse quantization operations (e.g., inverse vector quantization, inverse scalar quantization), prediction operations, and domain conversion operations (e.g., conversion from the frequency domain) to decode the set of amplitude values ​​(715) for the corresponding subframe.

[0157] The phase decoder (820) is configured to decode one or more phase value sets (822) using the quantization parameter (716) for the phase value sets (822). The phase value sets can be used for the reconstructed speech (775) in the low frequency band or the entire range. The phase decoder (820) can decode one phase value set (822) for each subframe or one phase value set (822) for a frame. In this case, the phase value set (822) for the frame can represent a phase value determined based on an average or otherwise aggregated complex amplitude value of the subframes of the frame (as described in Section III), and the decoded phase value (822) can be repeated for the corresponding subframes of the frame. Section VI explains the operation of the phase decoder (820) in detail. In particular, the phase decoder (820) can be configured to perform operations to reconstruct at least some phase values ​​(e.g., low frequency phase values, the entire phase value range, or some other phase value range) in the phase value set using a weighted sum of basis functions and a linear component. In this case, the count of the coefficients weighted by the basis function can be based at least in part on a target bit rate for the encoded data. In addition, the phase decoder (820) can be configured to perform operations to synthesize a second subset of the phase value set (e.g., a high frequency phase) using at least some of the first subset of the phase value set (e.g., the low frequency phase values), wherein each phase value of the second subset has a frequency above a cutoff frequency. The cutoff frequency can be determined based at least in part on the target bit rate for the encoded data, the pitch cycle information (722), and / or other criteria. Depending on the cutoff frequency, the high frequency phase values ​​can span the high frequency band, or the high frequency phase values ​​can span a portion of the low frequency band and the high frequency band.

[0158] The recovery and smoothing module (840) is configured to reconstruct complex amplitude values ​​based at least in part on the amplitude value set (812) and the phase value set (814). For example, the phase value set (814) for the frame is converted to the complex domain by using a complex exponential and multiplied by the harmonic amplitude value (812) to generate a complex amplitude value for the low frequency band. The complex amplitude value of the low frequency band can be repeated as the complex amplitude value of the high frequency band. Then, using the high frequency band energy level (714) that is dequantized, the high frequency band complex amplitude value can be scaled so that they more closely approximate the energy of the high frequency band. Alternatively, the recovery and smoothing module (840) can generate complex amplitude values ​​for more frequency bands (e.g., if the filter bank (760) combines more than two frequency bands) or for a single frequency band (e.g., if the filter bank (760) is bypassed or omitted).

[0159] The restoration and smoothing module (840) is also configured to adaptively smooth the complex amplitude values ​​based at least in part on the pitch cycle information (722) and / or differences in amplitude values ​​across boundaries. For example, the complex amplitude values ​​are smoothed across subframe boundaries (including subframe boundaries that are also frame boundaries).

[0160] For smoothing across subframe boundaries, the amount of smoothing can depend on the pitch frequency in adjacent subframes. Pitch cycle information (722) can be signaled per frame and indicates, for example, the subframe length or other frequency information of the subframe. The recovery and smoothing module (840) can be configured to use the pitch cycle information (722) to control the amount of smoothing. In some example implementations, if there is a large change in pitch frequency between subframes, the complex amplitude values ​​will not be smoothed as much due to the presence of real signal changes. On the other hand, if the pitch frequency between subframes does not change much, the complex amplitude values ​​can be smoothed more due to the absence of real signal changes. This smoothing tends to make the complex amplitude values ​​more periodic, thereby producing less noisy speech.

[0161] For smoothing across subframe boundaries, the amount of smoothing may also depend on the amplitude values ​​on either side of the boundary between subframes. In some example implementations, if there is a large change in amplitude values ​​across the boundary between subframes, the complex amplitude values ​​may not be smoothed much due to the presence of real signal changes. On the other hand, if there is not much change in amplitude values ​​across the boundary between subframes, the complex amplitude values ​​may be smoothed more due to the absence of real signal changes. Likewise, in some example implementations, the complex amplitude values ​​may be smoothed more at lower frequencies and less at higher frequencies.

[0162] Alternatively, smoothing the complex amplitude values ​​may be omitted.

[0163] The inverse frequency converter (850) is configured to apply an inverse frequency transform to the complex amplitude values. This produces low-band residual values ​​(857) and high-band residual values ​​(858). In some example implementations, the inverse 1D frequency transform is a variant of an inverse Fourier transform (e.g., inverse DFT, inverse FFT) that has no overlap, or alternatively, has overlap. Alternatively, the inverse 1D frequency transform is some other inverse frequency transform that produces time-domain residual values ​​from the complex amplitude values. The inverse frequency converter (850) can produce residual values ​​for more frequency bands (e.g., if the filter bank (760) combines more than two frequency bands) or a single frequency band (e.g., if the filter bank (760) is bypassed or omitted).

[0164] The correlation / sparseness decoder (830) is configured to decode the correlation value (837) and the sparsity value (838) using one or more quantization parameters (717) for the correlation value (837) and the sparsity value (838). In some example implementations, the correlation value (837) and the sparsity value (838) are recovered using vector quantization indices that collectively represent the correlation value (837) and the sparsity value (838). Examples of correlation values ​​and sparsity values ​​are described in Section III. Alternatively, the correlation value (837) and the sparsity value (838) may be recovered in some other manner.

[0165] The noise adding module (855) is configured to selectively add noise to the residual values ​​(857, 858) based at least in part on the correlation value (837) and the sparsity value (838). In many cases, adding noise can reduce metallic sounds (775) in the reconstructed speech.

[0166] In general, the correlation value (837) may be used to control how much noise (if any) is added to the residual values ​​(857, 858). In some example implementations, if the correlation value (837) is high (the signal is harmonic), little or no noise is added to the residual values ​​(857, 858). In this case, the model for encoding / decoding voiced content tends to work well. On the other hand, if the correlation value (837) is low (the signal is not harmonic), more noise is added to the residual values ​​(857, 858). In this case, the model for encoding / decoding voiced content does not work well (e.g., because the signal is not periodic and therefore averaging is not appropriate).

[0167] In general, the sparsity value (838) can be used to control where the noise is added (e.g., how the added noise is distributed around the pitch pulse). In general, noise is added at locations where it improves the perceived quality. For example, noise is added at strong non-zero pitch pulses. For example, if the energy of the residual values ​​(857, 858) is sparse (indicated by a high sparsity value), noise is added around the strong non-zero pitch pulse, but no noise is added to the remaining residual values ​​(857, 858). On the other hand, if the energy of the residual values ​​(857, 858) is not sparse (indicated by a low sparsity value), the noise is more evenly distributed throughout the residual values ​​(857, 858). Also, in general, more noise can be added at higher frequencies than at lower frequencies. For example, an increased amount of noise is added at higher frequencies.

[0168] exist Figure 8In the embodiment of the present invention, the noise adding module (855) adds noise to the residual values ​​for two frequency bands. Alternatively, the noise adding module (855) can add noise to the residual values ​​for more frequency bands (e.g., if the filter bank (760) combines more than two frequency bands) or a single frequency band (e.g., if the filter bank (760) is bypassed or omitted).

[0169] If the residual values ​​(727, 728) are for unvoiced speech, the residual decoder (720) includes one or more different processing paths (not shown) for the residual values. Depending on the implementation and the inversion operations typically performed during encoding (with some loss due to quantization), the unvoiced path in the residual decoder (720) can use any of a variety of combinations of inverse quantization operations (e.g., inverse vector quantization, inverse scalar quantization), energy / noise substitution operations, and filtering operations to decode the residual values ​​(727, 728) for unvoiced speech.

[0170] exist Figure 7 and Figure 8 , the residual encoder (720) is shown processing low-band residual values ​​(727) and high-band residual values ​​(728). Alternatively, the residual encoder (380) can process residual values ​​in more frequency bands or a single frequency band (e.g., if the filter bank (760) is bypassed or omitted).

[0171] return Figure 7 In the decoder system (700), the LPC recovery module (740) is configured to reconstruct the LP coefficients for the corresponding frequency band (or if there are no multiple frequency bands, the entire reconstructed speech). Depending on the implementation and the inverse operation usually performed during encoding (with some loss due to quantization), the LPC recovery module (740) can reconstruct the LP coefficients using any of various combinations of inverse quantization operations (e.g., inverse vector quantization, inverse scalar quantization), prediction operations, and domain conversion operations (e.g., conversion from the LSF domain).

[0172] Figure 7 The decoder system (700) includes two synthesis filters (360, 362), such as filter A -1 (z). The synthesis filters (750, 752) are configured to filter the residual values ​​(727, 728) according to the reconstructed LP coefficients. The filtering converts the low-band residual values ​​(727) and the high-band residual values ​​(728) to the speech domain, thereby producing reconstructed speech (757) for the low-band and reconstructed speech for the high-band (758). Figure 7In the embodiment of the present invention, the low-band synthesis filter (750) is configured to filter the low-band residual value (727) (if the filter bank (760) is bypassed, it is used for the entire range of reconstructed speech (775)) according to the restored low-band LP coefficients. The high-band synthesis filter (752) is configured to filter the high-band residual value (728) according to the restored high-band LP coefficients. If the filter bank (760) is configured to combine more frequency bands into the reconstructed speech (775), the decoder system (700) may include more synthesis filters for the corresponding frequency bands. If the filter bank (760) is omitted, the decoder system (700) may include a single synthesis filter for the entire range of reconstructed speech (775).

[0173] The filter bank (760) is configured to combine a plurality of frequency bands (757, 758) obtained by filtering the residual values ​​(727, 728) in the corresponding frequency bands through the synthesis filters (750, 752), thereby generating reconstructed speech (765). Figure 7 In the embodiment of the present invention, the filter bank (760) is configured to combine two equal frequency bands, namely a low frequency band (757) and a high frequency band (758). For example, if the reconstructed speech (775) is used for an ultra-wideband signal, the low frequency band (757) may include speech in the range of 0-8kHz, and the high frequency band (758) may include speech in the range of 8-16kHz. Alternatively, the filter bank (760) combines more frequency bands and / or unequal frequency bands to synthesize the reconstructed speech (775). Depending on the implementation, the filter bank (760) may use any of various types of HR or other filters.

[0174] The post-processing filter (770) is configured to selectively filter the reconstructed speech (765) to produce the reconstructed speech (775) for output. Alternatively, the post-processing filter (770) can be omitted and the reconstructed speech (765) from the filter bank (760) can be output. Alternatively, if the filter bank (760) is also omitted, the output from the synthesis filter (750) provides the reconstructed speech for output.

[0175] Depending on the implementation and the type of compression desired, modules of the decoder system (700) may be added, omitted, divided into multiple modules, combined with other modules, and / or replaced with similar modules. In alternative embodiments, decoders having different modules and / or other configurations of modules perform one or more of the described techniques. Specific embodiments of decoders typically use variants or supplementary versions of the decoder system (700). The relationships shown between modules within the decoder system (700) indicate the general flow of information in the decoder system (700); for simplicity, other relationships are not shown.

[0176] VI. Example of phase reconstruction in a speech decoder.

[0177] This section describes innovations in phase reconstruction during speech decoding. In many cases, the innovations can improve the performance of speech codecs in low bitrate scenarios, even when the encoded data is delivered over networks with insufficient bandwidth or transmission quality issues. The innovations described in this section fall into two major groups of innovations, which can be used individually or in combination.

[0178] According to the first group of innovations, when the speech decoder decodes the phase value set, the speech decoder uses the weighted sum of the basis function and the linear component to reconstruct at least some of the phase values ​​in the phase value set. Using the weighted sum of the basis function and the linear component, the phase value can be represented in a compact and flexible manner, which can improve the rate-distortion performance under low bit rate conditions. The speech decoder can decode the coefficient set weighted to the basis function, and then use the coefficient set when reconstructing the phase value. The speech decoder can also decode and use the offset value, slope value and / or other parameters that define the linear component. The coefficients weighted to the basis function can be predefined and remain unchanged. Alternatively, in order to provide flexibility for encoding / decoding speech with different target bit rates, the count of the coefficients can depend on the target bit rate.

[0179] According to the second group of innovations, when the speech decoder decodes the set of phase values, the speech decoder reconstructs low-frequency phase values ​​(which are below the cutoff frequency) and then uses at least some of the low-frequency phase values ​​to synthesize high-frequency phase values ​​(which are above the cutoff frequency). By synthesizing high-frequency phase values ​​based on the reconstructed low-frequency phase values, the speech decoder can efficiently reconstruct the entire range of phase values, which can improve rate-distortion performance in low bit rate scenarios. The cutoff frequency can be predefined and unchanged. Alternatively, in order to provide flexibility for encoding / decoding speech at different target bit rates or encoding / decoding speech with different characteristics, the speech decoder can determine the cutoff frequency based at least in part on the target bit rate of the encoded data, pitch cycle information, and / or other criteria.

[0180] A. Use the weighted sum of basis functions to reconstruct the phase value.

[0181] When decoding a set of phase values, the speech decoder can reconstruct the set of phase values ​​using a weighted sum of basis functions. For example, when the basis function is a sine function, the quantized set of phase values ​​Pi is defined as: Where 0≤i≤I-1

[0182] Where N is the count of the quantized coefficients (hereinafter referred to as “coefficients”) weighted by the basis function, and K nis one of the coefficients, and I is a count of complex amplitude values ​​(and thus frequency groupings of phase values). In some example implementations, the basis functions are sine functions, but the basis functions may alternatively be cosine functions or some other type of basis functions. The set of phase values ​​reconstructed from the quantized values ​​may be low frequency phase values ​​(if high frequency phase values ​​have been discarded, as described in the previous sections), a full range of phase values ​​(if high frequency phase values ​​have not been discarded), or some other range of phase values. The set of phase values ​​decoded may be a set of phase values ​​for a frame or a set of phase values ​​for a subframe of a frame.

[0183] Using the quantized phase value set P i (weighted sum of basis functions) and linear components to define the final quantized phase value set P final_i The linear component can be defined as a×i+b, where a represents the slope value and b represents the offset value. For example, P final_i =P i +a×i+b. Alternatively, other and / or additional parameters may be used to define the linear component.

[0184] To reconstruct the phase value set, the speech decoder performs the quantized coefficient set K n Perform entropy decoding. Coefficient K n Weight the basis functions. In some example implementations, K n The values ​​of are quantized to integer values. For example, the coefficient K n The value of is an integer value constrained in magnitude as follows.

[0185] If n = 1, then |K n |≤5

[0186] If n = 2, then |K n |≤3

[0187] If n = 3, then |K n |≤2

[0188] If n ≥ 4, then |K n |≤1.

[0189] Alternatively, the coefficient K can be limited according to other constraints n The value of .

[0190] Although the coefficient K n The count N can be predetermined and constant, but the coefficient K can be adaptively changed. n To provide flexibility for encoding / decoding speech at different target bit rates, the speech decoder may determine the coefficient K based at least in part on the target bit rate for the encoded data. nFor example, depending on the target bit rate, the speech decoder can set the coefficient K n The count N of complex amplitude values ​​is determined as a fraction of the count I of complex amplitude values ​​(the count of frequency groups with phase values). In some example implementations, the fraction ranges from 0.29 to 0.51. Alternatively, the fraction may have some other range. If the target bit rate is high, the coefficient K n The count N is high (i.e., the coefficient K n If the target bit rate is low, the coefficient K n The count N is low (i.e., the coefficient K n The speech decoder can use a lookup table that associates different coefficient counts with different target bit rates to determine the coefficient K. n Alternatively, the speech decoder may determine the coefficient K in some other manner according to rules, logic, etc. n The count N is as long as the coefficient K is set similarly at the corresponding speech encoder n The count of N is sufficient. The coefficient K n The count N of K may also depend on the average pitch frequency and / or other criteria. The speech decoder may determine the coefficient K on a frame-by-frame basis (eg, set to the average pitch frequency variation) or on some other basis. n The count N.

[0191] In addition to the reconstruction coefficient set K n In addition, the speech decoder also decodes the parameters of the linear component. For example, the speech decoder decodes the offset value b and the slope value a used to reconstruct the linear component. The offset value b represents the linear phase (offset) relative to the start of the weighted sum of the basis functions, so that the result P final_i The slope value a represents the overall slope, which is used as a multiplier or scaling factor for the linear component so that the result P final_i The original phase signal is more closely approximated. After entropy decoding the offset value, slope value and / or other values, the speech decoder inverse quantizes the values. Alternatively, the speech decoder can decode other and / or additional parameters for the weighted sum of the linear components or basis functions.

[0192] In some example implementations, a residual decoder in a speech decoder determines a count of coefficients for weighting a basis function based at least in part on a target bit rate for the encoded data. The residual decoder decodes the set of coefficients, the offset value, and the slope value. The residual decoder then uses the set of coefficients, the offset value, and the slope value to reconstruct an approximation of the phase value. The residual decoder applies the coefficients K n To obtain the weighted sum of the basis functions, for example, multiply the sine function by the coefficient K nThen, the residual decoder applies the slope value and the offset value to reconstruct the linear component, for example, multiplying the frequency by the slope value and adding the offset value. Finally, the residual decoder combines the weighted sum of the basis functions and the linear component. B. Synthesize high frequency phase values.

[0193] When decoding the phase value set, the speech decoder can use at least some of the low-frequency phase values ​​to reconstruct low-frequency phase values ​​below the cutoff frequency and synthesize high-frequency phase values ​​above the cutoff frequency. The decoded phase value set can be a phase value set for a frame or a phase value set for a subframe of a frame. The low-frequency phase values ​​can be reconstructed using the weighted sum of basis functions (as described in the previous section) or in some other way. The synthesized high-frequency phase values ​​can partially or completely replace the high-frequency phase values ​​discarded during encoding. Alternatively, the synthesized high-frequency phase values ​​can extend beyond the frequency of the discarded phase values ​​to higher frequencies.

[0194] Although the cut-off frequency can be predetermined and constant, there are advantages in adaptively changing the cut-off frequency. For example, in order to provide flexibility for encoding / decoding speech with different target bit rates or encoding / decoding speech with different characteristics, the speech decoder can determine the cut-off frequency based at least in part on the target bit rate and / or pitch cycle information of the encoded data, and the pitch cycle information can indicate the average pitch frequency. For example, if the frame includes high-frequency speech content, a higher cut-off frequency is used. On the other hand, if the frame only includes low-frequency speech content, a lower cut-off frequency is used. With respect to the target bit rate, if the target bit rate is low, the cut-off frequency is low. If the target bit rate is high, the cut-off frequency is high. In some example implementations, the cut-off frequency falls within the range of 962 Hz (for low target bit rates and low average pitch frequencies) to 4160 Hz (for high target bit rates and high average pitch frequencies). Alternatively, the cut-off frequency can vary within some other range and / or depend on other criteria.

[0195] The speech decoder can determine the cut-off frequency on a frame-by-frame basis. For example, the speech decoder can determine the cut-off frequency of a frame as the average pitch frequency variation from frame to frame, even if the target bit rate variation is not so frequent. Alternatively, the cut-off frequency can be changed on some other basis and / or depending on other criteria. The speech decoder can determine the cut-off frequency using a look-up table that associates different cut-off frequencies with different target bit rates and average pitch frequencies. Alternatively, the speech decoder can determine the cut-off frequency in some other manner according to rules, logic, etc., as long as the cut-off frequency is similarly set at the corresponding speech encoder.

[0196] Depending on the implementation, the phase value exactly at the cutoff frequency may be considered as one of the high frequency phase values ​​(synthesized) or one of the low frequency phase values ​​(reconstructed from the quantization parameter in the bitstream).

[0197] The high frequency phase values ​​may be synthesized in various ways, depending on the implementation. Figures 9a to 9c Features (901-903) of an exemplary method for synthesizing high frequency phase values ​​having a frequency above a cutoff frequency are shown. Figure 9a-9c In the simplified example of , the low-frequency phase values ​​include 12 phase values: 5 6 6 5 7 8 9 10 11 10 12 13.

[0198] To synthesize high frequency phase values, the speech decoder identifies a range of low frequency phase values. In some example implementations, the speech decoder identifies the upper half of the frequency range of the low frequency phase values ​​that have been reconstructed to potentially add or remove phase values ​​to have an even number of harmonics. Figure 9a In the simplified example of , the upper half of the low-frequency phase values ​​comprises six phase values: 9 10 11 10 12 13. Alternatively, the speech decoder may recognize some other range of low-frequency phase values ​​that have been reconstructed.

[0199] The speech decoder repeats the phase value based on the low frequency phase value in the identified range, which starts at the cutoff frequency and continues to the last phase value in the set of phase values. The low frequency phase value in the identified range may be repeated one or more times. If the repetition of the low frequency phase value in the identified range is not completely aligned with the end of the phase spectrum, the low frequency phase value in the identified range may be partially repeated. Figure 9b In , low-frequency phase values ​​within the identified range are repeated to generate high-frequency phase values ​​until the final phase value. Simply repeating low-frequency phase values ​​within the identified range may cause abrupt changes in the phase spectrum, but in typical cases, such abrupt changes are not found in the original phase spectrum. Figure 9b In , six phase values ​​are repeated: 9 10 11 10 12 13 causing the phase value to suddenly drop from 13 to 9 twice: 5 6 6 5 7 8 9 10 11 10 12 139 10 11 10 12 13 9 10 11 10 12 13.

[0200] To address this issue, the speech decoder can determine (as a pattern) the difference between adjacent phase values ​​within the identified low-frequency phase value range. That is, for each phase value within the identified low-frequency phase value range, the speech decoder can determine the difference relative to the previous phase value (in frequency order). The speech decoder can then repeat the phase value difference, starting from the cutoff frequency and continuing to the last phase value in the phase value set. The phase value difference can be repeated one or more times. If the repetition of the phase value difference is not completely aligned with the end of the phase spectrum, the phase value difference can be partially repeated. After repeating the phase value difference, the speech decoder can integrate the phase value differences between adjacent phase values ​​to generate a high-frequency phase value. That is, for each high-frequency phase value, starting from the cutoff frequency, the speech decoder can add the corresponding phase value difference to the previous phase value (in frequency order). In Fig.9c For example, for the six phase values ​​in the identified range (9 10 1110 12 13), the phase value difference is +1+1+1-1+2+1. From the cutoff frequency to the end of the phase spectrum, the phase value difference is repeated twice: 5 6 6 5 7 8 9 10 11 10 12 13+1+1+1-1+2+1+1+1+1-1+2+1. The phase value differences are then integrated to generate high-frequency phase values: 5 6 6 5 7 8 9 10 11 1012 13 14 15 16 15 17 18 19 20 2120 22 23.

[0201] In this way, the speech decoder can reconstruct phase values ​​for the reconstructed speech of the whole range. For example, if the reconstructed speech is the ultra-wideband speech that has been divided into low-frequency band and high-frequency band, the speech decoder can use the reconstructed phase value below the cut-off frequency in the low-frequency band to synthesize the phase value for a part of low-frequency band (higher than the cut-off frequency) and all high-frequency bands. Alternatively, the speech decoder can use the reconstructed phase value lower than the cut-off frequency in the low-frequency band, only for a part of low-frequency band (higher than the cut-off frequency) synthesized phase value.

[0202] Alternatively, in some other way, the speech decoder may use at least some of the already reconstructed low-frequency phase values ​​to synthesize high-frequency phase values.

[0203] C. Example techniques for phase reconstruction in speech decoding.

[0204] Fig.10a A general technique for speech decoding is shown (1001), which may include, for example, Fig.10b , Fig.10c or Fig.10d Additional operations shown. Fig.10bA general technique for speech decoding is shown (1002), which includes reconstructing a phase value represented using a weighted sum of basis functions and a linear component. Fig.10c A general technique for speech decoding is shown (1003), which includes synthesizing phase values ​​having frequencies above a cutoff frequency. Fig.10d A more specific example technique (1004) for speech decoding is shown, which includes: reconstructing low-frequency phase values ​​(which are below a cutoff frequency) represented using a weighted sum of basis functions and a linear component, and synthesizing high-frequency phase values ​​(which are above the cutoff frequency). Figure 7 and Figure 8 The described speech decoder or another speech decoder may perform the techniques (1001 to 1004).

[0205] refer to Fig.10a , the speech decoder receives (1010) the encoded data as part of a bitstream. For example, an input buffer implemented in a memory of a computer system is configured to receive and store the encoded data as part of a bitstream.

[0206] The speech decoder decodes the coded data (1020) to reconstruct the speech. As part of decoding (1020), the speech decoder decodes the residual values ​​and filters the residual values ​​according to the linear prediction coefficients. The residual values ​​can, for example, be used for the frequency bands of the reconstructed speech that are combined by the filter bank later. Alternatively, the residual values ​​can be used for the reconstructed speech that is not in multiple frequency bands. In any case, filtering produces the reconstructed speech, which can be further processed. Figure 10b-10d An example of operations that may be performed as part of the decoding (1020) stage is shown.

[0207] The speech decoder stores (1040) the reconstructed speech for output. For example, an output buffer implemented in a memory of a computer system is configured to store the reconstructed speech for output.

[0208] refer to Fig.10b , the speech decoder decodes (1021) a set of phase values ​​for residual values. The set of phase values ​​may be for a subframe of residual values ​​or for a frame of residual values. When decoding (1021) the set of phase values, the speech decoder reconstructs at least some of the phase values ​​in the set of phase values ​​using a weighted sum of basis functions and a linear component. For example, the basis function is a sine function. Alternatively, the basis function is a cosine function or some other basis function. The phase value represented as a weighted sum of basis functions may be a low frequency phase value (if the high frequency phase value has been discarded), a whole range of phase values, or some other range of phase values.

[0209] In order to decode the phase value set, the speech decoder can decode the coefficient set weighted to the basis function, and decode the offset value and slope value that parameterize the linear component, and then use the coefficient set, offset value and slope as part of the reconstruction of at least part of the phase value set. Alternatively, the speech decoder can use the coefficient set weighted to the basis function together with some other combination of parameters defining the linear component (e.g., no offset value or no slope value, or using one or more other parameters) to decode the phase value set. Alternatively, in combination with the coefficient set weighted to the basis function and the linear component, the speech decoder can also use other parameters to reconstruct at least some of the phase values ​​in the phase value set. The speech decoder can determine the count of the coefficients weighted to the basis function based at least in part on the target bit rate of the encoded data (as described above) and / or other criteria.

[0210] The speech decoder reconstructs (1035) the residual value based at least in part on the phase value set. For example, if the phase value set is for a frame, the speech decoder repeats the phase value set for one or more subframes of the frame. Then, based at least in part on the repeated phase value set for the corresponding subframe, the speech decoder reconstructs the complex amplitude value for the corresponding subframe. Finally, the speech decoder applies an inverse frequency transform to the complex amplitude value for the corresponding subframe. The inverse frequency transform can be a variant of an inverse Fourier transform (e.g., inverse DFT, inverse FFT), or some other inverse frequency transform to reconstruct the residual value from the complex amplitude value. Alternatively, the speech decoder reconstructs the residual value in some other way, for example, by reconstructing the phase value for the entire frame that has not yet been divided into subframes, and applying an inverse frequency transform to the complex amplitude value for the entire frame.

[0211] refer to Fig.10c , the speech decoder decodes (1025) a set of phase values. The set of phase values ​​can be for a subframe of residual values ​​or for a frame of residual values. When decoding (1025) the set of phase values, the speech decoder reconstructs a first subset of the set of phase values ​​(e.g., low-frequency phase values) and uses at least some of the phase values ​​in the first subset of phase values ​​to synthesize a second subset of the set of phase values ​​(e.g., high-frequency phase values). Each phase value in the second subset of phase values ​​has a frequency higher than a cutoff frequency. The speech decoder can determine the cutoff frequency based at least in part on a target bit rate for the encoded data, pitch cycle information, and / or other criteria. Depending on the implementation, the phase value exactly at the cutoff frequency can be considered as one of the high-frequency phase values ​​(synthesized) or one of the low-frequency phase values ​​(reconstructed based on a quantization parameter in the bitstream).

[0212] When using at least some of the phase values ​​in the first subset of phase values ​​to synthesize the second subset of phase values, the speech decoder can determine a pattern within the range of the first subset and then repeat the pattern above the cutoff frequency. For example, the speech decoder can identify the range and then determine the adjacent phase values ​​within the range as the pattern. In this case, the adjacent phase values ​​within the range are repeated after the cutoff frequency to generate the second subset. Or, as another example, the speech decoder can identify the range and then determine the difference between adjacent phase values ​​within the range as the pattern. In this case, the speech decoder can repeat the phase value difference above the cutoff frequency and then integrate the difference between adjacent phase values ​​after the cutoff frequency to determine the second subset.

[0213] The speech decoder reconstructs (1035) the residual value based at least in part on the set of phase values. Fig.10b Reconstruct the residual values ​​as described.

[0214] exist Fig.10d In the example technique (1004), when decoding a set of phase values ​​for a residual value, the speech decoder reconstructs low-frequency phase values ​​(which are below a cutoff frequency) represented as a weighted sum of basis functions, and synthesizes high-frequency phase values ​​(which are above the cutoff frequency).

[0215] The speech decoder decodes the coefficient set, the offset value, and the slope value (1022). The speech decoder reconstructs (1023) the low frequency phase value using a weighted sum of basis functions weighted according to the coefficient set and adjusted according to the linear component (based on the slope value and the offset value).

[0216] To synthesize the high frequency phase value, the speech decoder determines (1024) a cutoff frequency based on the target bit rate and / or the pitch cycle information. The speech decoder determines (1026) a pattern of phase value differences within a range of low frequency phase values. The speech decoder repeats (1027) the pattern above the cutoff frequency and then integrates (1028) the phase value differences between adjacent phase values ​​to determine the high frequency phase value. Depending on the implementation, the phase value exactly at the cutoff frequency may be considered as one of the high frequency phase values ​​(synthesized) or one of the low frequency phase values ​​(reconstructed based on a quantization parameter in the bitstream).

[0217] To reconstruct the residual values, the speech decoder (1029) repeats a set of phase values ​​for a subframe of a frame. Then, based at least in part on the repeated set of phase values, the speech decoder reconstructs (1030) complex amplitude values ​​for the subframes. Finally, the speech decoder applies (1031) an inverse frequency transform to the complex amplitude values ​​for the corresponding subframes, thereby generating residual values.

[0218] In view of the many possible embodiments to which the principles of the disclosed invention may be applied, it should be recognized that the illustrated embodiments are merely preferred examples of the invention and should not be considered as limitations on the scope of the invention. Instead, the scope of the invention is defined by the appended claims. We therefore claim as our invention all that falls within the scope and spirit of these claims.

Claims

1. A method in a computer system implementing a speech encoder, comprising: receiving (610) voice input; Encoding the speech input (620) to generate encoded data includes: filtering the input value based on the speech input according to the linear prediction coefficient to generate a residual value; Encoding the residual value comprises: determining (621) a set of phase values; and encoding (635) the set of phase values, including omitting any phase values ​​in the set of phase values ​​having a frequency above a cutoff frequency; and The encoded data is stored for output as part of a bitstream.

2. The method according to claim 1, wherein: Encoding the set of phase values ​​further includes selecting the cutoff frequency based at least in part on a target bit rate and / or pitch cycle information for the encoded data.

3. The method according to claim 1, wherein: Determining the phase value set includes: applying a frequency transform to one or more subframes of the current frame, thereby generating complex amplitude values ​​for the corresponding subframes; aggregating the complex amplitude values ​​of the corresponding subframes; and The set of phase values ​​is calculated based at least in part on the aggregated complex amplitude values.

4. The method according to claim 1, wherein: Encoding the set of phase values ​​further includes representing at least some of the set of phase values ​​using a weighted sum of basis functions and a linear component.

5. One or more computer-readable media having computer-executable instructions stored thereon, which, when programmed by one or more processors, are used to cause the one or more processors to perform the method according to any one of claims 1-4.

6. A computer system comprising: an input buffer implemented in a memory of the computer system and configured to receive speech input; A speech encoder (300), which is implemented using one or more processors of the computer system, is configured to encode the speech input to generate encoded data, the speech encoder comprising: one or more prediction filters (360, 362) configured to filter input values ​​based on the speech input according to linear prediction coefficients to produce residual values; A residual encoder (380) is configured to encode the residual value, wherein the residual encoder is configured to: determining a set of phase values; and Encoding the set of phase values ​​includes performing operations to: omit any phase values ​​in the set of phase values ​​having a frequency above a cutoff frequency; and An output buffer, implemented in a memory of the computer system, is configured to store the encoded data for output as part of a bitstream.

7. The computer system according to claim 6, wherein: The residual encoder is further configured to select the cutoff frequency based at least in part on a target bit rate and / or pitch cycle information of the encoded data.

8. The computer system according to claim 6, wherein: To encode the set of phase values, the residual encoder is further configured to perform the following operations as part of representing at least some of the phase values ​​in the set of phase values ​​using a weighted sum of basis functions and a linear component: Determining a set of coefficients for weighting the basis functions using a delayed decision method; setting a count of coefficients weighting the basis functions based at least in part on a target bit rate for the encoded data; and / or Scores for the set of candidate coefficients weighted for the basis functions are determined using a cost function based at least in part on a linear phase metric.

9. The computer system according to claim 6, wherein: The speech encoder also includes: A filter bank configured to separate the speech input into a plurality of frequency bands, wherein the plurality of frequency bands provide input values ​​that are filtered by the one or more prediction filters to produce the residual values ​​in corresponding frequency bands, wherein the set of phase values ​​is determined and encoded for a low frequency band in the corresponding frequency bands of the residual values, and wherein the residual value encoder is further configured to measure energy levels for a high frequency band in the corresponding frequency bands of the residual values.

10. The computer system according to claim 6, wherein: The speech encoder also includes one or more of the following: (a) one or more LPC analysis modules configured to determine the linear prediction coefficients, and one or more quantization modules configured to quantize the linear prediction coefficients; (b) a pitch analysis module, configured to perform pitch analysis to generate pitch cycle information, wherein the pitch cycle information is a set of subframe lengths corresponding to the pitch cycle; (c) a voicing decision module configured to perform voicing analysis to generate voicing decision information; and (d) a framer configured to organize the residual values ​​into variable length frames, wherein the framer is configured to: (1) setting a framing strategy based at least in part on voicing and unvoicing decision information, wherein the framing strategy is voiced or unvoiced; and (2) setting a frame length and a subframe length of one or more subframes, including: if the framing strategy is voiced speech, setting the subframe length based at least in part on pitch cycle information, so that each of the corresponding subframes includes a set of residual values ​​for a pitch cycle to facilitate encoding in a pitch synchronous manner, and setting the frame length to an integer count of the corresponding subframe.

11. The computer system according to claim 6, wherein: The residual value encoder is further configured to perform the following operations for the current frame: applying a one-dimensional frequency transform to one or more subframes of the current frame, thereby generating complex amplitude values ​​for the corresponding subframes; determining a set of amplitude values ​​for the corresponding subframe based at least in part on the complex amplitude value for the corresponding subframe; encoding the set of amplitude values ​​of the corresponding subframe; Encode the sparsity value; as well as Encodes the relevance value.

Images