Multistage Vector Quantization for Audio Coding

Abstract

A method implemented by an encoder is disclosed that includes obtaining a discrete cosine transform (DCT) target vector (1201), performing a suboptimal pairwise interior search in each segment of a codebook (104) having a plurality of segments, each segment having a truncated vector different from the truncated vectors of other segments of the plurality of segments, to determine a pairwise initial candidate set from each of the plurality of segments and form a plurality of pairwise initial candidates from the suboptimal pairwise interior search (1203), where the DCT target vector is a target of the suboptimal pairwise interior search in each of the plurality of segments, reconstructing the plurality of final candidates using an inverse type-II discrete cosine transform (DCT-II) to transform the plurality of final candidates into final candidate data in an original domain (1209), and providing the final candidate data to a second stage of a multi-stage vector quantizer (1209).

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of U.S. Patent Application No. 63 / 454,166, filed March 23, 2023, which claims the benefit of U.S. Provisional Patent Application No. 63 / 448,440, filed February 27, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates generally to encoding and decoding, and more particularly to multi-stage vector quantization methods and related encoders and / or decoders that support multi-stage vector quantization. [Background technology]

[0003] Vector quantization (VQ) is widely used in the development of data compression technology. In particular, VQ is an efficient data compression technique that organizes multiple scalar data columns into vectors and performs the entire quantization in the vector space. As a result, the data is compressed without losing much information.

[0004] A probabilistic codebook trained on 24 coefficients / vectors of general 37-bit dimension is 2 37 *24=3*2 40 =3.29*10 12 This would require read-only memory (ROM) storage of Word16. Designing, searching, and storing such a codebook is impractical in most real-world implementations.

[0005] In the 1980s and 1990s, the concepts of Split-VQ and Multistage VQ (MSVQ) were developed to reach more reasonable storage and retrieval complexity while still maintaining significant vector quantization gains. In the training of these VQs, the coefficient correlation property is still properly exploited.

[0006] An exemplary split VQ is the adaptive multi-rate wideband (AMR-WB) immittance spectrum frequency (ISF) VQ described in 3GPP Technical Standard (TS) 26.190, section "5.2.5 Quantization of the ISP coefficients." Here, the total number of coefficients is 16, which is split into 9 coefficients (8 bits) and 7 coefficients (8 bits) in the first stage. Similarly, in the second stage, the VQ is further split into {3,3,3} coefficients using (6,7,7) bits, followed by a higher stage 2 split of {3,4} using (5,5) bits. A total of 8+8+6+7+7+5+5=46 bits are used, in which case the total number of ROM table entries is, in Matlab syntax, sum((2.^

[0088] ).*

[0097] )+sum((2.^

[0677] ).*

[0333] )+sum((2.^

[0055] ).*

[0034] )=5.2kW(Word16), i.e. a much more realistic number of table ROMs.

[0007] A less complex alternative to trained probabilistic (SplitVQ, MSVQ) codebooks is a codebook with a given algebraic or lattice structure, which is easier to search; however, such codebooks have suboptimal Voronoi regions and are generally only efficient for certain input vector distributions (e.g., Gaussian, Laplacian, etc.). Furthermore, efficient indexing schemes (e.g., to construct the final codebook vectors from the bitstream) can be costly, and even more so, lattice structures are generally not very flexible with respect to vector length. Examples of lattice quantizers are D8 lattices, RE8 lattices, and pyramidal VQ (PVQ). Spherical PVQ is also used in Enhanced Voice Services (EVS) and Internet Engineering Task Force (IETF) Opus to encode Gaussian sources.

[0008] Another possibility is to transform the input signal into a better domain for efficient and rapid quantization. For example, a signal-optimized Karhunen Love transform (KLT) or the more general discrete cosine transform (DCT) can be used to obtain energy relations (i.e., via the KLT) or frequency relations (i.e., via the DCT) for the input signal. Summary of the Invention

[0009] Currently, certain challenges exist: Current solutions for the first stage (and subsequent stages) of MSVQ require large storage space. For example, in the case of EVS, the frequency domain-comfort noise generation (FD-CNG) VQ in the first stage uses 128 levels (7 bits) with 24 coefficients each, resulting in (128 x 24) = 3072 words of ROM storage (i.e., generally, a word can be a 16-bit Word16 integer or a word can be a single-precision 32-bit float). In an EVS-FD-CNG-VQ (Enhanced Voice Services-Frequency Domain-Comfort Noise Generation-Vector Quantization) implementation, this corresponds to 6 kilobytes (kB), with each byte being 8 bits.

[0010] The entire EVS 37-bit (7+5*6)-bit FD-CNG VQ (all stages) uses (128+5*64)*24=10752 Word16s or 20 kilobytes, which is twice the ROM size of the 46-bit AMR-WB ISF-VQ.

[0011] In the case of EVS FD-CNG-VQ, the unstructured first stage uses almost 30% of the quantizer's ROM space and also 19% (7 bits / 37 bits) of the information space.

[0012] The absence of an efficient structure in the first stage makes it difficult to provide a low storage space and low complexity search solution.

[0013] The iterative updating of Nc (i.e., the number of candidates to retain for the second stage) per current vector has a high worst-case "weighted millions of operations per second" (WMOPS) complexity because an existing list of Nc (8 in the case of EVS) is potentially updated for every analyzed codebook vector; i.e., an Nc-length candidate list may have to be updated 128 times.

[0014] Some aspects of the present disclosure and their embodiments may provide solutions to these or other problems: A transformed segmented structure with different truncation lengths in each segment (e.g., corresponding to different levels of detail) is introduced into a first-stage codebook.

[0015] Each segment with its respective truncation length is efficiently searched in an optimized inner loop that keeps and / or maintains the best candidate pairs (e.g., a list of two candidates) for each segment.

[0016] The number of segments Ns in the first stage is kept at about Ns≈Nc / 2, so that at the end of the search, approximately Nc=2*Ns candidates will be available as needed.

[0017] To further improve the quality of the remaining Nc candidates, the list of Nc candidates is updated by using the stored circular list of nearest first-stage vectors, based on evaluating the first and second best candidate neighbors (e.g., in terms of mean squared error (MSE)) against the worst existing entries in the original Nc list of candidates.

[0018] To further improve the storage and retrieval characteristics of the first stage, the actual coefficients of the vectors of the segment are stored in the byte (Word8) quantization domain with a special scaling (e.g., implemented as a shift factor) for each column of the segment.

[0019] According to some embodiments, the disclosed subject matter includes a method implemented by an encoder, including obtaining a discrete cosine transform (DCT) target vector. The method further includes performing a suboptimal pairwise interior search in each segment of a codebook having a plurality of segments, each segment having a truncated vector different from the truncated vectors of other segments of the plurality of segments, to determine a pairwise initial candidate set from each of the plurality of segments and form a plurality of pairwise initial candidates from the suboptimal pairwise interior search. The method includes performing a post-optimization on the plurality of pairwise initial candidates to replace several of the plurality of pairwise initial candidates with a list of candidate neighbors, thereby forming a plurality of final candidates. The method includes reconstructing the plurality of final candidates using an inverse type-II discrete cosine transform (DCT-II) to transform the plurality of final candidates into a plurality of reconstructed final candidates in the original domain. The method includes providing the plurality of reconstructed final candidates to a second stage of a multi-stage vector quantizer.

[0020] Some embodiments may provide one or more of the following technical advantage(s): The introduced codebook structure (e.g., transformed into N segments and selectively truncated) enables fast search using fewer operations (e.g., WMOPS).

[0021] Pairwise candidate extraction within each codebook segment enables a lower complexity (WMOPS) inner loop of the first stage by efficiently swapping in and out of new best entries in pairs (e.g., lists of two entries).

[0022] The codebook vectors are retrieved from memory (e.g., ROM) using specific scaling (exponents stored as shift factors) for each segment and coefficient column. Each codebook vector mantissa value (e.g., "segment codebook vector" value) is represented by a signed byte (Word8), which further enables fast ROM read access as a pair of bytes (Word16) (e.g., efficient Word8 mantissa retrieval in a digital signal processor (DSP) if the vector coefficient truncation length is even for all segments). Furthermore, the segmented stage 1 codebook search enables parallelization of the VQ inner loop of the first stage (e.g., stage #1). The Word8 and shift factor representation of each coefficient means that the coefficients are selectively truncated in the dynamic range for each column. Truncation for each segment means that each vector is also truncated at the detail level. When using DCT, the truncation corresponds to removing some high-frequency DCT components. The term "Word8" as used herein may refer to a signed 8-bit integer in ITU-T G.191 basic operators. Similarly, the term "Word16" may refer to a signed 16-bit integer in ITU-T G.191 basic operators.

[0023] According to some other embodiments, a method in a decoder for reconstructing a target vector includes receiving an index including a plurality of segments, corresponding shift values, and a global offset value vector. The method includes performing segment-wise and coefficient-wise upshifting of vectors in the plurality of segments using column shift (e.g., "col_shift") values ​​to form an upshifted vector within each of the plurality of segments. The method includes performing an inverse discrete cosine transform (DCT) Type II transform of the upshifted segments. The method includes scaling down an output vector from the DCT Type II transform to the original unscaled FDCNG domain vector. The method includes adding the global offset value vector to the unscaled frequency domain-comfort noise generation (FD-CNG) domain vector to form the target vector.

[0024] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, illustrate several non-limiting embodiments of the inventive concepts. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram of a system diagram for FD-CNG-CQ with low ROM and low WMOPS, according to some embodiments. [Figure 2] FIG. 2 is a diagram of an example operating environment of an encoder, according to some embodiments. [Figure 3] FIG. 2 is a block diagram of an audio encoder according to some embodiments. [Figure 4] FIG. 2 is a block diagram of an audio decoder according to some embodiments. [Figure 5] FIG. 2 is a block diagram of a host, according to some embodiments. [Figure 6]FIG. 2 is a block diagram of an exemplary virtualization environment in which some components of an encoder or decoder, or an encoder or decoder, may be implemented, according to some embodiments. [Figure 7] FIG. 1 is a block diagram of a first stage FD-CNG VQ search according to some embodiments. [Figure 8] FIG. 1 is a block diagram of a system diagram of FD-CNG-CQ with low ROM and low WMOPS supporting shorter input target vectors, according to some embodiments. [Figure 9] FIG. 1 is a block diagram of a first stage FD-CNG VQ search that supports shorter input target vectors, according to some embodiments. [Figure 10] FIG. 1 illustrates the operation of an encoder that implements DCT basis vector extrapolation to expand short input target vectors, according to some embodiments. [Figure 11] FIG. 1 is a block diagram illustrating FD-CNG vector reconstruction for the first stage of MSVQ, according to some embodiments. [Figure 12] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 13] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 14] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 15] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 16] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 17] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 18] 10 is a flowchart illustrating the operation of an encoder, according to some embodiments. [Figure 19] 10 is a flowchart illustrating the operation of a decoder according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0026] Some of the embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Examples of embodiments of the inventive concepts are shown. The embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art. However, the inventive concepts may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concepts to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. It may be implicitly assumed that elements from one embodiment are present / used in another embodiment.

[0027] As previously indicated, current solutions for the first stage (and subsequent stages) of MS-VQ generally still require large storage space. For example, in the case of EVS for FD-CNG VQ, the first stage uses 128 levels (7 bits) with 24 coefficients each, resulting in (128 x 24) = 3072 words (16-word) of ROM storage. In the EVS FD-CNG VQ implementation, this corresponds to 6 kilobytes, with each byte being 8 bits. The entire EVS 37-bit (7 + 5 * 6) FD-CNG VQ (all stages) uses (128 + 5 x 64) * 24 = 10752 16-words, or 20 kilobytes. This is twice the ROM size of a 46-bit AMR-WB ISF-VQ.

[0028] In some embodiments, the systems described herein show how to reduce storage space and include the first stage of a multi-stage VQ to improve the table ROM and worst-case WMOPS characteristics of 3GPP EVS-26.445 FD-CNG VQ spectral envelope quantization.

[0029] For example, EVS FD-CNG VQ is used to quantize the spectral envelope used in a silence insertion descriptor (SID) frame to 37 bits. The decoded SID frame can then be used to generate a time-domain background signal.

[0030] However, the optimization of the first stage (i.e., stage #1) of MSVQ can also be applied to any audio codec vector parameters that use multi-stage vector quantization. For example, the spectral envelope quantized by MSVQ can be used for Spectral Noise Shaping (SNS) during active music segments, or to quantize and represent the spectral envelope for synthesizing active unvoiced speech segments.

[0031] An exemplary improved structured stage 1 search (i.e., first stage search) system 100 is shown in Figure 1. The structural aspects performed offline a priori and the search and structural aspects employed during N candidate vector determination are shown.

[0032] Referring to FIG. 1, an initial stage 1 VQ codebook 102 trained with LBG (or K-means) has optimal Voronoi regions and uses 128 levels (7 bits) with 24 coefficients each, as previously described, resulting in (128 x 24) = 3072 words (Word16) of ROM storage. The codebook 102 allows for a full search of eight candidates (i.e., candidate vectors indicated by and / or pointed to by a codebook index). According to various embodiments, the codebook 102 is DCT24 transformed into a DCT24 truncated (8, 10, 16, 18) table Word8 codebook 104 (see, e.g., codebook 104 with truncated segment 105 containing a vector of coefficients of length 8), where the number of segments Ns is four segments with a maximum number of 18 coefficients, allowing for a full search of up to eight candidates. As shown below in Table 1, the truncated segment 105 contains 16 vectors (see, for example, nSeg[0]==16), each of which has a length of 8 coefficients, for a total of 128 values.

[0033] Codebook 104 is created offline, resulting in 987 words (Word 16) of ROM storage, approximately 30% of the size of codebook 102. In some embodiments, codebook 104 has four segments (i.e., the number of segments Ns=4). Optimization of codebook 104 (e.g., as shown in block 108 in FIG. 1 ) is also performed offline to create the vectors in codebook 104. This may include performing an IDCT transform of the vectors in codebook 104 to optimize them, as shown by inverse DCT (IDCT) block 106 and optimization block 108. In particular, spectral distortion (SD) may be maintained in the creation of codebook 104. Optimization also includes creating a “nearest neighbor” index circular list 110 containing vectors that point to the next or previous index in an approximate mean squared error (MSE) neighbor order. In some embodiments, nearest neighbor index circular list 110 has 128 entries in ROM Word 8.

[0034] 1 are part of an FD-CNG VQ search, according to various embodiments, that feeds and / or provides the eight best candidates to a second stage (e.g., stage 2) of the multi-stage VQ, as indicated by block 122, where further processing of the multi-stage VQ is performed. Before describing these blocks, the parameters used shall first be defined in terms of parameter name, dimension, value(s), and brief definition, as shown below in Table 1. TIFF2026508288000002.tif255167TIFF2026508288000003.tif174170

[0035] One feature shared by various embodiments of the disclosed subject matter is the performance savings when comparing the original table ROM space of 3072 single-precision floats (32 bits per coefficient) resulting in a total ROM storage of 12288 bytes in the EVS floating-point specification with the first stage of the FD-CNG VQ, which uses 128 levels (7 bits) with 24 coefficients each. These performance savings are shown in Table 2. Note that the corresponding table ROM space in the EVS fixed-point specification is 3072 16-bit integers (Word16), resulting in a total of 6144 bytes of ROM storage. TIFF2026508288000004.tif255169TIFF2026508288000005.tif202170

[0036] In some embodiments, the first-stage codebook 104 is structured such that the codebook 104 contains segments (e.g., four) with different DCT truncation lengths (e.g., 8, 10, 16, 18). Each segment has a fixed common truncation length (e.g., number of remaining coefficients) for that set of vectors. For example, in the context of this disclosure, a length of 24 coefficients would indicate no truncation of the length of the vector(s), and a truncation length of 8 coefficients would indicate that the vector(s) have been truncated and / or reduced to a length of 8 coefficients. The resulting dimensionality reduction reduces both storage size and search complexity.

[0037] In this disclosure, a design with Ns=4 segments with different truncation lengths will be used, each with different frequency characteristics, including spanning from a low-frequency codebook to one or more intermediate-frequency codebook(s) to one or more high-frequency codebook(s).

[0038] In some embodiments, the truncation length is optimized offline. One exemplary simple design method for establishing the truncation length is simply to set the relative energy requirements as follows:

[0039] In the first step, we train an untruncated probabilistic base codebook with a vector length of N_FDCNG using k-means or LBG VQ training algorithm.

[0040] For each vector contained in the base codebook 102, an initial individual truncation length is set to the length at which a significant portion of the vector energy remains, eg, 95%.

[0041] In the second step, the initial vector length is increased individually so that N subsets with equal truncation lengths can be created, each of which can then be designated as a segment with equal truncation length, and all subsets have at least 95% remaining energy.

[0042] In some embodiments, additional criteria may be added, such that the reconstruction point of each vector in the set of truncated codebook vectors should not move into the index Voronoi region of another vector in the original k-means / LBG trained base codebook (CB). Also, in further embodiments, the allowed movement away from the original reconstruction point may be limited to not move too far away from the Voronoi region setup of the original base codebook.

[0043] Before describing the first stage FD-CNG VQ search and reconstruction, first an overview of the operating environment, encoder, decoder, host, and virtualization environment will be described herein.

[0044] FIG. 2 illustrates a block diagram of an exemplary operating environment 200 in which various embodiments of the present disclosure may be implemented. Referring to FIG. 2 , in the exemplary operating environment 200, an encoder 202 receives data, such as an audio file, to be encoded from an entity, such as a host 206, over a network 204 and / or from storage 208. The encoder 202, in various embodiments, is a parametric stereo encoder. In some embodiments, the host 206 may communicate directly with the encoder 202. In some embodiments, the encoder 202 may encode the audio file as described herein and either store the encoded audio file in storage 208 or transmit the encoded audio file via a network 210 to a decoder 212. The decoder 212, in various embodiments, is a parametric stereo decoder. The decoder 212 decodes the audio file and transmits the decoded audio file to an audio player 214 for playback. The audio player 214 may be or be included within a user device, terminal, mobile phone, or the like. In other embodiments, the host 206 may transmit the encoded audio file to the decoder 212 over the network 210 .

[0045] 3 illustrates an audio encoder 202, implemented as a standalone device, according to some embodiments. As used herein, an audio encoder refers to a device capable of, configured to, and / or operable to encode an object and to communicate with a network node, encoder, and / or decoder. Examples of audio encoders include, but are not limited to, smartphones, mobile phones, cell phones, Voice over IP (VoIP) phones, wireless local loop telephones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, storage devices, playback appliances, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptop computers, laptop embedded appliances (LEEs), laptop mounted appliances (LMEs), smart devices, wireless customer premises equipment (CPEs), vehicle-mounted or vehicle-embedded / integrated wireless devices, etc.

[0046] An audio encoder may support device-to-device (D2D) communications, for example, by implementing 3GPP standards for sidelink communications, dedicated short-range communications (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, an encoder does not necessarily have a user in the sense of a human user who owns and / or operates an associated device.

[0047] The audio encoder 202 includes a processing circuit 302 operably coupled to an input / output interface 306, a power supply 308, a memory 310, a communication interface 312, and / or any other components, or any combination thereof, via a bus 304. Some decoders may utilize all or a subset of the components shown in FIG. 3. The level of integration between components may vary from decoder to decoder. Additionally, some decoders may include multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0048] The processing circuitry 302 is configured to process instructions and data and may be configured to implement any sequential state machine operable to execute instructions stored in memory 310 as a machine-readable computer program. The processing circuitry 302 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), programmable logic together with appropriate firmware, one or more stored computer programs such as a microprocessor or digital signal processor (DSP) together with appropriate software, a general-purpose processor, or any combination of the above. For example, the processing circuitry 302 may include multiple central processing units (CPUs).

[0049] In this example, the input / output interface 306 may be configured to provide one or more interfaces to an input device, an output device, or one or more input and / or output devices. Examples of output devices include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smart card, another output device, or any combination thereof. An input device may allow a user to capture information into the audio encoder 202. Examples of input devices include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a webcam, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smart card, etc. A presence-sensitive display may include a capacitive or resistive touch sensor for detecting input from a user. The sensor may be, for example, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, a light sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as the input device. For example, a universal serial bus (USB) port may be used to accommodate input and output devices.

[0050] In some embodiments, the power supply 308 is structured as a battery or battery pack. Other types of power sources may be used, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a battery. The power supply 308 may further include power circuitry for delivering power to various portions of the audio encoder 202 from the power supply 308 itself and / or from an external power source via an interface such as an input circuit or a power cable. Delivering power may be for charging the power supply 308, for example. The power circuitry may perform any formatting, converting, or other modification on the power from the power supply 308 to make it suitable for the respective components of the audio encoder 202 being powered.

[0051] The memory 310 may be or be configured to include memory, such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk, optical disk, hard disk, removable cartridge, flash drive, etc. In one example, the memory 310 includes one or more application programs 314, such as an operating system, a web browser application, a widget, a gadget engine, or other applications, and corresponding data 316. The memory 310 may store any of a variety of different operating systems or combinations of operating systems for use by the audio encoder 202.

[0052] The memory 310 may be configured to include several physical drive units, such as a redundant array of independent disks (RAID), flash memory, a USB flash drive, an external hard disk drive, a thumb drive, a pen drive, a key drive, a high-density digital versatile disc (HD-DVD) optical disc drive, an internal hard disk drive, a Blu-ray optical disc drive, a holographic digital data storage (HDDS) optical disc drive, an external mini dual in-line memory module (DIMM), a synchronous dynamic random access memory (SDRAM), an external micro-DIMM SDRAM, a smart card memory, such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) containing one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC, commonly known as a "SIM card." The memory 310 may enable the audio encoder 202 to access, offload, or upload data, instructions, application programs, etc. stored on a temporary or non-transitory memory medium. An article of manufacture, such as an article of manufacture utilizing a communication system, may be tangibly embodied as or in the memory 310, which may be or comprise a device-readable storage medium.

[0053] The processing circuit 302 may be configured to communicate with an access network or other networks using a communication interface 312. The communication interface 312 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 322. The communication interface 312 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., a UE or a network node in an access network). Each transceiver may include a transmitter 318 and / or a receiver 320 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Moreover, the transmitter 318 and receiver 320 may be coupled to one or more antennas (e.g., antenna 322) and may share circuit components, software, or firmware, or may alternatively be implemented separately.

[0054] In the illustrated embodiment, the communication capabilities of communication interface 312 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth, near-field communication, location-based communication such as using a Global Positioning System (GPS) to determine location, another similar communication capability, or any combination thereof. Communication may be implemented according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0055] Regardless of the type of sensor, the audio encoder may provide an output of encoded data to a network node via a wireless connection through the audio encoder's communication interface 312.

[0056] The audio encoder, when in the form of an Internet of Things (IoT) device, can be a device for use in one or more application domains, including, but not limited to, urban wearable technology, augmented industrial applications, and healthcare. Non-limiting examples of such IoT devices are or are incorporated into a connected refrigerator or freezer, a TV, a connected lighting device, an energy meter, a robotic vacuum cleaner, a voice-controlled smart speaker, a home security camera, a thermostat, an electric door lock, a connected doorbell, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smart watch, a fitness tracker, a head-mounted display for augmented reality (AR) or virtual reality (VR), or a wearable for haptic augmentation or sensory augmentation. A decoder in the form of an IoT device comprises circuitry and / or software depending on the intended application of the IoT device, in addition to the other components described with respect to the audio encoder 202 shown in FIG. 3 .

[0057] 4 illustrates an audio decoder 212 (e.g., a parametric stereo encoder) according to some embodiments, where the audio decoder 212 is implemented as a standalone device. As used herein, an audio decoder refers to a device capable of, configured to, and / or operable to decode an object and to communicate with a network node, an encoder, and / or a decoder. Examples of audio decoders include, but are not limited to, smartphones, mobile phones, cell phones, voice-over-IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, gaming consoles or devices, storage devices, playback appliances, wearable terminal devices, wireless endpoints, mobile stations, tablets, laptop computers, laptop embedded appliances (LEEs), laptop mounted appliances (LMEs), smart devices, wireless customer premises equipment (CPEs), vehicle-mounted or vehicle-embedded / integrated wireless devices, etc.

[0058] The audio decoder may support device-to-device (D2D) communications, for example, by implementing 3GPP standards for sidelink communications, dedicated short-range communications (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, the decoder does not necessarily have a user in the sense of a human user who owns and / or operates the associated device.

[0059] The audio decoder 212 includes a processing circuit 402 operably coupled via a bus 404 to an input / output interface 406, a power supply 408, a memory 410, a communication interface 412, and / or any other components, or any combination thereof. Some decoders may utilize all or a subset of the components shown in FIG. 4. The level of integration between components may vary from decoder to decoder. Additionally, some decoders may include multiple instances of a component, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0060] The processing circuit 402 is configured to process instructions and data and may be configured to implement any sequential state machine operable to execute instructions stored in the memory 410 as a machine-readable computer program. The processing circuit 302 may be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), etc.), programmable logic with appropriate firmware, one or more stored computer programs such as a microprocessor or digital signal processor (DSP) with appropriate software, a general-purpose processor, or any combination of the above. For example, the processing circuit 402 may include multiple central processing units (CPUs).

[0061] In this example, the input / output interface 406 may be configured to provide one or more interfaces to an input device, an output device, or one or more input and / or output devices. Examples of output devices include a speaker, a sound card, a video card, a display, a monitor, a printer, an actuator, an emitter, a smart card, another output device, or any combination thereof. An input device may allow a user to capture information into the audio decoder 212. Examples of input devices include a touch-sensitive or presence-sensitive display, a camera (e.g., a digital camera, a digital video camera, a webcam, etc.), a microphone, a sensor, a mouse, a trackball, a directional pad, a trackpad, a scroll wheel, a smart card, etc. A presence-sensitive display may include a capacitive or resistive touch sensor for detecting input from a user. The sensor may be, for example, an accelerometer, a gyroscope, a tilt sensor, a force sensor, a magnetometer, a light sensor, a proximity sensor, a biometric sensor, etc., or any combination thereof. An output device may use the same type of interface port as the input device. For example, a universal serial bus (USB) port may be used to accommodate input and output devices.

[0062] In some embodiments, the power source 408 is structured as a battery or battery pack. Other types of power sources may be used, such as an external power source (e.g., an electrical outlet), a photovoltaic device, or a battery. The power source 408 may further include power circuitry for delivering power to various portions of the audio decoder 212 from the power source 408 itself and / or from an external power source via an interface such as an input circuit or a power cable. Delivering power may be for charging the power source 408, for example. The power circuitry may perform any formatting, converting, or other modification on the power from the power source 408 to make it suitable for the respective components of the audio decoder 212 being powered.

[0063] The memory 410 may be or be configured to include memory, such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic disk, optical disk, hard disk, removable cartridge, flash drive, etc. In one example, the memory 410 includes one or more application programs 414, such as an operating system, a web browser application, a widget, a gadget engine, or other applications, and corresponding data 416. The memory 410 may store any of a variety of different operating systems or combinations of operating systems for use by the audio decoder 212.

[0064] The memory 410 may be configured to include several physical drive units, such as a redundant array of independent disks (RAID), flash memory, a USB flash drive, an external hard disk drive, a thumb drive, a pen drive, a key drive, a high-density digital versatile disc (HD-DVD) optical disc drive, an internal hard disk drive, a Blu-ray optical disc drive, a holographic digital data storage (HDDS) optical disc drive, an external mini dual in-line memory module (DIMM), a synchronous dynamic random access memory (SDRAM), an external micro-DIMM SDRAM, a smart card memory, such as a tamper-resistant module in the form of a universal integrated circuit card (UICC) containing one or more subscriber identity modules (SIMs), such as a USIM and / or ISIM, other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC, commonly known as a "SIM card." The memory 310 may enable the audio decoder 212 to access, offload, or upload data, instructions, application programs, etc. stored on a temporary or non-transitory memory medium. An article of manufacture, such as an article of manufacture utilizing a communication system, may be tangibly embodied as or in the memory 410, which may be or comprise a device-readable storage medium.

[0065] The processing circuit 402 may be configured to communicate with an access network or other networks using a communication interface 412. The communication interface 412 may comprise one or more communication subsystems and may include or be communicatively coupled to an antenna 422. The communication interface 412 may include one or more transceivers used to communicate, such as by communicating with one or more remote transceivers of another device capable of wireless communication (e.g., a UE or a network node in an access network). Each transceiver may include a transmitter 318 and / or a receiver 320 suitable for providing network communication (e.g., optical, electrical, frequency allocation, etc.). Moreover, the transmitter 418 and receiver 420 may be coupled to one or more antennas (e.g., antenna 422) and may share circuit components, software, or firmware, or may alternatively be implemented separately.

[0066] In the illustrated embodiment, the communication capabilities of communication interface 412 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth, near-field communication, location-based communication such as using a Global Positioning System (GPS) to determine location, another similar communication capability, or any combination thereof. Communication may be implemented according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0067] Regardless of the type of sensor, the audio decoder may provide an output of the decoded data to a network node via a wireless connection through the audio decoder's communication interface 412.

[0068] The audio decoder, when in the form of an Internet of Things (IoT) device, can be a device for use in one or more application domains, including, but not limited to, urban wearable technology, augmented industrial applications, and healthcare. Non-limiting examples of such IoT devices are or are incorporated into a connected refrigerator or freezer, a TV, a connected lighting device, an energy meter, a robotic vacuum cleaner, a voice-controlled smart speaker, a home security camera, a thermostat, an electric door lock, a connected doorbell, an autonomous vehicle, a surveillance system, a weather monitoring device, a vehicle parking monitoring device, an electric vehicle charging station, a smartwatch, a fitness tracker, a head-mounted display for augmented reality (AR) or virtual reality (VR), or a wearable for haptic augmentation or sensory augmentation. A decoder in the form of an IoT device comprises circuitry and / or software depending on the intended application of the IoT device, in addition to the other components described with respect to the audio decoder 212 shown in FIG. 4 .

[0069] 5 is a block diagram of a host 206 in accordance with various aspects described herein. As used herein, a host 206 may be or comprise various combinations of hardware and / or software, including a standalone server, a blade server, a cloud-implemented server, a distributed server, a virtual machine, a container, or processing resources in a server farm. The host 206 may provide one or more services to one or more UEs.

[0070] Host 206 includes a processing circuit 502 operably coupled to an input / output interface 506, a network interface 508, a power supply 510, and a memory 512 via a bus 504. In other embodiments, other components may be included. Features of these components may be substantially similar to those described with respect to the devices of previous figures, such as FIGS. 3 and 4, and therefore, those descriptions are generally applicable to the corresponding components of host 206.

[0071] The memory 512 may include one or more computer programs, including one or more host application programs 514 and data 516, which may include user data, e.g., data generated by an encoder or decoder for the host 206 or data generated by the host 206 for the UE. An embodiment of the host 206 may utilize only a subset or all of the shown components. The host application programs 514 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), Moving Picture Experts Group (MPEG), VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711, EVS, Immersive Voice and Audio Services (IVAS)), including transcoding for multiple different classes, types, or implementations of UE (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application program 514 may also provide user authentication and license checks, and may periodically report health, route, and content availability to a central node, such as a device in the core network or a device on the edge of the core network. Thus, the host 206 may select and / or indicate a different host for over-the-top services for the UE. The host application program 514 may support various protocols, such as HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0072] FIG. 6 is a block diagram illustrating a virtualization environment 600 in which functions implemented by some embodiments of the audio encoder 202 or components thereof, or by some embodiments of the audio decoder 212 or components thereof, may be virtualized. In this context, virtualizing means creating a virtual version of an apparatus or device, which may include virtualizing a hardware platform, storage devices, and networking resources. As used herein, virtualization may apply to any device described herein, or components thereof, and relates to implementations in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments 500 hosted by one or more hardware nodes, such as a decoder, encoder, network node, UE, core network node, or hardware computing device acting as a host. Furthermore, in embodiments in which the virtual node does not require wireless connectivity (e.g., to a core network node or host), the node may be fully virtualized.

[0073] An application 602 (which may alternatively be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) is run in the virtualized environment 600 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0074] The hardware 604 includes processing circuitry, memory that stores software and / or instructions executable by the hardware processing circuitry, and / or other hardware devices described herein, such as network interfaces, input / output interfaces, etc. Software is executed by the processing circuitry to instantiate one or more virtualization layers 600 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 608A and 608B (one or more of which may be referred to generically as VMs 608), and / or implement any of the functions, features, and / or benefits described with respect to some embodiments described herein. The virtualization layer 600 may present to the VMs 608 a virtual operating platform that appears to be networking hardware.

[0075] The VMs 608 may comprise virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be run by a corresponding virtualization layer 600. Different embodiments of the virtual appliance 602 instance may be implemented on one or more of the VMs 608, and the implementation may be done in different ways. Hardware virtualization is referred to in some contexts as network functions virtualization (NFV). NFV may be used to consolidate many network equipment types onto industry-standard high-volume server hardware, physical switches, and physical storage that may be located in data centers and customer premises equipment.

[0076] In the context of NFV, a VM 608 may be a software implementation of a physical machine that runs programs as if those programs were running on a physical, non-virtualized machine. Each VM 608 and the portion of the hardware 604 on which it runs, whether hardware dedicated to that VM and / or hardware shared by that VM with other VMs, form a separate virtual network element. Further, in the context of NFV, a virtual network function is responsible for handling a particular network function running in one or more VMs 608 on the hardware 604 and corresponds to the application 602.

[0077] The hardware 604 may be implemented in a standalone network node with general or specific components. The hardware 604 may implement some functions via virtualization. Alternatively, the hardware 604 may be part of a larger cluster of hardware (e.g., as in a data center or CPE) where many hardware nodes cooperate and are managed via a management and orchestration 610 that, among other things, oversees the lifecycle management of the application 602. In some embodiments, the hardware 604 is coupled to one or more radio units, each including one or more transmitters and one or more receivers, which may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with virtual components to provide a virtual node with wireless capabilities, such as a radio access node or base station. In some embodiments, some signaling may be provided using a control system 612, which may alternatively be used for communication between the hardware nodes and the radio units.

[0078] Referring now to FIG. 7, the first stage (e.g., stage #1) FD-CNG VQ search will now be described using a codebook 104 created offline (e.g., as shown above and in FIG. 1).

[0079] In some embodiments, the encoder 202 (i.e., the encoder 202 shown in FIG. 2) obtains a target vector in block 700. For example, an input target vector of length N_target may be obtained using the 3GPP EVS codec analysis algorithm, as outlined in the encoder of reference 3GPP TS26.445.

[0080] In some cases, the FD-CNG noise estimator relies on a hybrid spectral analysis approach: the low frequencies corresponding to the core bandwidth are covered by a high-resolution Fast Fourier Transform (FFT) analysis, while the remaining higher frequencies are captured by a CLDFB, which exhibits a significantly lower spectral resolution of 400 Hz.

[0081] The input signal to the EVS audio / speech encoder may be configured to quantize the spectrum in the FD_CNG domain so that the EVS algorithm can quantize FDCNG target vectors of dimensions 17, 20, 21, and 24. This disclosure describes the case where N_target is 24, but the first-stage MSVQ method may be applied to other dimensions. In alternative embodiments, N_target may be 21 (e.g., 21(N_WB)) without departing from the scope of the disclosed subject matter. However, when using the same stored codebook (e.g., a set of DCT-II trained codebooks trained for dimension 24) for other smaller dimensions (e.g., 17, 20, 21), special care should be taken to generate nearly distortion-free target vectors of dimension 24 (e.g., the same DCT dimension used in codebook training).

[0082] An example operation is when the unnormalized target signal in the EVS description is N FD-CNG (i) (where "N" represents noise). Furthermore, the EVS specification states in "5.6.3.5 Encoding SID frames in FD-CNG" that N FD-CNG (i) is provided. In particular, the length L SID corresponds to the variable N_target disclosed in this specification. FD-CNG The final determination of the variables for the first SID frame after a speech frame can be found in section "5.6.3.3 Adjusting the first SID frame in FD-CNG" (Equation 1395).

[0083] The second operation involves both log-domain conversion and spectral envelope normalization. FD-CNG (i) The signal is converted to the log10 dB domain in Section 5.6.3.5, equation (1396). Then, in equation (1397), the normalized signal TIFF2026508288000006.tif7170 is acquired. In this specification, the target signal target[i] of length N_target is acquired by SID Signal Supports TIFF2026508288000007.tif7170.

[0084] In some embodiments, the encoder 202 removes a global offset value vector, denoted herein as a midvalue vector, in operation 702. In other embodiments, the offset value vector may be a mean value vector, a median value vector, etc. This is done to reduce signal dynamics both during quantization and storage. The obtained target signal is subtracted by the offline analyzed global offset value vector in the FD-CNG domain. An example of such an operation may be mathematically described as follows: target_mr[i]=target[i]-midQ[i], where i∈0...(N_target-1)

[0085] In some embodiments, the encoder may add a global scale factor using DCT and IDCT-related operations to scale up the target vector to a search domain with maximized dynamics and / or range in operation 704. More specifically, the global scale factor is applied to the median-cleaned target vector to maximize storage and search accuracy within the Word8 mantissa. An example of such an operation may be mathematically described as follows: target_mr_scaled[i]=target_mr[i]*dct_invScaleF[1], where i∈0...(N_target-1) Here, dct_invScaleF[1] maximizes storage dynamics and also scales up the target vector to a search domain containing four binomials or bits (i.e., four "fractions" in the binary domain). The purpose of this DCT implementation-related operation is to scale up the median-cleared signal to the target level before applying the DCT to improve search precision (e.g., granularity). Without this upscaling, the dct_target vector (after DCT processing) would exhibit a low dynamic range. In floating-point implementations, the lack of upscaling is largely inconsequential, but in fixed-point implementations (i.e., implementations with limited precision), it is important to maximize the DCT input signal dynamic range and the DCT output signal range. Otherwise, the DCT transformation would likely add analytical noise without upscaling.

[0086] In some embodiments, the scaled target vector signal is transformed into a DCT24 search domain by the encoder 202 in operation 706. The transformation results in a DCT target vector, denoted herein as "dct_target." A DCT Type II transformation (e.g., operation 706 in FIG. 7) may be applied as follows: dct_target=dct(target_mr_scaled) Here, target_mr_scaled always has dimension 24 (NMAX_FDCNG). Even if N_target is lower than NMAX_FDCNG, the analysis DCT is performed using dimension NMAX_FDCNG. This avoids implementing several different DCTs (and IDCTs), and the mantissa and scale factors can be optimized for one DCT length.

[0087] In some embodiments, the encoder 202 calculates a common MSE contribution for globally truncated DCT coefficients. Potential common truncation in the DCT search domain for the DCT truncation length in all segments in the segmented first-stage codebook results in a common high coefficient error contribution. This error needs to be calculated if one additional first-stage segment does not employ a transformation to provide a fair comparison of the first-stage mean squared error. One example of such an operation may include optionally synthesizing the truncated targets and calculating the common error, i.e., setting and / or truncating the DCT target (e.g., dct_target) vector components to zero from max(Nseg)==18 to N_target-1==23.

[0088] In operation 112, the encoder 202 performs a pairwise inner loop search to establish eight (i.e., 4×2) “winners” from the four pairwise searches. The pairwise search is a globally suboptimal pairwise search of four different codebooks in the codebook 104, followed by a post-optimization in operation 116.

[0089] To perform the search, the encoder 202 may establish a segment loop setup. The operations and / or steps described below are performed in the specific order from segm=0 to segm=Ns-1, inclusive. For example, in the first exemplary step, tLen is set to truncLen[segm] for the duration of segment segm. Furthermore, st1_mse_pair[segm][0] is set to a very large value (MAX_FLOAT), and st1_mse_pair[segm][1] is set to a very large value (MAX_FLOAT). This initialization ensures that both values ​​will be updated. Furthermore, p_max, which points to the worst vector in the segment pair, is initialized to 0.

[0090] As part of the disclosed operations / steps, the encoder may calculate the common MSE contribution for the truncated coefficients of a segment. A truncation error of up to truncLen[Ns-1] per segment is required to be able to ultimately compare the MSE error between different segments. For example, a first operation may include initializing an MSE variable by setting mse_trunc_segm[segm] = mse_trunc_all_segms. Furthermore, the error energies of the truncated target coefficients of this segment may be summed up to a global maximum truncation length truncLen[Ns-1] >= tLen. Furthermore, mse_trunc_segm[segm] + = (dct_target[tLen+i]) 2 , where i∈0...(truncLen[Ns-1]-tLen-1).

[0091] As part of the disclosed operations / steps, the encoder 202 points to the codebook and scale factors for the current segment. An exemplary first operation and / or step includes pointing the current segment, cb, to a common codebook for this segment, where all vectors have a truncation length, truncLen[segm], where cb = cb_segmW8[segm]. In a subsequent step / operation, the current segment, col_shift vector, is pointed to, to the coefficient column scaling codebook for this segment, segm, where all col_shift vectors have a length, tLen, which may be mathematically expressed as dct_col_shift_tab = col_shift[segm]. As used herein, the term "column shift" or "col_shift" refers to a scaling factor, e.g., 2 [col_shift] may represent an integer indicating the exponent value for a scaling factor equal to

[0092] As part of the disclosed operations / steps, the encoder 202 performs a setup for each vector within each segment segm. The steps in summing the MSE for untruncate coefficients are completed in that particular order from idx=0 to idx=nSeg[segm-1], inclusive. For example, a first exemplary operation may include calculating idx_full for each of the Nc candidate post-optimization steps, so as to be able to store the MSE of this CB vector in a structured manner. In particular, idx_full=idx+nSegCum[segm]. The local mse is then initialized with the common contribution for this segment, i.e., mse idx = mse_trunc_segm[segm]. Then, for the current codebook vector idx, the MSE is tmp[c] = dct_target[c] - cb[idx][c] * 2 for c∈0...(tLen-1). col_shift_tab[c] Using mse for c∈0...(tLen-1)idx +=tmp[c] 2 In particular, this is the inner MSE calculation loop, where the DCT truncation is beneficial in terms of reducing WMOPS complexity, since now only coefficients ranging from 0 to tLen-1 affect the MSE summation.

[0093] In some embodiments, the encoder 202 then saves the MSE value for the current vector index. This step is optional in some embodiments and serves as an "extended candidate" analysis or post-analysis step to select the final candidate vector from Stage 1. For example, the MSE value can be calculated as follows: st1_mses[idx_full]=mse idx As previously shown, these stored MSE values ​​can be used in subsequent low-complexity post-analysis steps.

[0094] In some embodiments, the encoder 202 then conditionally updates the best value pair for this segment. For example, a first exemplary operation and / or step includes first evaluating whether the current vector is better than the worst one in the segment pair (i.e., whether the current vector has a lower MSE than the best MSE among the segment vector pairs evaluated so far). As used herein, a vector is "better" if it has a lower MSE, but a vector is "worst" if it has the highest MSE. In particular, we can start with a conditional update of the worst index pointed to by p_max: TIFF2026508288000008.tif24170

[0095] In a second operation, the best MSE may be conditionally updated via: TIFF2026508288000009.tif31170

[0096] In the third operation, p_max is updated after each new candidate, always re-evaluating whether a new worst candidate has been stored. TIFF2026508288000010.tif37170

[0097] Here, the legacy MSVQ Phase #1 solution maintains a complex bookkeeping record of, for example, a list of 8 (or even 24) values, which potentially creates a large worst-case (WC) complexity problem.

[0098] At this stage, all Ns segments have been processed by the encoder 202.

[0099] In particular, there are Ns*2 stored preliminary candidates for the variables, i.e., st1_mse_pair[Ns][2] (best mse value from per-segment search) st1_idx_pair[Ns][2] (best idx_full value from per-segment search) Nv (128) stored MSEs in st1_mse are created.

[0100] In this example with Ns=4, the required Nc=8 stage #2 candidate values ​​have already been determined. Among these Nc values, it is known from the serial pairwise search that at least two of them correspond to the global minimum among all eight values. However, Nc-2 (i.e., six in this case) values ​​may represent suboptimal candidates for stage #2 of the multi-stage VQ.

[0101] Since the truncated segments of the DCT-Type II have different truncation lengths, these segments will have different high-frequency content. In other words, different codebook segments can be considered as having different low-pass filters applied to them.

[0102] In general, these Ns*2=Nc candidates may be acceptable for the second stage search. However, for input target signals with more than a pair of good vectors in a segment, it is beneficial to check whether some of the candidates (e.g., the final pairwise search candidate vectors) can be replaced by better candidates.

[0103] In preparation for the second stage, the st1_mse_pair matrix is ​​serialized into an MSE vector dist of length Ns*2, and the candidate index vector st1_idx_pair matrix is ​​serialized in the same way into a vector indices of length Ns*2.

[0104] (Note: this serialization in place is a free operation in C.)

[0105] In some embodiments, the pointer p_max is updated to point to the worst candidate in dist and indices by searching for the maximum MSE value in dist.

[0106] For example, a first operation may include updating a pointer p_max to point to the worst candidate in vectors dist and indices by searching for the maximum value in dist, p_max=maximum(dist,Nc), where max_index=maximum(vec,len) is a function that selects the maximum index for values ​​in vector vec of length len and returns the index of the maximum value in vec as max_index.

[0107] Furthermore, the two best candidates (ie, lowest MSE) are arranged as follows: TIFF2026508288000011.tif38170Here, min_index=maximum(vec,len) is a function that selects the minimum index for the values ​​in a vector vec of length len and returns the index of the minimum value in vec as min_index.

[0108] In operation 116, the encoder 202 selects a final set of Nc candidates using the circular MSE neighboring index list.

[0109] In some embodiments, the circular MSE neighboring index list is implemented by the encoder 202 in operation 708. Offline, the vectors in the global temporary codebook (e.g., in this case the serial concatenation of the vectors in the Ns=4 segment codebook cb_temporary_full=[cb_segmW8[0],cb_segmW8[1],cb_segmW8[2],cb_segmW8[Ns-1]]) are analyzed (e.g., analyzed offline) into self-MSE ordered vectors of length Nv indices.

[0110] In some embodiments, a version of this circularly ordered vector can be obtained by analyzing the MSE between codebook vectors and applying an approximate Traveling Sales Person (TSP-closed) problem-solving solution to the vectors in cb_temporary_full. The distance measure between two cities corresponds to the MSE between two distinct vectors in the codebook. Essentially, each vector index in the codebook indicates a city in the Traveling Sales Person problem statement, and the MSE between the vectors corresponds to the distance between the two cities.

[0111] In this example, the convex hull method for the closed TSP problem is used to obtain an ordering vector mse_order_circ of length Nv over all segments in the full concatenated codebook cb_temporary_full. To save cycle complexity at the expense of limited ROM cost, the circularly ordered index vector mse_order_circ can be used to create two auxiliary vectors, neighb_mse_fwd and neighb_mse_rev, as shown below in Table 3. TIFF2026508288000012.tif84170

[0112] In particular, neighb_mse_fwd and neighb_mse_rev can also be created during runtime using the mse_order_circ vector of indices.

[0113] As part of the optimization in operation 116 (e.g., in FIG. 1 or FIG. 7), the encoder 202 may perform a first operation and / or step that includes setting up a vector check_ind of a total of Ncheck=8 likely intersecting segment candidate indices (total indices). / * Use cyclic MSE neighbors for the two best candidates so far * / TIFF2026508288000013.tif31170

[0114] Then, the second operation involves following the circular list one additional step in each direction by: TIFF2026508288000014.tif31170

[0115] In a third operation, the saved global MSE values ​​are used to check whether they are better candidates than the initial N candidates from the fast pairwise search. In some cases, only the top N-2 candidates are replaced (e.g., in steps described below).

[0116] For example, the two best candidates are initially excluded (note: global exclusion to never reselect / use the two (best) MSE values ​​so far). TIFF2026508288000015.tif121170

[0117] In the fourth operation, there may be a search portion of the first stage for second stage candidate indices, where Nc candidate indices are available in indices[] and their MSEs are available in dist of length Nc.

[0118] 7, the results of operation 116 are provided by encoder 202 to operation 118. The final candidates are converted by encoder 202 back to the FD-CNG domain.

[0119] In some embodiments, the second stage needs to have the correct error signal in that particular domain since the upper stage(s) were scaled and prepared in that domain (e.g., each of the Nc vectors may have a corresponding error vector signal res[cand], also known as the residual signal, as input to stage 2).

[0120] For example, the upper FD-CNG stage can be found in the EVS 3GPP specification TS26.445.

[0121] In particular, for that candidate, the best Nc MSE values ​​do not need to be recalculated because the DCT transform / rotation does not change the MSE distance to the target signal. However, if the input signal length is shorter than the search (and stored codebook) DCT length, the Nc best MSEs may be updated to the correct shorter MSE domain by excluding the error from the upper zeroed or upper extended portion. In operation 710, the following steps are performed by the encoder 202. In particular, the first operation and / or step includes obtaining a residual vector signal res[cand][] for each of the selected Nc candidates in indices. Obtained by decoding TIFF2026508288000016.tif45170ll,

[0122] In a second operation, the search for Stage #2 (i.e., the second stage) can begin, with Nc candidate indices available in indices[], their MSEs available in dist[], and the residual signal in the FDCNG domain available as res[Nc][N_target]. Furthermore, the worst candidate is denoted by p_max.

[0123] The encoder 202 (or decoder 212) receives an index idx_full in the range [0...(Nv-1)] and calculates the transmitted FDNCG vector fdcng_final of length N_target as follows: idx_full Reconstruct.

[0124] In the first operation, idx_full is decomposed into segment segm and local idx, i.e., first identify segment segm and then identify local index idx. TIFF2026508288000017.tif53170idx is here a local index into the segment-specific codebook cb_segmW8[segm].

[0125] In the second operation, the algorithm points to the correct exponent table for segment segm by: TIFF2026508288000018.tif8170 Note: The expTable vector has length tLen = truncLen[segm].

[0126] In a third operation, the process includes retrieving and scaling the FDCNG vector with the DCT domain coefficients corresponding to idx from the table ROM memory. TIFF2026508288000019.tif30170

[0127] In particular, the mantissa value cb_segmW8[segm][idx][c] stored as a byte (Word8) can be extracted and upscaled using the C binary left shift operator "<<". For all float arithmetic, the dct_vec vector is obtained equivalently as follows: dct_vec[c]=(cb_segmW8[segm][idx][c])*2 expTable[c] , where c∈0...(tLen-1)

[0128] Optionally, in an optional DSP-optimized fixed-point implementation, the DSP instruction set may not be able to efficiently fetch Word 8 from ROM or RAM. To manage that case, it may be preferable to fetch the coefficients in consecutive byte pairs (Word 16), as follows:

[0129] Fetch the FDCNG vector with the DCT domain coefficients corresponding to idx from the table ROM memory and scale it. TIFF2026508288000020.tif53170 where AND(a,b) is a bitwise "and" function operating on 16-bit (Word16) variables. For example, the mantissa value cb_segmW8[segm][idx] is extracted as a 2-byte chunk (Word16) in this DSP-optimized version. In some embodiments, each word is masked with a bitwise AND and appropriately scaled using the C binary left-shift operator "<<". This DSP optimization can be applied to both the first-stage search in the encoder and the first-stage reconstruction in the decoder.

[0130] In a fourth operation, the DCT domain vector is transformed to the FDCNG domain with the middle removed by setting the dct_vec vector components above tLen to 0, thus dct_vec[i]=0.0, where i∈tLen...(NMAX_FDCNG-1) is.

[0131] Furthermore, IDCT, inverse DCT type II transform is applied as follows. idct_vec1=idct(dct_vec), where dct_vec has dimension 24 (NMAX_FDCNG).

[0132] In particular, if N_target is lower than NMAX_FDCNG, the composite IDCT is still run using the dimensions NMAX_FDCNG.

[0133] In the fifth operation, the vector is scaled down to the original unscaled FDCNG domain (e.g., the Word8 mantissa representation was upscaled using the global value) by: idct_vec_global_scaled[i]=idct_vec1[i]*dct_scaleF[1], where i∈0...(N_target-1)

[0134] In the sixth operation, the intermediate stage #1 CB vector is added via: fdcng_final idx_full [i]=idct_vec_global_scaled[i]+midQ[i], where i∈0...(N_target-1)

[0135] In some embodiments, this reconstruction principle allows for fast searches in the residual (offset removed) truncated DCT-II domain and further allows for storage of CB code vectors with mantissas of size Word8.

[0136] In particular, a codebook 104 created for dimension 24 (NMAX_FDCNG) can be used for vectors of a different dimension. This is shown in FIGS. 8 and 9, where codebook 104 is used to quantize a vector of dimension 21 (N_WB). FIG. 8 is a block diagram of a system diagram of FD-CNG-VQ with low ROM and low WMOPS, supporting shorter input target vectors. FIG. 8 is similar to FIG. 1, but has two additional blocks: blocks 800 and 802. Unless otherwise described, the encoder 202 performs the same operations in blocks 102-122 of FIG. 8 as the corresponding blocks in FIG. 1 described above. In block 801, FDCNG21 domain values ​​for the target vector are extrapolated to FDCNG24 domain values. As previously shown, the output of block 120 has values ​​corresponding to the dimension DCT24 domain. In block 802, the DCT24 domain MSE values ​​are updated to the FDCNG21 domain.

[0137] Referring now to Figure 9, the first stage (i.e., stage #1) of the FD-CNG VQ search will now be described, using a codebook 104 created offline for dimension 24 (NMAX_FDCNG) but used to quantize a vector of dimension 21 (N_WB). Figure 9 is similar to Figure 7, but has two additional blocks: blocks 900 and 902. Unless otherwise described, the encoder 202 performs the same operations in blocks 104, 110-120, 122, and 700-704 shown in Figure 9 as the corresponding blocks in Figure 7 described above.

[0138] In block 700, the encoder 202 obtains a target vector of dimension 21 (e.g., a dimension shorter than the dimension used for codebook training). An input target vector of length N_WB may be obtained using the 3GPP EVS codec analysis algorithm, as outlined in the encoder of reference 3GPP TS26.445.

[0139] In some embodiments, an input target_wb signal of dimension N_WB is converted to dimension N_MAX_FDCNG in operation 900 by an improved extrapolation method to the FDNCG 24 input domain by encoder 202. The extrapolation method results in an extended input target of length N_MAX_FDCNG.

[0140] In a first operation, the FDCNG target_wb signal is extrapolated in the FDCNG domain by applying a shorter N=21 DCT type II transform as follows: dct_target_wb=dct(target_wb) Here, target_wb has dimension 21 (N_WB), and the applied DCT function is for the same dimension, i.e., N_WB is lower than NMAX_FDCNG, so an additional analysis DCT is performed using dimension N_WB (==21). This is to avoid storing several different DCT domain FD-CNG codebooks. Therefore, the stored mantissa and scale factor values ​​can be optimized for one single larger DCT length (e.g., 24), saving ROM space.

[0141] In this example, a DCT-II (N_WB) is performed only to obtain coefficients 0 to (Ntr_WB-1). This is to reduce the complexity of the additional DCT-II (N==21) analysis and subsequent extrapolation. By not analyzing higher frequency components (e.g., upper basis vectors), a truncation length other than Ntr_WB==18 can be used in this DCT analysis operation. However, it may be appropriate to provide somewhat more detail in the target extrapolation than the truncation used for the stored DCT(24) codebook.

[0142] In this example, the DCT-II (N_WB==21) analysis is truncated after 18 coefficients (i.e., 85.7% of the total bandwidth), while the stored DCT-II (N==24) vector is truncated at 75% of the bandwidth.

[0143] 10, the DCT-II (N=21) is shown as the set of DCT coefficients of the input signal target_wb (i.e., the scale factors for the cosine basis vectors of the DCT), with Ntr_WB coefficients in dct_target_wb. For example, dct_target_wb[0] is the scaling of the DC basis vector "BAS[0]," and dct_target_wb[6] is the scaling of the 3-periodic basis vector "BAS[6]" shown in FIG. 10. There are also DCT-II (N=21) basis vectors (used for extrapolation) available either as stored in the transform matrix bas(k,t), or, alternatively, by dynamically constructing the basis vectors using the IDCT(type II, N=N_WB) formula, as follows: TIFF2026508288000021.tif53170

[0144] Each available basis vector can be accessed as follows: Let bas(k), k∈0...(Ntr_WB-1) specify the kth basis vector, where bas(k,t) indexes the full matrix using the time index element t∈0...(N_WB-1) of each bas(k) basis vector.

[0145] Furthermore, the input signal target_wb is then extended by (N_MAX_FDCNG-N_WB) samples by extrapolation using a subset of the DCT basis vectors and scaling each basis vector bas(k) by the corresponding DCT coefficient dct_target_wb(i), followed by summation of the extended parts of the basis vectors.

[0146] This extension / extrapolation ensures that the extended target_wb signal target_ext will maintain the major frequency components in target_wb without adding undesirable and unnecessary extension / extrapolation noise that would cause degraded overall VQ performance.

[0147] Other extrapolation methods, such as zero extension, or repeating the last value, interpolation, or linear or polynomial extrapolation, will not preserve the cosine waveform content to the same extent after DCT-II(24) analysis. Preserving undistorted tonality in the extended signal as the extended cosine is important because the subsequent transform step (DCT-II(N==24)) is also based on cosine analysis.

[0148] Moreover, to save complexity, the target_wb signal can be reused for the initial part of the extension signal target_ext as follows: target_ext(t)=target_wb(t),t∈0...(N_WB-1)

[0149] Alternatively, this initial part can be recreated using IDCT-II (N=N_WB) by using the available DCT coefficients up to coefficient (Ntr_WB-1).

[0150] Furthermore, an extension creation by scaled basis vector summation can be achieved. Due to the reflexive nature of DCT type II basis vectors, only the higher part of the basis vectors is actually used for the proposed extrapolation method. In some embodiments, this can be achieved as follows: TIFF2026508288000022.tif88170

[0151] In a final operation, the extended target signal is copied to the full-length target buffer as follows: target(t)=target_ext(t),t∈0...(N_MAX_FDCNG-1)

[0152] In some embodiments, the encoder 202 may then continue searching the DCT-II (N=24) domain codebook in the manner previously described, starting at block 702, using the augmented input signal target(t).

[0153] In Figure 10, the extrapolation method for the first seven unscaled DCT-II (N==21) basis vectors is outlined.

[0154] Note that while the extrapolation described here is performed in the normalized FDCNG envelope signal domain of the incoming input, the extrapolation may equivalently be performed after intermediate subtraction and global scaling (i.e., after operation 702 or after operation 704 in FIG. 9). In particular, FIG. 10 shows several basis vectors that have been initially expanded and / or extrapolated and then scaled using DCT21 coefficients. These scaled basis vectors are then summed to produce the expanded portion of the target as output.

[0155] The search process continues as previously described above in FIG. 7 for a target vector of size N_MAX_FDCNG, and then proceeds entirely to block 902 in FIG. 9, where IDCT synthesis can be performed using the full vector length of N_MAX_FDCNG so that the error contribution of the upper extension can be calculated.

[0156] At block 710 of FIG. 9, residual calculations in preparation for stage #2 may be performed over a shorter N_WB samples.

[0157] In block 902, the MSE for each candidate is updated to properly reflect the mean squared error of the reduced dimension N_WB (i.e., the core DCT-II (N==24) search, the MSE reflects the error in the extended DCT-II (N=24) domain and needs to be updated for stage #2 and the forward search). If there is a new worst candidate after this MSE update for dimension N_WB, the p_max index of the worst candidate is also updated.

[0158] In some embodiments, the MSE stored in the dist[Nc] vector is updated by subtracting the MSE contribution for the extension part, ext_err[c], as follows: TIFF2026508288000023.tif98170

[0159] Furthermore, p_max may be updated using p_max=maximum(dist, Nc). In particular, a new current worst candidate among all 8(Nc) in vector dist is established using the MSE of the shorter dimension N_WB.

[0160] In some embodiments, the search portion of the first stage (e.g., Stage #1) may be finished, using the stored codebook preparation of DCT-II (N=24) for the candidate indexes of the second stage (e.g., Stage #2). Thus, Nc candidate indices are available in indices[], and their MSEs for the shorter dimension N_WB are available in dist[Nc]. Furthermore, p_max points to the worst candidate from the first stage.

[0161] 11 illustrates an overall synthesis model that may be used in both the encoder and / or decoder. For example, FIG. 11 illustrates generally how the encoder 202 may reconstruct an FD-CNG vector in operation 120. Referring to FIG. 11, the encoder 202 obtains an index idx_full 1100 in the range [0...(Nv-1)], which contains Ns-1 segments and corresponding column shift values ​​1102 (e.g., described herein as "col_shift"). In the case of a decoder, the above-described fdcng_final idx_full There will only be one.

[0162] The encoder 202 or decoder 212 performs a segment-wise and coefficient-wise upshift in operation 1104. In operation 1106, the encoder 202 or decoder 212 performs an inverse DCT type II transform and scales the output down to the original unscaled FDCNG domain in operation 1108.

[0163] At operation 1110, the intermediate stage #1 codebook vector (i.e., the vector shown as midQ

[24] vector) is added to the fdcng_final idx_full Appended to vectors to output and / or create vectors.

[0164] The operation of the encoder 202 (e.g., implemented using the structure of the block diagram of FIG. 3) will now be described with reference to the flowchart of FIG. 12, in accordance with some embodiments of the inventive concepts. For example, modules may be stored in the memory 310 of FIG. 3 that provide instructions such that, when the instructions of the modules are executed by the respective encoder processing circuitry, the encoder 202 performs the respective operations of the flowchart.

[0165] Figure 12 illustrates operations performed by the encoder 202, according to some embodiments. Referring to Figure 12, in block 1201, the encoder 202 obtains a DCT target vector (e.g., a "dct_target" vector). In some embodiments, the operations of block 1201 may be collectively represented by blocks 1301-1307 of FIG. 13. More specifically, FIG. 13 illustrates operations performed by the encoder 202, according to some embodiments, to obtain a dct_target vector. Referring to FIG. 13, in block 1301, the encoder 202 obtains an input target vector. In block 1303, the encoder 202 removes a global offset value vector (e.g., a median value vector) from the input target vector to form an offset target vector. In block 1305, the encoder 202 applies a global scale factor to the offset target vector. In block 1307, the encoder 202 transforms the offset target vector into a target discrete cosine transform search domain to form the dct_target vector.

[0166] As previously described, the input target vectors used in obtaining the dct_target vectors may have dimensions different from the dimensions of the codebook 104. As shown in block 1401 of FIG. 14, the encoder 202 extrapolates the input target vectors to the dimensions of the codebook. In some embodiments, the encoder 202 extrapolates the input target vectors to the dimensions of the codebook by extrapolating the input target vectors through an extension of the input domain basis vectors for the DCT transform used. In some embodiments, the extension of the input domain basis vectors for the DCT transform used is based on a subset of the input domain basis vectors.

[0167] Returning to FIG. 12, in block 1203, the encoder 202 performs a suboptimal pairwise interior search in each segment of a codebook 104 having a plurality of segments, each segment having a truncated vector that is different from the truncated vectors of other segments of the plurality of segments, to determine a pairwise initial candidate set from each of the plurality of segments and form a plurality of pairwise initial candidates from the suboptimal pairwise interior search.

[0168] Figure 15 shows an example embodiment of performing a suboptimal pairwise inner search in codebook 104 (as shown in block 1203 of Figure 12). Referring to Figure 15, in block 1501, the encoder 202 initializes, for each segment of a plurality of segments, a segment pair to a value large enough to ensure that both values ​​in the segment pair will be updated.

[0169] In block 1503, the encoder 202 determines, for each vector index in each segment, whether the mean squared error (MSE) of the vector index being analyzed is less than the MSE of the segment pair.

[0170] In block 1505, the encoder 202 updates the pairwise initial candidate set to include the vector index in response to the MSE of the vector index being less than the worst MSE of the segment pair. In block 1507, the encoder 202 updates the segment pair to include the vector index in response to the MSE of the vector index being less than the best MSE of the segment pair.

[0171] Alternatively, and in some embodiments, the encoder 202 does not update the segment pair to include the vector index in response to the MSE of the current vector being higher than the worst MSE of the segment pair. Returning to Figure 12, at block 1205, the encoder 202 (optionally) performs a post-optimization on the plurality of pairwise initial candidates to replace several of the plurality of pairwise initial candidates with the list of candidate neighbors, thereby forming a plurality of pairwise final candidates.

[0172] 16A-16B show an example embodiment of performing post-optimization (as shown in block 1205 of FIG. 12). Referring to FIG. 16A, in block 1601, the encoder 202 determines which of the plurality of pairwise initial candidates has the lowest MSE of the plurality of pairwise initial candidates. For each pairwise initial candidate other than the pairwise initial candidate with the lowest MSE, the encoder 202 performs the operations of blocks 1603-1613.

[0173] In block 1603, the encoder 202 compares the MSE of the pairwise initial candidate with the MSE of its neighbor vector in the forward direction.

[0174] In block 1605, the encoder 202 updates the pairwise initial candidate to the neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the forward direction.

[0175] In block 1607, in response to the pairwise initial candidate updated to the neighbor vector in the forward direction having a lower MSE than the pairwise initial candidate with the lowest MSE, the encoder 202 sets the pairwise initial candidate updated to the neighbor vector in the forward direction as the pairwise candidate with the lowest MSE.

[0176] In block 1609, the encoder 202 compares the MSE of the pairwise initial candidate with the MSE of the neighbor vector in the reverse direction. In block 1611, the encoder 202 updates the pairwise initial candidate to the neighbor vector in the reverse direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the reverse direction.

[0177] Referring to FIG. 16B, in block 1613, in response to the pairwise initial candidate updated to the neighbor vector in the reverse direction having a lower MSE than the pairwise initial candidate with the lowest MSE, the encoder 202 sets the pairwise initial candidate updated to the neighbor vector in the reverse direction as the pairwise candidate with the lowest MSE.

[0178] In some embodiments, the pairwise initial candidate is compared with the next neighbor vector in the forward and backward directions. This is shown in Figure 17. Referring to Figure 17, in block 1701, the encoder 202 compares the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the forward direction.

[0179] In block 1703, the encoder 202 updates the pairwise initial candidate to the next neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the forward direction.

[0180] In block 1705, the encoder 202 compares the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the backward direction.

[0181] In block 1707, the encoder 202 updates the pairwise initial candidate to the next neighbor vector in the backward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the backward direction.

[0182] In some embodiments, the next neighbor vector and neighbor vector in the forward direction are part of a forward circular MSE neighboring index list, and the next neighbor vector and neighbor vector in the reverse direction are part of a reverse circular MSE neighboring index list. The forward and reverse circular MSE neighboring index lists are created based on a vector ordering mse_order_circ of length N across all segments in the complete concatenated codebook cb_temporary_full.

[0183] Returning to FIG. 12, in block 1207, the encoder 202 reconstructs the multiple pairwise final candidates using an inverse type-II discrete cosine transform (DCT-II) to transform the multiple pairwise final candidates back to the original domain.

[0184] As previously explained, the input target vector used in obtaining the dct_target vector may have dimensions different from the dimensions of the codebook 104. When this occurs, as shown in block 1403 of Figure 14, the encoder 202 updates the reconstructed candidates to the dimensions of the input target vector.

[0185] 18 illustrates an exemplary embodiment of reconstructing multiple pairwise final candidates. Referring to FIG. 18, in block 1801, the encoder 202 obtains an index idx_full 1100 in the range [0...(N_v-1)], which includes multiple segments in the range 0 to Ns-1 and corresponding column shift values, e.g., col_shift values ​​1102.

[0186] In block 1803, the encoder 202 performs a segment and coefficient-based upshift. In block 1805, the encoder 202 performs an inverse discrete cosine transform (DCT) Type II transform. In block 1807, the encoder 202 scales down the output vector from the DCT Type II transform to the original unscaled FD-CNG domain vector. In some embodiments, scaling can include scaling up or scaling down.

[0187] In block 1809, the encoder 202 idx_full To form a vector, a global offset value vector (eg, intermediate value and / or intermediate stage #1 codebook vector) is added to the unscaled FD-CNG domain vector.

[0188] A similar reconstruction may be performed in the decoder 212. This is shown in Figure 19. Referring to Figure 19, in block 1901, the decoder 212 receives an index corresponding to a segment, a segment codebook vector (e.g., a mantissa value vector), an associated column shift value, and a global offset value vector.

[0189] In block 1903, the decoder 212 performs an upshift operation on the segment codebook vector using the column shift value to form an upshifted vector. In block 1905, the encoder 202 performs an inverse DCT type II transform of the upshifted vector to produce an output vector. In block 1907, the decoder 212 scales down the output vector from the inverse DCT type II transform to the original unscaled FD-CNG domain vector.

[0190] In block 1909, the decoder 212 generates the final output vector (e.g., fdcng_final idx_fullAdd a global offset vector (e.g., an intermediate vector) to the unscaled FD-CNG domain vector to form the vector and / or final FDCNG domain vector.

[0191] 10 is a summary of ROM savings in the encoder embodiment described above.

[0192] The total ROM Word8 storage in the proposal is sum([128 170 272 1404) = 1974 bytes.

[0193] The forward and reverse circular list vectors for improved searching each add 128 Word8, or +256 bytes.

[0194] The total ROM storage for the original LBG codebook was 3072 Word16 = 6144 bytes.

[0195] Total ROM storage for the original LBG codebook: 3072 single precision floats = 9216 bytes.

[0196] Approximately, the table ROM reduction is (6144-(1974+256)) / 6144 => approximately 63%.

[0197] Summary of WMOPS savings per cycle (operation) for the encoder embodiment described above.

[0198] The number of inner loop coefficients to be processed was reduced by 30% (from 3072 to 1974).

[0199] The inner best MSE update loop (running 128 times) was optimized (using pairs) to use only 7-8 cycles.

[0200] In particular, the reference uses approximately 35 cycles to maintain and update the list of the best Nc=8 candidates in the worst case. Therefore, a worst-case (WC) savings of (35-7)=28 ops (or cycles) can be expected for updating the best candidates.

[0201] In our proposal, the MSE calculation for each vector is increased by 1op*tLen due to the upshifting of the coefficient mantissa, and the MSE savings for each index cost 2 ops, but this increase (1op*tLen+2ops) is always lower than the WC savings of the best MSE update loop (28).

[0202] When used for target vectors of a different dimension (e.g., N_WB), similar inner-loop MSE savings can be achieved, for example, to use a codebook 104 created for dimension 24 (NMAX_FDCNG).

[0203] While the computing devices described herein (e.g., decoders, audio object renderers, encoders, hosts) may include the depicted combinations of hardware components, other embodiments may comprise computing devices with different combinations of components. It should be understood that these computing devices may comprise any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, obtaining, or similar operations described herein may be performed by processing circuitry, which may process information by, for example, converting obtained information to other information, comparing the obtained or converted information with information stored in a network node, and / or performing one or more operations based on the obtained or converted information and as a result of the processing making a decision. Moreover, while a component is illustrated as a single box located within a larger box or nested within multiple boxes, in reality the computing device may comprise multiple different physical components that make up the single depicted component, and functionality may be partitioned among the separate components. For example, a communications interface may be configured to include any of the components described herein, and / or the functionality of those components may be partitioned between the processing circuitry and the communications interface. In another example, non-computationally intensive functionality of any of such components may be implemented in software or firmware, and computationally intensive functionality may be implemented in hardware.

[0204] In some embodiments, some or all of the functionality described herein may be provided by a processing circuit executing instructions stored in a memory, which in some embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by a processing circuit without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hardwired manner. In any of these particular embodiments, the processing circuit may be configured to perform the described functionality, regardless of whether it executes instructions stored on a non-transitory computer-readable storage medium. Benefits provided by such functionality are enjoyed not by the processing circuit alone or by other components of the computing device, but by the computing device as a whole and / or by end users and wireless networks generally. While the computing devices described herein (e.g., UEs, encoders, decoders, network nodes, hosts) may include the depicted combinations of hardware components, other embodiments may comprise computing devices with different combinations of components. It should be understood that these computing devices may comprise any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, obtaining, or similar operations described herein may be performed by processing circuitry, which may process the information by, for example, converting the obtained information into other information, comparing the obtained or converted information with information stored in the network node, and / or performing one or more operations based on the obtained or converted information and as a result of the processing making a decision.Moreover, while components are illustrated as a single box located within a larger box or nested within multiple boxes, in reality a computing device may comprise multiple different physical components that make up the single depicted component, and functionality may be partitioned among the separate components. For example, a communications interface may be configured to include any of the components described herein, and / or functionality of those components may be partitioned between processing circuitry and the communications interface. In another example, non-computationally intensive functionality of any of such components may be implemented in software or firmware, and computationally intensive functionality may be implemented in hardware.

[0205] In some embodiments, some or all of the functionality described herein may be provided by a processing circuit executing instructions stored in a memory, which in some embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuit without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hardwired manner. In any of these particular embodiments, the processing circuit may be configured to perform the described functionality, regardless of whether or not it executes instructions stored on a non-transitory computer-readable storage medium. Benefits provided by such functionality are not limited to the processing circuit alone or to other components of the computing device, but are enjoyed by the computing device as a whole and / or by end users and wireless networks generally.

[0206] Below, several exemplary tables are presented that are used in the detailed description of various embodiments. For example, Table 4 shows an exemplary global scaling and global subtraction table. In particular, the second row of Table 4 shows an exemplary global scaling constant, and the third row shows an exemplary offset vector. TIFF2026508288000024.tif89170

[0207] Segment Codebook for MSVQ Stage #1

[0208] Total ROM Word8 storage is sum([128 170 272 1404) = 1974 bytes.

[0209] In some embodiments, examples of the above-mentioned segment codebooks are shown in Table 5. In particular, these segment codebooks may include "optimized" segment codebooks that include a mantissa constraint of 8 and an exponent that is not significantly constrained. For example, these segment codebooks may be signal-to-noise ratio (SNR)-limited or granularity-limited. TIFF2026508288000025.tif255169TIFF2026508288000026.tif255169TIFF20265082880 00027.tif255169TIFF2026508288000028.tif255169TIFF2026508288000029.tif238170

[0210] Table 6 below shows exemplary segment scaling factors. TIFF2026508288000030.tif89170

[0211] Circularly ordered vector

[0212] Table 7 below shows an example MSE sorted cyclic neighbor table (i.e., mse_order_circ[]) for the codebook shown above. In particular, Table 7 was created using a solution of the TSP problem (e.g., the closed-loop TSP problem) with a convex hull approach. Table 7 further includes the forward neighbor vector, neighb_mse_fwd, mentioned above, which is the preferred way to traverse the cyclic MSE neighbor list in the forward direction. Additionally, Table 7 also includes the reverse vector, neighb_mse_rev, mentioned above in the description, which is the preferred way to traverse the cyclic MSE neighbor list in the reverse direction. TIFF2026508288000031.tif255169TIFF2026508288000032.tif27170

Claims

1. 1. A method implemented by an encoder, the method comprising: Obtaining a Discrete Cosine Transform (DCT) target vector (1201); performing (1203) a suboptimal pairwise interior search in each segment of a codebook (104) having a plurality of segments, each segment having a truncated vector different from the truncated vectors of other segments of the plurality of segments, to determine a pairwise initial candidate set from each of the plurality of segments and form a plurality of pairwise initial candidates from the suboptimal pairwise interior search, wherein the DCT target vector is a target of the suboptimal pairwise interior search in each of the plurality of segments; Reconstructing 1207 the final candidates using an inverse type II discrete cosine transform (DCT-II) to transform the final candidates into final candidate data in the original domain; providing the final candidate data to a second stage of a multi-stage vector quantizer (1209); A method comprising:

2. The method of claim 1 , wherein the plurality of segments comprises four segments.

3. obtaining the DCT target vectors, Obtaining an input target vector (1301); removing 1303 a global offset value vector from the input target vector to form an offset target vector; applying 1305 a global scale factor to the offset target vector to form a scaled offset target vector; Transforming the scaled offset target vector into a target discrete cosine transform search domain to form the DCT target vector (1307).

3. The method of claim 1 or 2, comprising:

4. In response to the input target vector having a dimension different from a dimension of the codebook, extrapolating the input target vector to the dimension of the codebook (1401); further comprising 4. The method of claim 3, wherein transforming the plurality of final candidates into a plurality of reconstructed final candidates in the original domain comprises updating (1403) the plurality of reconstructed candidates to the dimensions of the input target vector.

5. 5. The method of claim 4, wherein extrapolating the input target vector to the dimension of the codebook comprises extrapolating the input target vector through an extension of input domain basis vectors for a DCT transform used.

6. The method of claim 5 , wherein the extension of input domain basis vectors for the DCT transform used is based on a subset of input domain basis vectors.

7. performing the suboptimal pairwise internal search for each segment of the plurality of segments, initializing a segment pair to a value large enough to ensure that both values ​​in the segment pair will be updated (1501); For each vector index in each segment, determining 1503 whether the mean square error (MSE) of the vector index being analyzed is less than the MSE of the segment pair; updating the pairwise initial candidate set to include the vector index in response to the MSE of the vector index being less than the worst MSE of the segment pair (1505); updating the segment pair to include the vector index in response to the MSE of the current vector being less than the best MSE of the segment pair (1507).

7. The method of claim 1, comprising:

8. performing post-optimization on the plurality of pairwise initial candidates determining 1601 which pairwise initial candidate of the plurality of pairwise initial candidates has the lowest mean square error (MSE) of the plurality of pairwise initial candidates; For each pairwise initial candidate other than the pairwise initial candidate having the lowest MSE, comparing the MSE of the pairwise initial candidates with the MSE of neighbor vectors in the forward direction (1603); updating the pairwise initial candidate to the neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the forward direction (1605); in response to the pairwise initial candidate updated to the neighbor vector in the forward direction having a lower MSE than the pairwise initial candidate with the lowest MSE, setting the pairwise initial candidate updated to the neighbor vector in the forward direction as the pairwise candidate with the lowest MSE (1607); comparing 1609 the MSE of the pairwise initial candidate with the MSE of a neighbor vector in the reverse direction; updating the pairwise initial candidate to the neighbor vector in the reverse direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the reverse direction (1611); and setting the pairwise initial candidate updated to the neighbor vector in the reverse direction as the pairwise candidate with the lowest MSE in response to the pairwise initial candidate updated to the neighbor vector in the reverse direction having a lower MSE than the pairwise initial candidate with the lowest MSE (1613).

8. The method of claim 1, comprising:

9. comparing the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the forward direction (1701); updating the pairwise initial candidate to the next neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the forward direction (1703); comparing the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the reverse direction (1705); updating the pairwise initial candidate to the next pairwise vector in the reverse direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the reverse direction (1707); The method of claim 8 further comprising:

10. 10. The method of claim 8 or 9, wherein the next neighbor vector and the neighbor vector in the forward direction are part of a forward circular MSE neighboring index list, and the next neighbor vector and the neighbor vector in the reverse direction are part of a reverse circular MSE neighboring index list.

11. 11. The method of claim 10, wherein the forward and backward circular MSE neighboring index lists are created based on a vector ordering mse_order_circ of length Nv across all segments in a complete concatenated codebook cb_temporary_full.

12. Reconstructing multiple pairs of finalists Obtaining 1801 an index idx_full in the range [0...(Nv-1)], which includes a number of segments in the range 0 to Ns-1 and corresponding column shift values; performing a per-segment and per-coefficient upshift of the plurality of segments using the column shift values ​​for the indexes to form upshifted segments (1803); performing an inverse discrete cosine transform (DCT) type II transform of the upshifted segments (1805); Scaling 1807 the output vector from the inverse DCT type II transform back to the original unscaled FDCNG domain vector; fdcng_final ｉｄｘ＿ｆｕｌｌ adding 1809 a global offset value vector to the unscaled frequency domain-comfort noise generation (FD-CNG) domain vector to form a vector; 12. The method of any one of claims 1 to 11, comprising:

13. The method of claim 1 , wherein the truncated vector comprises vector segments having different degrees of high frequency content compared to other segments.

14. The method of claim 1 , wherein the final candidate data includes one or more of a vector index, a reconstructed candidate vector, and / or a transformed codebook entry.

15. 12. The method of claim 1, further comprising: performing a post-optimization on the plurality of pairwise initial candidates (1205) to replace a plurality of the plurality of pairwise initial candidates with a list of candidate neighbors, thereby forming a plurality of final candidates.

16. 1. A method in a decoder (212) for reconstructing a target vector, said method comprising: receiving 1901 an index corresponding to a segment, an associated column shift value, a segment codebook vector, and a global offset value vector; performing an upshift operation on the target vector using the column shift value to form an upshifted vector (1903); performing 1905 an inverse discrete cosine transform (DCT) Type II transform of the upshifted vector to produce an output vector; Scaling 1907 the output vector from the inverse DCT type II transform back to the original unscaled frequency domain-comfort noise generation (FDCNG) domain vector; fdcng_final ｉｄｘ＿ｆｕｌｌ adding 1909 the global offset value vector to the unscaled FD-CNG domain vector to form a vector; A method comprising:

17. An encoder (202), A processing circuit (302); a memory (310) coupled to the processing circuit; 16. An encoder (202) comprising:

18. An encoder (202) adapted to implement a method according to at least one of claims 1 to 15.

19. An encoder (202), Obtaining a discrete cosine transform (DCT) target vector; performing a suboptimal pairwise inner search in each segment of a codebook (104) having a plurality of segments, each segment having a truncated vector different from the truncated vectors of other segments of the plurality of segments, to determine a pairwise initial candidate set from each of the plurality of segments and form a plurality of pairwise initial candidates from the suboptimal pairwise inner search, wherein the DCT target vector is a target of the suboptimal pairwise inner search in each of the plurality of segments; reconstructing the plurality of final candidates using an inverse type II discrete cosine transform (DCT-II) to transform the plurality of final candidates into final candidate data in the original domain; providing said final candidate data to a second stage of a multi-stage vector quantizer; An encoder (202) adapted to:

20. The encoder of claim 19, wherein the plurality of segments comprises four segments.

21. obtaining an input target vector; removing a global offset value vector from the input target vector to form an offset target vector; applying a global scale factor to the offset target vector to form a scaled offset target vector; transforming the scaled offset target vector into a target discrete cosine transform search domain to form the DCT target vector.

21. An encoder according to claim 19 or 20, adapted to obtain the DCT target vector by:

22. In response to the input target vector having a dimension different from a dimension of the codebook, extrapolating the input target vector to the dimension of the codebook; is further adapted to 22. The encoder of claim 21 , wherein transforming the plurality of final candidates into a plurality of reconstructed final candidates in the original domain comprises updating the plurality of reconstructed candidates to the dimensions of the input target vector.

23. 23. The encoder of claim 22, wherein extrapolating the input target vector to the dimension of the codebook comprises extrapolating the input target vector through an extension of input domain basis vectors for a DCT transform used.

24. 24. The encoder of claim 23, wherein the extension of input domain basis vectors used for the DCT transform is based on a subset of input domain basis vectors.

25. performing the suboptimal pairwise internal search for each segment of said plurality of segments, initializing a segment pair to a value large enough to ensure that both values ​​in said segment pair will be updated; For each vector index in each segment, determining whether the mean squared error (MSE) of the vector index being analyzed is less than the MSE of the segment pair; updating the pairwise initial candidate set to include the vector index in response to the MSE of the vector index being less than the worst MSE of the segment pair; updating the segment pair to include the vector index in response to the MSE of the current vector being less than the best MSE of the segment pair.

25. An encoder according to any one of claims 19 to 24, comprising:

26. performing post-optimization on the plurality of pairwise initial candidates determining which pairwise initial candidate of the plurality of pairwise initial candidates has the lowest mean squared error (MSE) of the plurality of pairwise initial candidates; For each pairwise initial candidate other than the pairwise initial candidate having the lowest MSE, comparing the MSE of the pairwise initial candidates with the MSE of neighbor vectors in the forward direction; updating the pairwise initial candidate to the neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the forward direction; in response to the pairwise initial candidate updated to the neighbor vector in the forward direction having a lower MSE than the pairwise initial candidate with the lowest MSE, setting the pairwise initial candidate updated to the neighbor vector in the forward direction as the pairwise candidate with the lowest MSE; comparing the MSE of the pairwise initial candidates with the MSE of neighbor vectors in the reverse direction; updating the pairwise initial candidate to the neighbor vector in the reverse direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the neighbor vector in the reverse direction; and in response to the pairwise initial candidate updated to the neighbor vector in the reverse direction having a lower MSE than the pairwise initial candidate with the lowest MSE, setting the pairwise initial candidate updated to the neighbor vector in the reverse direction as the pairwise candidate with the lowest MSE.

26. An encoder according to any one of claims 19 to 25, comprising:

27. comparing the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the forward direction; updating the pairwise initial candidate to the next neighbor vector in the forward direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the forward direction; comparing the MSE of the pairwise initial candidate with the MSE of the next neighbor vector in the reverse direction; updating the pairwise initial candidate to a next pairwise vector in the reverse direction in response to the MSE of the pairwise initial candidate being higher than the MSE of the next neighbor vector in the reverse direction.

27. The encoder of claim 26, further adapted to:

28. 28. The encoder of claim 26 or 27, wherein the next neighbor vector and the neighbor vector in the forward direction are part of a forward circular MSE neighboring index list, and the next neighbor vector and the neighbor vector in the reverse direction are part of a reverse circular MSE neighboring index list.

29. 29. The encoder of claim 28, wherein the forward and backward circular MSE neighboring index lists are created based on a vector ordering mse_order_circ of length Nv across all segments in a complete concatenated codebook cb_temporary_full.

30. Reconstructing multiple pairs of finalists Obtaining an index idx_full in the range [0...(Nv-1)] that includes a number of segments in the range 0 to Ns-1 and corresponding column shift values; performing a per-segment and per-coefficient upshift of the plurality of segments using the column shift values ​​for the indexes to form upshifted segments; performing an inverse discrete cosine transform (DCT) type II transform of the upshifted segments; scaling the output vectors from the inverse DCT type II transform back to the original unscaled FDCNG domain vectors; fdcng_final ｉｄｘ＿ｆｕｌｌ adding a global offset value vector to the unscaled frequency domain-comfort noise generation (FD-CNG) domain vector to form a vector; 30. An encoder according to any one of claims 19 to 29, comprising:

31. 20. The encoder of claim 19, wherein the truncated vectors include vector segments having different degrees of high frequency content compared to other segments.

32. 20. The encoder of claim 19, wherein the final candidate data includes one or more of a vector index, a reconstructed candidate vector, and / or a transformed codebook entry.

33. 20. The encoder of claim 19, further adapted to perform a post-optimization on the plurality of pairwise initial candidates to replace several of the plurality of pairwise initial candidates using a list of candidate neighbors, thereby forming a plurality of final candidates.

34. 16. A computer program comprising program code to be executed by a processing circuit (302) of an encoder (202), whereby execution of the program code causes the encoder (202) to perform the operations of any one of claims 1 to 15.

35. 16. A computer program product comprising a non-transitory storage medium containing program code to be executed by a processing circuit (302) of an encoder (202), whereby execution of the program code causes the encoder (202) to perform the operations of any one of claims 1 to 15.

36. A decoder (212), A processing circuit (402); a memory (410) coupled to said processing circuit; 17. A decoder (212) comprising: said memory containing instructions that, when executed by said processing circuitry, cause said decoder to perform the operations of claim 16.

37. A decoder (212) adapted to perform the operations of claim 16.

38. 17. A computer program comprising program code to be executed by a processing circuit (402) of a decoder (212), whereby execution of said program code causes said decoder (212) to perform the operations of claim 16.

39. 17. A computer program product comprising a non-transitory storage medium containing program code to be executed by a processing circuit (402) of a decoder (212), whereby execution of the program code causes the decoder (212) to perform the operations recited in claim 16.

Images