Low-complexity bandwidth expansion target generation

Abstract

An encoder, a computer program, a computer program product, and a method implemented in an encoder (210, 1108A, 1108B). The method includes receiving (1203) a full-band input frame. The method includes interpolating (620, 1205) the full-band input frame toward an intermediate frame. The method includes performing (630, 1207) a spectral inversion of the intermediate frame to produce a spectrally-reversed intermediate frame. The method includes interpolating (640, 1209) the interpolated spectrally-reversed intermediate frame to match a sampling frequency of the low-pass filter and decimation process. The method includes performing (650, 1211) low-pass filtering and decimation of the spectrally-reversed spectrum to produce a bandwidth-extended (BWE) target signal.

Description

[Technical Field]

[0001] This disclosure relates in general to communications, and more particularly to communication methods, as well as related devices and nodes, that support wireless communication for voice and audio coding. [Background technology]

[0002] Most existing communication systems operate on limited audio bandwidth. Due to the limitations of landline telephony systems, most voice services are restricted to transmitting only the lower end of the frequency spectrum and only on a single mono channel. While lower bandwidth mono signals are sufficient for most conversations, there is a desire to increase bandwidth and spatial reproduction to improve intelligibility and presence. Although the capacity of communication networks is continuously increasing, limiting the bandwidth required per communication channel remains a major concern. In mobile networks, smaller transmit bandwidth per call results in lower power consumption at both mobile devices and base stations. This leads to energy and cost savings for mobile operators, and end users experience extended battery life and increased talk time. Furthermore, as less bandwidth is consumed per user, mobile networks can serve a larger number of users in parallel.

[0003] A key characteristic of the human auditory system is that perception is frequency-dependent. In particular, human hearing becomes less accurate at higher frequencies. This has inspired so-called bandwidth expansion (BWE) techniques, in which the higher frequency band is reconstructed from the lower frequency band using only minimal additional information.

[0004] Conventional BWEs reconstruct the spectral fine structure of a signal by using an extended representation of the high-bandwidth signal's spectral envelope and a modified version of the low-bandwidth signal. When the high-bandwidth envelope is represented by a filter, the fine structure signal is often called the excitation signal. The accurate representation of the high-bandwidth envelope is perceptually more important than the fine structure. Therefore, available resources, represented by bits, are spent on the envelope representation, while the fine structure is typically reconstructed from the encoded low-bandwidth signal without additional side information. Furthermore, the temporal shape of the high-bandwidth can be modulated using a temporal envelope. The basic concept of BWE is shown in Figure 1, which will be described in more detail below in a detailed description of various embodiments.

[0005] In general stereo recording, channel pairs exhibit a high degree of similarity, or correlation. State-of-the-art stereo coding schemes exploit this correlation by employing parametric coding, where a single channel is complemented with parametric descriptions that enable reconstruction of the full stereo image. The process of reducing the channel pair to a single channel is often referred to as downmixing, and the resulting channel is the downmix channel. The downmix procedure generally attempts to maintain energy by aligning the inter-channel time difference (ITD) and inter-channel phase difference (IPD) before mixing the channels. To maintain the energy balance of the input signals, the inter-channel level difference (ILD) is also measured. The ITD, IPD, and ILD are then encoded and can be used in an inverse upmix procedure when reconstructing the stereo channel pair at the decoder. The aforementioned parameters describe the correlated components of the channel pair, but stereo channel pairs generally also contain uncorrelated components that cannot be reconstructed from the downmix. This component can be represented by the inter-channel coherence parameter (ICC). The uncorrelated components can be artificially synthesized at the stereo decoder by operating on the decoded downmix channel through an uncorrelator filter that aims to create a signal with low correlation to the decoded downmix. The strength of the uncorrelated components is then controlled by the ICC parameter. The same principle applies to multi-channel audio representations such as 5.1 and 7.1.4, as well as spatial audio representations such as ambisonics or spatial audio object coding. The number of channels is reduced by exploiting the correlation between channels and bundling the reduced channel set with metadata or parameters for channel reconstruction or spatial audio rendering at the decoder.

[0006] To facilitate stereo or spatial audio signal reproduction at low bitrates, the BWE technique can be used in combination with parametric methods of spatial reconstruction. In this case, a downmix is also created for the BWE target band. SUMMARY OF THE INVENTION

[0007] Currently, there are (one or more) certain problems. Target generation from the frequency domain is computationally complex. In a downmix encoder operating in some modes, one solution is to input 0 instead of synthesizing a target when operating in a mode where BWE is not used, but this can lead to 0s in the target frame when switching back to a mode using BWE. This leads to energy loss and transition artifacts and proceeding to a mode using BWE.

[0008] Some aspects of the present disclosure and their embodiments may provide solutions to these or other problems. According to some embodiments, a band-limited BWE target signal is extracted from a full-band target signal by a low-complexity operation consisting of linear interpolation, spectral inversion, linear interpolation, and low-pass filtering and decimation.

[0009] According to some embodiments, a method in an encoder includes receiving a full-band input frame. The method further includes interpolating the full-band input frame towards an intermediate frame. The method further includes performing an inversion of the spectrum of the intermediate frame to create a spectrally inverted intermediate frame. The method further includes interpolating the spectrally inverted intermediate frame to match the sampling frequency of a low-pass filtering and decimation process. The method further includes performing low-pass filtering and decimation of the interpolated spectrally inverted intermediate frame to create a bandwidth expansion (BWE) target signal.

[0010] Similar encoders, computer programs, and computer program products are provided.

[0011] Some embodiments may offer one or more of the following technical advantages. Various embodiments may provide a target BWE signal with no additional delay and low computational complexity.

[0012] To provide a further understanding of this disclosure, the accompanying drawings, which are included and incorporated into this application and form part of this application, illustrate some non-limiting embodiments of the inventive concept. [Brief explanation of the drawing]

[0013] [Figure 1] This is a graph illustrating the basic concept of bandwidth expansion (BWE). [Figure 2] This is a block diagram of a stereo encoder and decoder system. [Figure 3] This is a block diagram showing a downmix encoder in several embodiments. [Figure 4] This is a block diagram showing a downmix decoder in several embodiments. [Figure 5] This is a block diagram showing a BWE target extractor in several embodiments. [Figure 6] This is a flowchart showing the operation of an encoder according to several embodiments. [Figure 7] A to H are exemplary sample images and target bands according to several embodiments. [Figure 8] This is a graphical representation of interpolation in several embodiments. [Figure 9] This is a block diagram of an encoder according to several embodiments. [Figure 10] This is a block diagram of a decoder according to several embodiments. [Figure 11] This is a block diagram of a virtualization environment according to several embodiments. [Figure 12] This is a flowchart showing the operation of an encoder according to several embodiments. [Figure 13] This is a flowchart showing the operation of an encoder according to several embodiments. [Modes for carrying out the invention]

[0014] With reference to the accompanying drawings, some of the embodiments contemplated herein will be described more thoroughly. Examples of embodiments of the inventive concept are shown; embodiments are provided as examples to convey the scope of the subject matter to those skilled in the art. However, the inventive concept can be embodied in many different forms and should not be construed as being limited to the embodiments described herein. Rather, these embodiments are provided to ensure that this disclosure is thorough and complete and to fully convey the scope of the inventive concept to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive; it can be implicitly assumed that components from one embodiment exist / are used in another embodiment.

[0015] As previously shown, a target BWE signal is provided that has no additional delay and low computational complexity.

[0016] In one exemplary embodiment of the present disclosure, the embodiment operates in a stereo encoder of a stereo encoder and decoder system outlined in Figure 2. The stereo encoder 210 processes the input left channel signal and the input right channel signal in a segment called a frame. A stereo analysis and downmix block 212 performs parameter analysis and produces a downmix. For a given frame m, the input channels may be written as follows: TIFF0007909691000001.tif10170 Here, n=0,1,2,...,N indicates the number of samples in frame m, and N is the input sampling frequency f inputThis is the length of frame m in the following explanation. The following explanation assumes that frames are extracted by overlap in the encoder so that the decoder can reconstruct the stereo signal using an overlap-adding strategy. This is the analysis frame length N. a This generally means that is greater than the input frame length N. The input signals on the left and right channels are windowed with a suitable windowing function w(n) and transformed into the DFT (Discrete Fourier Transform) domain. Note that other frequency domain representations may be used here, such as a QMF (quadrature mirror filter) filter bank, a hybrid QMF filter bank, or an odd DFT (ODFT) representation composed of MDCT (modified discrete cosine transform) and MDST (modified discrete sine transform) transform components.

[0017] In parameter analysis, the frequency spectrum is divided into band b, where each band is within the range of frequency coefficients. k = k start(b) ...k end(b) b=0,1,2,...N bands -1 Corresponds to, here, N bands This indicates the total number of bandwidths. Bandwidth limits are generally set to reflect the resolution of human hearing, which suggests a narrower bandwidth for lower frequencies and a wider bandwidth for higher frequencies. Note that different bandwidth resolutions may be used for different parameters.

[0018] Next, the signal is analyzed within the parameter analysis block to extract the ITD, IPD, and ILD parameters. Furthermore, channel coherence may be analyzed and ICC parameters may be derived. These parameters are encoded by the parameter encoder 218, added to the bitstream, and stored or transmitted to the decoder. Optionally, a stereo residual bitstream may be generated by the residual encoder 216, added to the bitstream, and stored or transmitted to the decoder.

[0019] Before creating the downmix channel, it may be beneficial to compensate for the ITD and IPD to reduce cancellation and maximize the downmix energy. ITD compensation can be implemented in the time domain before frequency conversion, or in both the time domain and frequency domain, but essentially involves performing a time shift on one or both channels to eliminate the ITD. Phase matching can be implemented in different ways, but its purpose is to match the phase so that cancellation is minimized. This ensures maximum energy in the downmix. ITD and IPD adjustment can be performed in the frequency band or over the entire frequency spectrum, and preferably should be done using quantized ITD and IPD parameters to ensure that the correction can be reversed in the decoder stage. The various embodiments described herein are independent of the IPD and ITD parameter analysis and the implementation of compensation. Here, the adjusted channels for ITD and IPD are shown below. TIFF0007909691000003.tif10170

[0020] Downmix signal X M (m,k) is encoded by the downmix encoder 214 and stored or transmitted to the decoder. This encoding can be performed in the frequency domain, but it can also be performed in the time domain. In the case of a time-domain downmix encoder, the DFT combining stage processes the downmix signal x M It is required to produce at least one time-domain version of (m,n), which is fed to the downmix encoder. If the downmix encoder operates on several frequency bands of the downmix signal, some DFT synthesis may be performed to generate time-domain signals of the required bands.

[0021] The downmix encoder 214 is described in more detail in FIG. 3. The downmix encoder 214 may include at least two encoding modes, where at least one of the encoding modes 310 operates on at least two frequency bands using a low-band encoder 312 and a BWE encoder 314. The encoding mode may be selected, for example, using signal analysis to select the most appropriate mode, or the encoding mode may be selected by operating all possible modes and selecting the mode that gives the best performance for the current frame m. The low-band encoder 312 receives the low-band downmix signal x M,LO (m,n) and encodes the representation of this frequency band. The BWE encoder 314 receives, from the BWE target signal buffer 316, the delay-adjusted high-band downmix signal x’ M,HI (m,n). The buffer enables alignment of the analysis frames, providing the desired analysis length and alignment, and compensating for any possible delay from the BWE process. In general, the BWE encoder also uses parameters from the low-band encoder, such as the low-band excitation signal in a low-band ACELP (Algebraic Code-Excited Linear Prediction) encoder.

[0022] During use of the encoding mode 310, the BWE target buffer 316 is updated with the high-band downmix signal x M,HI (m,n) generated by DFT synthesis. In the encoding mode 320, the full-band downmix signal x M(m,n) is used as input to the full-band encoder 322. The full-band signal is also added to the low-complexity BWE target signal extractor 324. An alternative form of extracting the BWE target signal is to run DFT synthesis to generate the target signal, as when mode 310 is used. However, this solution can be computationally complex. Another solution is to skip target signal generation when using mode 320 and fill buffer 316 with 0s. This would give an additional complexity of almost 0s, but the buffer may partially contain 0s when the next frame is encoded using mode 310. This can lead to energy loss in the BWE region during the transition to mode 310, which negatively impacts performance.

[0023] In decoder 220, stereo parameters are decoded by parameter decoder 228, and optionally, the reconstructed residual signal is produced by residual decoder 226. Downmix decoder 224 is configured to decode and reconstruct the downmix signal encoded by downmix encoder 214. As shown in Figure 4, the downmix decoder comprises at least a decoding mode 420 having a full-band decoder 422, and a decoding mode 410 having a low-band decoder 412 and a BWE decoder 414. Similar to the encoder, the BWE decoder may use parameters from the low-band decoder, such as the low-band excitation signal of the ACELP decoder. The output of the decoding mode currently used for the frame is the reconstructed downmix. The reconstructed downmix, reconstructed stereo parameters, and optionally reconstructed residual signal are fed to stereo upmixer 222 to produce the reconstructed stereo signal.

[0024] An efficient implementation of the BWE target extractor 324 can be seen in Figure 5 and may follow the steps outlined in Figure 6. If the BWE operates over several different target bands, an optional initial step 610 may be performed by the intermediate length determiner 510 to determine the intermediate length of the interpolated signal. The intermediate length is determined by the input sampling frequency f such that the upper limit of the target band matches the Nyquist frequency in the interpolated frame. input This is determined by rescaling the input frame length N. The target bandwidth is limited (f lo ,f hi If it has an intermediate sampling frequency f inter teeth, f inter =2f hi The intermediate length is N inter teeth, The filename is TIFF0007909691000004.tif11170.

[0025] Different target bands may depend on the bandwidth of the low-bandwidth encoder. If the low-bandwidth encoder uses a lower bandwidth, the start of the BWE target band should coincide with the end of the low-bandwidth bandwidth. Generally, the BWE target band is extracted to have a small overlap with the low-bandwidth encoder bandwidth to ensure smooth frequency transitions between encoded bands.

[0026] In step 620, the interpolator 520 processes the intermediate frame x inter Full bandwidth input frame x to (m,n) M Perform interpolation on (m,n). Length N=640 and f input An exemplary input frame with a sampling frequency of 32 kHz is shown in Figure 7A, and the frequency spectrum and target bandwidth are shown in Figure 7B.

[0027] The interpolator may use linear interpolation, as shown in Figure 8. It is assumed that the sample points are connected by straight lines, and resampling uses the sample points that intersect these lines. Generally, a linear interpolation function that stretches or compresses a frame of length N1 to a frame of length N2 is used for i=0,1,...,N2. It can be written as TIFF0007909691000005.tif20170, and here, The filename is TIFF0007909691000006.tif26170.

[0028] x int The first two lines of the equation for (i) handle cases on the edge of the frame where the new sampling point is extrapolated from the last two points of the frame or the first two points of the frame. Index offset i offset This handles cases where N2 > N1, the first sample point in the stretched frame is below i=0, and the last sample point is above i=N1-1. In some cases, a displacement is desirable, denoted here by ∈. In one embodiment, ∈ is It may be defined according to TIFF0007909691000007.tif11170.

[0029] Linear interpolation offers inexpensive resampling at the expense of high aliasing. Some weak low-pass filtering is incorporated by linear interpolation. However, aliasing is limited if the resampling is not excessive. Furthermore, the BWE target signal is not directly encoded and is used only for spectral shaping, temporal shaping, and energy measurement. For this reason, the BWE target signal is not very sensitive to aliasing. Note that other variations of low-complexity interpolation may also be used.

[0030] The result of interpolation step 620 is shown in Figure 7C, where N inter= 560, and the upper limit of the target bandwidth here coincides with the Nyquist frequency in Figure 7D. In step 630, the spectral inverter 530 performs spectral inversion of the intermediate frame. This can be implemented by changing the sign of the second sample-by-sample signal. TIFF0007909691000008.tif13170

[0031] Alternatively, a spectral inversion equivalent for this purpose is: It can be obtained using TIFF0007909691000009.tif13170.

[0032] The result of spectral inversion is shown in Figure 7E, and the inverted spectrum is in Figure 7F. In step 640, the interpolator 540 adjusts the frame length to match the sampling frequency of step 650, where the low-pass filter and decimator 550 perform low-pass filtering and decimation to produce the BWE target signal. Here, the decimator performs decimation at 2, which gives an output sampling frequency of half the input sampling frequency. The low-pass filter has a cutoff frequency in the middle of the spectrum. In the example shown in Figure 7, the low-pass filter and decimator are f dec It operates at 32kHz, which means, The decimator frame length of TIFF0007909691000010.tif10170 is given, which in the illustrated example is N dec The value becomes =640. The decimator frame x after performing this interpolation. dec(m,n) is shown in Figure 7G. In the frequency spectrum shown in Figure 7H, the target band is matched here in the lower half of the spectrum. This is the frequency band that the low-pass filter and decimator 550 then extract to give the extracted BWE target signal. Note that the spectrum of the BWE target signal is inverted here. When the BWE encoder operates on the inverted frequency spectrum, the target signal is ready to be fed into the BWE target buffer 316. If the spectrum needs to be inverted, spectral inversion can be done by changing the sign of the second sample, as described previously.

[0033] The downmix encoder outputs an encoded representation from the downmix encoding mode to create a downmix bitstream. This bitstream is then joined with the bitstream of the parameter encoder 218. Optionally, a stereo residual bitstream may be produced by the residual encoder 216. The bitstream components of the active module are joined to the composite bitstream and stored or sent to the decoder.

[0034] As previously described, in decoder 220, the stereo parameters are decoded by parameter decoder 228, and optionally, the reconstructed residual signal is produced by residual decoder 226. Downmix decoder 224 is configured to decode and reconstruct the downmix signal encoded by downmix encoder 214. The output in active decoding mode is the reconstructed downmix. The reconstructed downmix, reconstructed stereo parameters, and optionally reconstructed residual signal are fed to stereo upmixer 222 to produce the reconstructed stereo signal.

[0035] Before describing the operation from the perspective of encoder 210, Figure 9 is a block diagram showing elements of encoder 210 configured to encode audio frames according to various embodiments of this specification. As shown, encoder 210 may include a network interface circuit 905 (also called a network interface) configured to provide communication with other devices / entities / functions / etc. Encoder 210 may also include a processing circuit 901 (also called a processor and processor circuit) coupled to the network interface circuit 905, and a memory circuit 903 (also called memory) coupled to the processing circuit. The memory circuit 903 may include computer-readable program code that, when executed by the processing circuit 901, causes the processing circuit to perform the operation according to embodiments disclosed herein.

[0036] In other embodiments, the processing circuit 901 may be defined to include memory such that a separate memory circuit is not required. As described herein, the operation of the encoder 210 may be carried out by the processing circuit 901 and / or the network interface circuit 905. For example, the processing circuit 901 may control the network interface 905 to send communications to the decoder 220 and / or receive communications through the network interface 905 from one or more other network nodes / entities / servers, such as other encoder nodes or a deposit server. Furthermore, modules may be stored in memory 903, and these modules may provide instructions so that the processing circuit 901 carries out its respective operations when the module's instructions are executed by the processing circuit 901.

[0037] Figure 10 is a block diagram showing elements of a decoder 220 configured to decode audio frames according to several embodiments of the inventive concept. As shown, the decoder 220 may include a network interface circuit 1005 (also called a network interface) configured to provide communication with other devices / entities / functions, etc. The decoder 220 may also include a processing circuit 1001 (also called a processor or processor circuit) coupled to the network interface circuit 1005, and a memory circuit 1003 (also called memory) coupled to the processing circuit. The memory circuit 1003 may include computer-readable program code that, when executed by the processing circuit 1001, causes the processing circuit to perform operations according to embodiments disclosed herein.

[0038] In other embodiments, the processing circuit 1001 may be defined to include memory such that a separate memory circuit is not required. As described herein, the operation of the decoder 220 may be carried out by the processor 1001 and / or the network interface 1005. For example, the processing circuit 1001 may control the network interface circuit 1005 to receive communications from the encoder 210. Furthermore, modules may be stored in memory 1003, and these modules may provide instructions so that the processing circuit 1001 performs its respective operations when the module's instructions are executed by the processing circuit 1001.

[0039] In some embodiments, various operations of the encoder 210 and / or decoder 220 may be distributed across various components. Figure 11 is a block diagram showing a virtualized environment 1100 in which functions implemented by some embodiments may be virtualized. In this context, virtualization means creating a virtual version of a device or apparatus, such as an encoder and / or decoder, which may include virtualizing a hardware platform, storage devices, and networking resources. The virtualization used herein may apply to any device or component thereof described herein and relates to an implementation in which at least a portion of the functionality is implemented as one or more virtual components. Some or all of the functionality described herein may be implemented as virtual components, run by one or more virtual machines (VMs) implemented in one or more virtual environments 1100 hosted by one or more hardware nodes, such as an encoder, decoder, network node, UE, or hardware computing device acting as a core network node.

[0040] Application 1102 (which may alternatively be referred to as a software instance, virtual appliance, network function, virtual node, virtual network function, etc.) runs in a virtualized environment 1100 to implement some of the features, functions, and / or benefits of some of the embodiments disclosed herein.

[0041] Hardware 1204 includes processing circuits, memory for storing software and / or instructions executable by the hardware processing circuits, and / or other hardware devices described herein, such as network interfaces and input / output interfaces. The software is executed by the processing circuits to instantiate one or more virtualization layers 1106 (also called hypervisors or virtual machine monitors (VMMs)), providing VM1108A and 1108B (one or more of which may commonly be referred to as VM1108), and / or may implement any of the functions, features, and / or benefits described with respect to some embodiments described herein. The virtualization layer 1106 may present VM1108 with a virtual operating platform that looks like networking hardware.

[0042] VM1108 may feature virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be powered by the corresponding virtualization layer 1106. Different embodiments of the virtual appliance 1102 may be implemented on one or more of the VM1108s, and the implementation may be carried out in different ways. Hardware virtualization is referred to as network function virtualization (NFV) in several contexts. NFV may be used to consolidate many types of network equipment onto industry-standard high-volume server hardware, physical switches, and physical storage, which may reside in data centers and customer premises equipment.

[0043] In the context of NFV, VM1108 can be a software implementation of a physical machine, where programs run as if they were running on a physical, non-virtualized machine. Each VM1108 and its portion of the hardware 1104 on which it runs, whether that hardware is dedicated to the VM and / or shared by that VM with other VMs in the VM, form a separate virtual network element. Furthermore, in the context of NFV, the virtual network function is responsible for handling specific network functions running in one or more VM1108s on the hardware 1104 and corresponds to application 1102.

[0044] Hardware 1104 may be implemented in a standalone network node with general or specific components. Hardware 1104 may implement some functions through virtualization. Alternatively, hardware 1104 may be part of a larger cluster of hardware (such as in a data center or CPE) where many hardware nodes cooperate and are managed via management and orchestration 1110, in particular overseeing the lifecycle management of applications 1102. In some embodiments, hardware 1104 is coupled to one or more radio units, each including one or more transmitters and one or more receivers, which may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more suitable network interfaces and may be used in combination with virtual components to provide a virtual node with radio capabilities, such as a radio access node or base station. In some embodiments, some signaling may be provided using a control system 1112, which may be used alternatively for communication between hardware nodes and radio units.

[0045] Next, the operation of the encoder 210 (implemented using the structure of the block diagram in Figure 9) will be described with reference to the flowchart in Figure 12, according to several embodiments of the inventive concept. For example, modules may be stored in the memory 903 in Figure 9, and these modules may provide instructions such that the encoder 210 performs each operation in the flowchart when the module's instructions are executed by the respective encoder processing circuit 901.

[0046] Referring to Figure 12, in block 1201, encoder 210 determines the intermediate length of the intermediate frame. In some embodiments, encoder 210 adjusts the input sampling frequency f until the upper limit of the target bandwidth matches the Nyquist frequency in the intermediate frame. input The intermediate length is determined by rescaling the input frame length N.

[0047] In some of these embodiments, the encoder 210 is The intermediate length is determined according to TIFF0007909691000011.tif11170, Here, the target bandwidth is limited (f lo ,f hi ) when f inter =2f hi And N inter f is the intermediate length, N is the length of the input frame, and f inter This is the intermediate sampling frequency.

[0048] In block 1203, encoder 210 receives the full-band input frame. For example, in some embodiments, encoder 210 receives the full-band input frame by receiving the full-band downmix signal generated by discrete Fourier transform (DFT) synthesis, as described above.

[0049] In block 1205, encoder 210 interpolates the full-band input frames toward intermediate frames. In some embodiments, encoder 210 interpolates the full-band input frames by interpolating the full-band input frames using linear interpolation.

[0050] In some embodiments using linear interpolation, the encoder 210 interpolates for i=0,1,...,N2, Linear interpolation is used to stretch or compress a frame of length N1 to a frame of length N2 according to TIFF0007909691000012.tif20170, where, TIFF0007909691000013.tif26170, x int (i) is the interpolated sampled value at point i, x(0) is the sampling point of the source vector x at the first point 0, x(1) is the sampling point of the source vector x at the second point 1, i frac This is a fractional point where the source vector x is estimated, and i offset is the offset, and ∈ is the displacement. As explained earlier, offset i offset N2 <bold> >< / bold> This handles cases where N1 is the value of i, the first sample point in the expanded frame is below i=0, and the last sample point is above i=N1-1.

[0051] In some of these embodiments, ∈ is It is defined according to TIFF0007909691000014.tif11170.

[0052] Other displacements may be used.

[0053] In block 1207, encoder 210 performs spectral inversion of the intermediate frame to produce spectrally inverted intermediate frames. In some embodiments, encoder 210, Perform spectral inversion according to TIFF0007909691000015.tif13170.

[0054] In some other embodiments, the encoder 210 is Perform spectral inversion according to TIFF0007909691000016.tif13170.

[0055] In block 1209, encoder 210 interpolates spectrally inverted midframes to match the sampling frequency of the low-pass filtering and decimation processes. In block 1211, encoder 210 performs low-pass filtering and decimation on the interpolated spectrally inverted midframes to produce a bandwidth-extended (BWE) target signal.

[0056] In some embodiments, as shown in Figure 13, the encoder 210 selects the cutoff frequency for low-pass filtering to be substantially near the center of the spectrum, as shown in block 1301. In block 1303, the encoder 210 performs low-pass filtering and decimation of the inverted spectrum to produce a BWE target signal by performing low-pass filtering and decimation to match the BWE target signal in the lower half of the spectrum.

[0057] In some of these embodiments, the encoder 210 is The decimator frame length is determined according to TIFF0007909691000017.tif10170, Here, N dec The length of the decimator frame is N inter is the intermediate length, and f dec This is the operating frequency of the decimator, and f inter This is the frequency of the intermediate frame.

[0058] Returning to Figure 12, in block 1213, encoder 210 inputs the BWE target signal to BWE encoder 314 via BWE target buffer 316, and BWE encoder 314 and BWE target buffer 316 are part of encoder 210 operating on at least two frequency bands. In one embodiment, BWE encoder 314 and BWE target buffer 316 are part of downmix encoder 214. The downmix encoder 214 of encoder 210 has at least two encoding modes in some embodiments, and at least one of the at least two encoding modes has BWE encoder 314.

[0059] In some of these embodiments, the encoder 210 operates on a downmix signal in a parametric stereo encoder.

[0060] The various operations from the flowchart in Figure 12 may be arbitrary with respect to several embodiments of the communication device and the associated methods. With respect to the method of Exemplary Embodiment 1 (described below), for example, the operations of blocks 1201 and 1213 in Figure 12 may be arbitrary.

[0061] The computing devices described herein (e.g., encoders, decoders, UEs, network nodes) may include the shown combinations of hardware components, but other embodiments may comprise computing devices with different combinations of components. It should be understood that these computing devices may comprise any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, acquiring, or similar operations described herein may be performed by processing circuits, which may process information by, for example, converting acquired information to other information, comparing acquired or converted information to information stored in a network node, and / or performing one or more operations based on the acquired or converted information and as a result of the processing making a decision. Furthermore, although components are illustrated as a single box located within a larger box, or as a single box nested within multiple boxes, in practice, computing devices may comprise multiple different physical components that constitute a single shown component, and functions may be separated between the distinct components. For example, a communication interface may be configured to include any of the components described herein, and / or the functions of those components may be separated between the processing circuit and the communication interface. In another example, the non-computationally intensive functions of any of such components may be implemented in software or firmware, while the computationally intensive functions may be implemented in hardware.

[0062] In some embodiments, some or all of the functions described herein may be provided by a processing circuit that executes instructions stored in memory, which in some embodiments may be a computer program product in the form of a non-temporary computer-readable storage medium. In alternative embodiments, some or all of the functions may be provided by a processing circuit without executing instructions stored in a separate or individual device-readable storage medium, such as in a hardwired manner. In any of those particular embodiments, whether or not it executes instructions stored in a non-temporary computer-readable storage medium, the processing circuit may be configured to perform the functions described. The benefits provided by such functions are enjoyed by the processing circuit alone, or by the computing device as a whole, but not limited to other components of the computing device, and / or generally by the end user and the wireless network.

[0063] Embodiment Embodiment 1. A method implemented in encoders (210, 1108A, 1108B), wherein the method is Receiving full-band input frames (1203), Interpolating the full bandwidth input frame toward the intermediate frame (620, 1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (630, 1207), Interpolating spectrally inverted intermediate frames (640, 1209) to match the sampling frequencies of the low-pass filter and decimation process, To create a bandwidth-extended (BWE) target signal, perform low-pass filtering and decimation on the interpolated spectrally inverted midframes (650, 1211) and Methods that include... Embodiment 2. The method according to Embodiment 1, wherein interpolating the full-band input frame includes using linear interpolation in interpolating the full-band input frame. Embodiment 3. Using linear interpolation to interpolate the full bandwidth input for i=0,1,...,N2, This includes using linear interpolation to stretch or compress a frame of length N1 to a frame of length N2, in accordance with TIFF0007909691000018.tif20170, where, TIFF0007909691000019.tif26170, x int (i) is the interpolated sampled value at point i, x(0) is the sampling point of the source vector x at the first point 0, x(1) is the sampling point of the source vector x at the second point 1, i offset The method according to Embodiment 2, wherein is the offset and ∈ is the displacement. Embodiment 4. ∈ is The method according to Embodiment 3, as defined in accordance with TIFF0007909691000020.tif11170. Embodiment 5. The method according to any one of Embodiments 1 to 4, wherein receiving a full-band input frame includes receiving a full-band downmix signal generated by discrete Fourier transform (DFT) synthesis. Embodiment 6. Performing spectral inversion of the intermediate frame is The method according to any one of Embodiments 1 to 5, comprising performing spectral inversion according to TIFF0007909691000021.tif13170. Embodiment 7. Performing spectral inversion of the intermediate frame is The method according to any one of Embodiments 1 to 5, comprising performing spectral inversion according to TIFF0007909691000022.tif13170. Embodiment 8. Determine the intermediate length of the intermediate frame (610, 1201). A method according to any one of embodiments 1 to 7, further comprising: Embodiment 9. Determining the intermediate length involves adjusting the input sampling frequency f until the upper limit of the target bandwidth matches the Nyquist frequency in the intermediate frame. input The method according to Embodiment 8, comprising rescaling the length N of the input frame. Embodiment 10. Determining the intermediate length is This includes determining the intermediate length according to TIFF0007909691000023.tif11170, Here, the target bandwidth is limited (f lo ,f hi ) when f inter =2f hi And N inter f is the intermediate length, N is the length of the input frame, and f input The method according to embodiment 8 or 9, wherein the intermediate sampling frequency is . Embodiment 11. The method according to any one of Embodiments 1 to 10, further comprising selecting the cutoff frequency of the low-pass filter to be substantially near the center of the spectrum (1301). Embodiment 12. The method according to Embodiment 11, wherein performing low-pass filtering and decimation of the inverted spectrum to produce a BWE target signal is performed to match the BWE target signal in the lower half of the spectrum (1303). Embodiment 13. The decimator frame length is, Determined according to TIFF0007909691000024.tif10170, Here, N dec The length of the decimator frame is N inter is the intermediate length, and f dec This is the operating frequency of the decimator, and f inter The method according to embodiment 12, wherein the frequency is the frequency of the intermediate frame. Embodiment 14. The method according to any one of Embodiments 1 to 13, further comprising inputting a BWE target signal to a BWE encoder (314) via a BWE target buffer (316) (1213), wherein the BWE encoder (314) and the BWE target buffer (316) are part of an encoder (210, 1108A, 1108B) operating on at least two frequency bands. Embodiment 15. The method according to Embodiment 14, wherein the BWE encoder (314) and the BWE target buffer (316) are part of the downmix encoder (214) of the encoders (210, 1108A, 1108B). Embodiment 16. The method according to Embodiment 15, wherein the downmix encoder (214) of the encoders (210, 1108A, 1108B) has at least two encoding modes, and at least one of the at least two encoding modes has a BWE encoder (314). Embodiment 17. The method according to Embodiment 16, wherein the encoders (210, 1108A, 1108B) operate on a downmix signal in a parametric stereo encoder. Embodiment 18. Encoders (210, 1108A, 1108B), Receiving full-band input frames (1203), Interpolating the full bandwidth input frame toward the intermediate frame (620, 1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (630, 1207), Interpolating spectrally inverted intermediate frames (640, 1209) to match the sampling frequencies of the low-pass filter and decimation process, To create a bandwidth-extended (BWE) target signal, perform low-pass filtering and decimation on the interpolated spectrally inverted midframes (650, 1211) and Encoders (210, 1108A, 1108B) adapted to perform this function. Embodiment 19. The encoder (210, 1108A, 1108B) according to Embodiment 18, wherein interpolating the full-band input frame includes using linear interpolation in interpolating the full-band input frame. Embodiment 20. Using linear interpolation to interpolate the full bandwidth input for i=0,1,...,N2, This includes using linear interpolation to stretch or compress a frame of length N1 to a frame of length N2, according to TIFF0007909691000025.tif20170, where, TIFF0007909691000026.tif26170, x int (i) is the interpolated sampled value at point i, x(0) is the sampling point of the source vector x at the first point 0, x(1) is the sampling point of the source vector x at the second point 1, i offset Encoders (210, 1108A, 1108B) according to Embodiment 19, wherein is the offset and ∈ is the displacement. Embodiment 21. ∈ is Encoders (210, 1108A, 1108B) according to Embodiment 20, as defined in accordance with TIFF0007909691000027.tif11170. Embodiment 22. An encoder (210, 1108A, 1108B) according to any one of Embodiments 18 to 21, wherein receiving a full-band input frame includes receiving a full-band downmix signal generated by discrete Fourier transform (DFT) synthesis. Embodiment 23. Performing spectral inversion of the intermediate frame is Encoders (210, 1108A, 1108B) according to any one of embodiments 18 to 22, including performing spectral inversion according to TIFF0007909691000028.tif13170. Embodiment 24. Performing spectral inversion of the intermediate frame is Encoders (210, 1108A, 1108B) according to any one of embodiments 18 to 22, including performing spectral inversion according to TIFF0007909691000029.tif13170. Embodiment 25. Encoders (210, 1108A, 1108B) Determine the intermediate length of the intermediate frame (610, 2201) Encoders (210, 1108A, 1108B) according to any one of embodiments 18 to 24, further adapted to perform the same function. Embodiment 26. Determining the intermediate length involves adjusting the input sampling frequency f until the upper limit of the target bandwidth matches the Nyquist frequency in the intermediate frame. input The encoder (210, 1108A, 1108B) according to Embodiment 25, which includes rescaling the length N of the input frame. Embodiment 27. Determining the intermediate length is This includes determining the intermediate length according to TIFF0007909691000030.tif11170, Here, the target bandwidth is limited (f lo ,f hi ) when f inter =2f hi And N inter f is the intermediate length, N is the length of the input frame, and f input An encoder (210, 1108A, 1108B) according to embodiment 25 or 26, wherein the intermediate sampling frequency is . Embodiment 28. The encoder (210, 1108A, 1108B) according to any one of Embodiments 18 to 27, further adapted to perform low-pass filtering cutoff frequency selection so that substantially the cutoff frequency is near the center of the spectrum (1301). Embodiment 29. The encoder (210, 1108A, 1108B) according to Embodiment 28, wherein performing low-pass filtering and decimation of the inverted spectrum to produce a BWE target signal includes performing low-pass filtering and decimation to match the BWE target signal in the lower half of the spectrum (1303). Embodiment 30. The decimeter frame length is, Determined according to TIFF0007909691000031.tif10170, Here, N dec The length of the decimator frame is N inter is the intermediate length, and f dec This is the operating frequency of the decimator, and f inter The encoders (210, 1108A, 1108B) according to Embodiment 29, wherein the frequency of the intermediate frame is [frequency]. Embodiment 31. The encoder (210, 1108A, 1108B) according to any one of Embodiments 18 to 30, further adapted to input a BWE target signal to a BWE encoder (314) via a BWE target buffer (316) (1213), wherein the BWE encoder (314) and the BWE target buffer (316) are part of the encoder (210, 1108A, 1108B) operating on at least two frequency bands. Embodiment 32. The encoders (210, 1108A, 1108B) according to Embodiment 31, wherein the BWE encoder (314) and BWE target buffer (316) are part of the downmix encoder (214) of the encoders (210, 1108A, 1108B). Embodiment 33. The encoder (210, 1108A, 1108B) according to Embodiment 30, wherein the downmix encoder (214) of the encoder (210, 1108A, 1108B) has at least two encoding modes, and at least one of the at least two encoding modes has a BWE encoder (314). Embodiment 34. The encoders (210, 1108A, 1108B) according to Embodiment 33, wherein the encoders (210, 1108A, 1108B) operate on a downmix signal in a parametric stereo encoder. Embodiment 35. Encoders (210, 1108A, 1108B), Processing circuit (901), Memory (903) coupled to the processing circuit and The memory contains instructions that, when executed by the processing circuit, cause the encoders (210, 1108A, 1108B) to perform an operation, and the above operation is Receiving full-band input frames (1203), Interpolating the full-band input frame toward the intermediate frame (1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (1207), Interpolating frequency-inverted intermediate frames to match the sampling frequency of the low-pass filter and decimation process (1209), To create a bandwidth-extended (BWE) target signal, perform low-pass filtering and decimation on the interpolated spectrally inverted midframes (1211) Encoders (210, 1108A, 1108B), including those mentioned above. Embodiment 36. The encoder (210, 1108A, 1108B) according to Embodiment 35, wherein the memory contains further instructions that, when executed by the processing circuit, cause the encoder (210, 1108A, 1108B) to perform the operations described in any one of Embodiments 2 to 17. Embodiment 37. A computer program comprising program code to be executed by a processing circuit (903) of an encoder (210, 1108A, 1108B), wherein the execution of the program code causes the encoder (210, 1108A, 1108B) to perform an operation, and the above operation is Receiving full-band input frames (1203), Interpolating the full-band input frame toward the intermediate frame (1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (1207), Interpolating spectrally inverted intermediate frames to match the sampling frequency of the low-pass filter and decimation process (1209), To create a bandwidth-extended (BWE) target signal, perform low-pass filtering and decimation on the interpolated spectrally inverted midframes (1211) A computer program that includes [this]. Embodiment 38. The computer program according to Embodiment 37, wherein when the computer program is executed by the processing circuit (903) of the encoder (210, 1108A, 1108B), it includes further program code that causes the encoder (210, 1108A, 1108B) to perform further operations described in any one of Embodiments 2 to 17. Embodiment 39. A computer program product comprising a non-temporary computer-readable storage medium having program code to be executed by a processing circuit (903) of an encoder (210, 1008A, 1008B), wherein the encoder (210, 1108A, 1108B) performs an operation upon execution of the program code, and the above operation is Receiving full-band input frames (1203), Interpolating the full-band input frame toward the intermediate frame (1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (1207), Interpolating spectrally inverted intermediate frames to match the sampling frequency of the low-pass filter and decimation process (1209), To create a bandwidth-extended (BWE) target signal, perform low-pass filtering and decimation on the interpolated spectrally inverted midframes (1211) Computer program products, including [this]. Embodiment 40. The computer program product according to Embodiment 39, which includes further program code that causes the encoders (210, 1008A, 1008B) to perform further operations described in any one of Embodiments 2 to 17 when the non-temporary computer-readable storage medium is executed by the processing circuit (903) of the encoders (210, 1008A, 1008B).

Claims

1. A method implemented in encoders (210, 1108A, 1108B), wherein the method is Receiving full-band input frames (1203), Interpolating the aforementioned full-band input frames into intermediate frames (620, 1205), To create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted (630, 1207), Interpolating the spectrally inverted intermediate frames (640, 1209) to match the sampling frequencies of the low-pass filter and decimation process, To produce a bandwidth-extended (BWE) target signal, low-pass filtering and decimation are performed on the interpolated spectrally inverted intermediate frames (650, 1211) Methods that include...

2. The intermediate length N of the aforementioned intermediate frame ｉｎｔｅｒ To decide (610, 1201) The method according to claim 1, further comprising:

3. Determining the intermediate length involves determining the input sampling frequency f until the upper limit of the target bandwidth matches the Nyquist frequency in the intermediate frame. ｉｎｐｕｔ The method according to claim 2, comprising rescaling the length N of the input frame.

4. Determining the aforementioned intermediate length This includes determining the intermediate length in accordance with the following: Here, the target bandwidth is limited (f ｌｏ , f ｈｉ ) when f ｉｎｔｅｒ =2f ｈｉ And N ｉｎｔｅｒ The method according to claim 2 or 3, wherein n is the intermediate length, N is the length of the input frame, and f inter is the intermediate sampling frequency.

5. Interpolating the full-band input frame includes using linear interpolation in interpolating the full-band input frame, Using linear interpolation in interpolating the full-band input frames is possible for i = 0, 1, ..., N ２ Regarding in accordance with a length N １ of a frame to a length N ２ of the frame for stretching or compressing using the linear interpolation, where and x ｉｎｔ (i) is the interpolated sampling value at point i, x(0) is the sampling point in the source vector x at the first point 0, x(1) is the sampling point in the source vector x at the second point 1, i ｆｒａｃ This is a fractional point where the source vector x is estimated, i ｏｆｆｓｅｔ The method according to any one of claims 1 to 3, wherein is the offset and ε is the displacement.

6. The method according to any one of claims 1 to 3, wherein receiving the full-band input frame includes receiving a full-band downmix signal generated by discrete Fourier transform (DFT) synthesis.

7. Performing the inversion of the spectrum of the intermediate frame is ,or This includes performing the inversion of the spectrum in accordance with the above, Here, N ｉｎｔｅｒ The method according to any one of claims 1 to 3, wherein x is the length of the intermediate frame, m is the frame index, and x inter(m, n) is the intermediate frame.

8. The method according to any one of claims 1 to 3, further comprising selecting the cutoff frequency of the low-pass filter processing to be in the middle of the spectrum (1301).

9. The method according to claim 8, wherein performing the low-pass filtering and decimation of the inverted spectrum to produce the BWE target signal includes performing the low-pass filtering and decimation to align the BWE target signal to the lower half of the spectrum (1303).

10. The method according to any one of claims 1 to 3, further comprising inputting the BWE target signal to a BWE encoder (314) via a BWE target buffer (316) (1213), wherein the BWE encoder (314) and the BWE target buffer (316) are part of an encoder (210, 1108A, 1108B) operating on at least two frequency bands.

11. Encoders (210, 1108A, 1108B), Processing circuit (901), The memory (903) coupled to the processing circuit and The memory includes instructions that, when executed by the processing circuit, cause the encoders (210, 1108A, 1108B) to perform an operation, and the operation is Receiving full-band input frames and Interpolating the aforementioned full-band input frame into an intermediate frame, In order to create a spectrally inverted intermediate frame, the spectrum of the intermediate frame is inverted, Interpolating the spectrally inverted intermediate frames to match the sampling frequency of the low-pass filter and decimation process, To generate a bandwidth-extended (BWE) target signal, low-pass filtering and decimation are performed on the interpolated spectrally inverted intermediate frames. Encoders (210, 1108A, 1108B), including those mentioned above.

12. The memory contains, when executed by the processing circuit, the encoders (210, 1108A, 1108B), The intermediate length N of the aforementioned intermediate frame ｉｎｔｅｒ To decide The encoder according to claim 11, further comprising an instruction to perform an operation which includes an operation.

13. Determining the intermediate length involves determining the input sampling frequency f until the upper limit of the target bandwidth matches the Nyquist frequency in the intermediate frame. ｉｎｐｕｔ The encoder according to claim 12, further comprising rescaling the length N of the input frame.

14. Determining the aforementioned intermediate length This includes determining the intermediate length in accordance with the following: Here, the target bandwidth is limited (f ｌｏ , f ｈｉ ) when f ｉｎｔｅｒ =2f ｈｉ And N ｉｎｔｅｒ The encoder according to claim 12 or 13, wherein n is the intermediate length, N is the length of the input frame, and f inter is the intermediate sampling frequency.

15. Interpolating the full-band input frame includes using linear interpolation in interpolating the full-band input frame, Using linear interpolation in interpolating the full-band input frames is possible for i = 0, 1, ..., N ２ Regarding Length N １ The frame has a length N ２ This includes using the linear interpolation to stretch or compress the frame, where, and x ｉｎｔ (i) is the interpolated sampling value at point i, x(0) is the sampling point in the source vector x at the first point 0, x(1) is the sampling point in the source vector x at the second point 1, i ｆｒａｃ This is a fractional point where the source vector x is estimated, i ｏｆｆｓｅｔ The encoder according to any one of claims 11 to 13, wherein ε is the offset and ε is the displacement.

16. The encoder according to any one of claims 11 to 13, wherein receiving the full-band input frame includes receiving a full-band downmix signal generated by discrete Fourier transform (DFT) synthesis.

17. Performing the inversion of the spectrum of the intermediate frame is ,or This includes performing the inversion of the spectrum in accordance with the above, Here, N ｉｎｔｅｒ The encoder according to any one of claims 11 to 13, wherein x is the length of the intermediate frame, m is the frame index, and x inter(m, n) is the intermediate frame.

18. The memory contains, when executed by the processing circuit, the encoders (210, 1108A, 1108B), The cutoff frequency of the low-pass filter is selected to be in the center of the spectrum. The encoder according to any one of claims 11 to 13, further comprising an instruction to perform an operation including an operation.

19. The encoder according to claim 18, wherein performing the low-pass filtering and decimation of the inverted spectrum to produce the BWE target signal includes performing the low-pass filtering and decimation to align the BWE target signal to the lower half of the spectrum.

20. The memory contains, when executed by the processing circuit, the encoders (210, 1108A, 1108B), The BWE target signal is input to the BWE encoder (314) via the BWE target buffer (316). The encoder according to any one of claims 11 to 13, wherein the instruction to perform an operation further includes the BWE encoder (314) and the BWE target buffer (316) are part of an encoder (210, 1108A, 1108B) that operates on at least two frequency bands.

21. The encoder according to claim 20, wherein the BWE encoder (314) and the BWE target buffer (316) are part of the downmix encoder (214) of the encoder (210, 1108A, 1108B).

22. The encoder according to claim 21, wherein the downmix encoder (214) of the encoder (210, 1108A, 1108B) has at least two encoding modes, and at least one of the at least two encoding modes has the BWE encoder (314).

Images