Adaptive predictive coding

Abstract

Embodiments of the present disclosure relate to adaptive predictive coding. A method performed in an encoder (300) for selecting a codec mode for a current frame includes obtaining (725) a bitrate for absolute codec and predictive codec. The method comprises calculating (730) a bit rate difference based on the acquired bit rate. The method includes low-pass filtering (735) of the bit rate difference. The method comprises selecting (740) a codec mode based on the bit rate difference, the low-pass filtered bit rate difference, and a prediction mode counter.

Description

[0001] This application is a divisional application of the invention patent application with an international application date of December 15, 2021, which entered the Chinese national stage on June 15, 2024, with Chinese national application number 202180105041.5 and invention name “Adaptive Prediction Coding”. Technical Field

[0002] The present disclosure relates generally to communications, and more particularly to communication methods and related devices and nodes that support wireless communications. Background Art

[0003] In communication networks, obtaining good performance and capacity can be challenging given the communication protocol, its parameters, and the physical environment in which the communication network is deployed.

[0004] For example, despite the increasing capacity of telecommunication networks, it is still important to limit the resource usage required for each user. In mobile telecommunication networks, less resource usage required for each call means that the mobile telecommunication network can serve a greater number of users in parallel. Reducing resource usage can also reduce the power consumption of user-side devices (such as terminal devices) and network-side devices (such as network nodes). This means energy and cost savings for network operators, while enabling terminal devices to have extended battery life and increased talk time.

[0005] One mechanism for reducing the required resource usage for voice communication applications in mobile telecommunication networks is to exploit natural pauses in speech. In more detail, in most conversations only one party is active at a time, so speech pauses in one communication direction typically take up more than half of the signal. One way to exploit this property to reduce the required resource usage is to employ a discontinuous transmission (DTX) system, where the active signal encoding is discontinuous during speech pauses.

[0006] During pauses in speech, very low bitrate encoded background noise is often transmitted so that a comfort noise generator (CNG) system on the receiving end can fill the pause with background noise that has similar characteristics to the original noise. Because the background noise is preserved and does not turn on and off with the speech, CNG makes it sound more natural than silence during speech pauses. Complete silence during speech pauses is often considered irritating and often leads to the misunderstanding that the call has been disconnected.

[0007] DTX systems may further rely on a voice activity detector (VAD) which indicates to the transmitting device whether to use active signal coding or low-rate background noise coding. In this regard, the transmitting device may be configured to distinguish other source types by using a (generic) voice activity detector (GSAD or SAD), which can not only distinguish speech from background noise, but may also be configured to detect music or other signal types that are considered relevant.

[0008] By supporting stereo or multi-channel audio transmission, communication services can be further enhanced. In these cases, the DTX / CNG system may also take into account the spatial characteristics of the signal to provide comfort noise that sounds pleasing.

[0009] A common mechanism for generating comfort noise is to transmit information about the energy and spectral shape of the background noise in speech pauses. This can be done using a much smaller number of bits than conventional speech segment codecs.

[0010] On the receiving device side, comfort noise is generated by creating a pseudo-random signal and then shaping the spectrum of the signal using a filter based on information received from the transmitting device. Signal generation and spectrum shaping can be performed in the time domain or the frequency domain.

[0011] For stereo operation, additional parameters are transmitted to the receiving side. In a typical stereo signal, channel pairs show a high degree of similarity or correlation. The most advanced stereo codec scheme currently exploits this correlation by adopting parametric codecs, in which a single channel is encoded with high quality and supplemented with a parameter description that allows reconstruction of a complete stereo image. The process of reducing a channel pair to a single channel is generally referred to as downmixing, and the resulting channel is referred to as a downmix channel. The downmixing process generally attempts to maintain energy by aligning the inter-channel time difference (ITD) and inter-channel phase difference (IPD) before mixing the channels. In order to maintain the energy balance of the input signal, the inter-channel level difference (ILD) is further measured. Then, ITD, IPD and ILD are encoded and can be used in the reverse upmixing process when reconstructing the stereo channel pair at the decoder.

[0012] Most audio codec systems process the input audio signal in segments (usually called frames). For stable audio scenes, the stereo parameters between adjacent frames will show a high degree of similarity. In order to exploit this similarity, predictive coding schemes are usually applied. In this scheme, the prediction of the current frame parameters is based on the parameters decoded in the past, and the difference from the true parameters is what is encoded and transmitted between the transmitter and the receiver. A simple but effective prediction is to use the last decoded parameters as the prediction, in which case the predictive coding scheme can also be called a differential coding scheme. Summary of the invention

[0013] In general, one problem with predictive coding schemes is that they are sensitive to errors. If one or more elements of the prediction sequence are missing, the decoder will suffer from prediction errors that may persist long after the error occurred. This problem is called error propagation and is a known problem with all predictive coding schemes. One remedy is to force non-predictive coding at fixed time intervals, which will stop the error propagation. However, this will have the disadvantage that the non-predictive coding may result in a less efficient coding of the parameter, thereby increasing the overall bit rate of the encoder and / or allowing fewer bits for the coding of other parameters.

[0014] Certain aspects of the present disclosure and its embodiments may provide solutions to these or other challenges. A method for selecting a codec mode for a current frame performed in an encoder includes obtaining bit rates for absolute codecs and predictive codecs, rather than forcing non-predictive codecs to be performed at fixed time intervals for specific parameters. The method includes calculating a bit rate difference based on the obtained bit rate. The method includes low-pass filtering the bit rate difference. The method includes selecting a codec mode based on the bit rate difference, the low-pass filtered bit rate difference, and a predictive mode counter.

[0015] Various embodiments take into account features of previous encoding of another parameter so that forced non-predictive codec for a first parameter is invoked only when encoding of the second parameter may require fewer bits, thereby increasing the likelihood that the total bit consumption is below a fixed level.

[0016] An encoder is adapted to perform operations including obtaining bit rates for an absolute codec and a predictive codec. The operations include calculating a bit rate difference based on the obtained bit rates. The operations include low pass filtering the bit rate difference. The operations include selecting a codec mode based on the bit rate difference, the low pass filtered bit rate difference, and a predictive mode counter.

[0017] An encoder (300) includes: a processing circuit (401); and a memory (403) coupled to the processing circuit, wherein the memory includes instructions that, when executed by the processing circuit, cause the encoder (300) to perform operations, the operations including obtaining bit rates for absolute coding and predictive coding. The operations include calculating a bit rate difference based on the obtained bit rates. The operations include low-pass filtering the bit rate difference. The operations include selecting a coding mode based on the bit rate difference, the low-pass filtered bit rate difference, and a predictive mode counter.

[0018] A computer program is provided, comprising program code to be executed by a processing circuit (401) of an encoder (300), whereby execution of the program code causes the encoder (300) to perform operations, the operations comprising obtaining bit rates for an absolute codec and a predictive codec. The operations comprise calculating a bit rate difference based on the obtained bit rates. The operations comprise low pass filtering the bit rate difference. The operations comprise selecting a codec mode based on the bit rate difference, the low pass filtered bit rate difference, and a predictive mode counter.

[0019] A computer program product is provided, comprising a non-transitory storage medium, the non-transitory storage medium comprising program code to be executed by processing circuitry (401) of an encoder, whereby execution of the program code causes the encoder to perform operations, the operations comprising obtaining bit rates for an absolute codec and a predictive codec. The operations comprise calculating a bit rate difference based on the obtained bit rates. The operations comprise low pass filtering the bit rate difference. The operations comprise selecting a codec mode based on the bit rate difference, the low pass filtered bit rate difference, and a predictive mode counter.

[0020] Certain embodiments may provide one or more of the following technical advantages: Various embodiments may utilize predictive coding of one parameter to reduce error propagation problems while minimizing the negative impact on coding of other parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The attached drawings illustrate certain non-limiting embodiments of the inventive concept, and are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application. In the drawings:

[0022] Figure 1 is a graphical representation of error propagation;

[0023] Figure 2 is a block diagram illustrating components of a stereo encoder and decoder according to some embodiments of the present disclosure;

[0024] Figure 3 is a block diagram illustrating an example environment of an encoder and decoder system in which selection of a codec mode may be performed according to some embodiments of the present disclosure;

[0025] Figure 4 is a block diagram illustrating components of a stereo encoder according to some embodiments of the present disclosure;

[0026] Figure 5 is a block diagram illustrating components of a stereo decoder according to some embodiments of the present disclosure;

[0027] Figure 6 is a block diagram of a virtualization environment according to some embodiments of the present disclosure;

[0028] Figure 7 is a flowchart illustrating the operation of an encoder according to some embodiments of the present disclosure;

[0029] Figure 8 is a diagram showing the implementation of some embodiments of the present disclosure Figure 8 The operation shown in Figure 4 A block diagram of additional components of an encoder;

[0030] Figure 9-10 is a flowchart illustrating the operation of an encoder according to some embodiments of the present disclosure; and

[0031] Fig.11 is a graphical representation of decision thresholds according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0032] Some embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art, wherein examples of embodiments of the inventive concept are shown. However, the inventive concept can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided so that the disclosure will be comprehensive and complete, and the scope of the inventive concept will be fully conveyed to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be assumed to exist / use in another embodiment by default.

[0033] As mentioned previously, one problem with predictive coding schemes is error propagation. Figure 1 A schematic diagram of error propagation is illustrated. An absolute codec frame is lost before a sequence of consecutive predictive codec frames, which may be referred to as a predictive codec stripe. A bad frame indicator signals when a frame is lost (or corrupted). Since the memory is corrupted by the frame loss, the error will persist during the entire predictive codec stripe (indicated by "P") and only terminate when a new absolute codec frame (indicated by "A") is received. The effect on the composite signal may be an erroneously perceived location of the source that persists throughout the predictive codec stripe.

[0034] Instead of forcing non-predictive codec at fixed time intervals for specific parameters, the various embodiments described herein consider the estimated bit consumption of forcing non-predictive codec at fixed time intervals and try to seize the opportunity of lower bit rate consumption by: obtaining the bit rates for non-predictive (i.e., absolute) codec and predictive codec; calculating the bit rate difference based on the obtained bit rates; low-pass filtering the bit rate difference; and selecting the codec mode based on the bit rate difference, the low-pass filtered bit rate difference, and the prediction mode counter.

[0035] According to some embodiments, the characterization of the previous encoding of another parameter enables forced non-predictive codec for the first parameter to be invoked only when encoding of the second parameter may require fewer bits, thereby increasing the likelihood that the total bit consumption is below a fixed level.

[0036] According to some of the various embodiments, encoding and decoding comprises encoding and decoding the first parameter, wherein the first parameter is encoded based on inter-frame and intra-frame predictions, and these predictions are combined using a weighting factor α.

[0037] Various embodiments utilize predictive coding of one parameter to reduce error propagation problems while minimizing the negative impact on the coding of other parameters.

[0038] In the following description, various embodiments are described in Figure 2 1 and 2. The stereo encoder and decoder system shown in FIG. 2 operates in the stereo encoder of the stereo encoder and decoder system. The stereo encoder 210 processes the input left and right channels in segments called frames. The stereo analysis and downmixing block 212 performs parameter analysis and generates a downmix. For a given frame m, the input channels can be written as: Where n = 0, 1, 2, ..., N-1 represents the sample number in frame m, and N is the length of the analysis frame. In various embodiments, the analysis frame is extracted by overlapping in the encoder so that the decoder can use an overlap-add strategy to reconstruct the stereo signal. The input channel is windowed using a suitable windowing function w(n) and transformed to the discrete Fourier transform (DFT) domain, which can be written as:

[0039] Note that other frequency domain representations may be used here, such as a quadrature mirror filter (QMF) filter bank, a hybrid QMF filter bank, or an odd DFT (ODFT) representation, which consists of modified discrete cosine transform (MDCT) and modified discrete sine transform (MDST) transform components.

[0040] For parametric analysis, the spectrum is divided into frequency bands b, where each band corresponds to a series of frequency coefficients k=k start(b) …k end(b) ,b=0,1,2,…N bands -1 Where N bands Indicates the total number of bands. The band limits are usually set to reflect the resolution of human auditory perception, which means narrower bands for low frequencies and wider bands for high frequencies. Note that different band resolutions can be used for different parameters.

[0041] The signal is then analyzed in the parameter analysis module to extract the ITD, IPD, and ILD parameters. It is worth noting that ILD has a significant impact on the perceived location of the sound. Therefore, it is critical to reconstruct the ILD parameters with high accuracy to maintain stability and correct location.

[0042] In addition, channel coherence can be analyzed and ICC parameters can be derived. The stereo parameter set for frame m can be represented as P(m), which contains the complete set of ITD, IPD, ILD and ICC parameters used in parameter representation. Parameters are encoded by parameter encoder 218 and added to the bit stream to be stored or transmitted to a decoder.

[0043] Before generating the downmix channels, it is beneficial to compensate for ITD and IPD to reduce cancellation and maximize the energy of the downmix. ITD compensation can be implemented in the time domain or in the frequency domain before frequency transformation, but it is essentially performing a time shift on one or both channels to eliminate ITD. Phase alignment can be achieved in different ways, but the goal is to align the phases to minimize cancellation. This ensures maximum energy in the downmix. ITD and IPD adjustments can be made in frequency bands or over the full spectrum and should preferably be done using quantized ITD and IPD parameters to ensure that the modifications can be reversed in the decoder stage.

[0044] The embodiments described below are independent of the implementation of IPD and ITD parameter analysis and compensation. In other words, the embodiments do not depend on how to analyze or compensate IPD and ITP. In such embodiments, the ITD and IPD adjustment channels can be represented by a prime ('):

[0045] The ITD and IPD adjusted input channels may then be downmixed by the parameter analysis and downmixing block 212 to produce a mid / side representation, also referred to as a downmix / side representation. One method of performing downmixing is to use the sum and difference of the signals:

[0046] Downmix signalX M (m, k) is encoded by the downmix encoder 214 to be stored or transmitted to the decoder. The encoding can be done in the frequency domain, but can also be done in the time domain. In this case, a DFT synthesis stage is required to generate a time domain version of the downmix signal, which is then fed to the downmix encoder in turn. However, the transformation to the time domain may introduce a delay misalignment with the stereo parameters, which requires special processing. This can be solved by introducing additional delays or by interpolating the parameters to ensure that the decoder synthesis of the downmix and stereo parameters is aligned.

[0047] Side signal X S The reconstruction of (m,k) can be generated from the downmix and the acquired stereo parameters by local parameter synthesis. Side signal prediction It can be derived based on the downmix signal: Where p(·) is the prediction function. The prediction residual can be formed according to the following formula: It is then encoded by the residual encoder 216. The encoded representation of the downmix, stereo parameters and potentially the residual signal are transmitted or stored for reconstruction by the decoder 220. The downmix decoder 224 reconstructs the decoded downmix signal, the reconstructed parameters are obtained by the parameter decoder 228, and the potentially reconstructed residual signal is generated by the residual decoder 226. The reconstructed downmix signal, the reconstructed parameters and the residual signal are input to the stereo synthesizer 222 to produce a reconstructed stereo signal.

[0048] Figure 3 An example operating environment of an encoder 300 and a decoder 302 that can be used to encode and decode a multi-channel bitstream as described herein is shown. The encoder 300 receives audio from a network 304 and / or from a memory 306 and / or from a recorder 308, and encodes the audio into a bitstream as described below, and sends the encoded audio to the decoder 302 via a network 310. The storage device 306 can be part of a repository of multi-channel audio signals, such as a repository for storing or streaming audio services, a separate storage component, a component of a mobile device, etc. The decoder 302 can be part of a device 312 with a media player 314. The device 312 can be a mobile device, a set-top device, a desktop computer, etc. In other embodiments, the decoder 300 can be embodied in hardware and / or software of a stand-alone server, a cloud-implemented server, a distributed server, or as a processing resource in a server cluster. For example, the decoder can be part of a cloud-implemented teleconferencing application. The decoder 300 receives an encoded bitstream sent via a transport layer of a network. As described herein, during operation, the decoder 300 receives and processes frames of a bitstream.

[0049] Figure 4is a block diagram illustrating elements of an encoder 300 configured to encode audio frames according to some embodiments of the inventive concept. As shown, the encoder 300 may include a network interface circuit 405 (also referred to as a network interface) configured to provide communication with other devices / entities / functions / etc. The encoder 300 may also include a processor circuit 401 (also referred to as a processor) coupled to the network interface circuit 405 and a memory circuit 403 (also referred to as a memory) coupled to the processor circuit. The memory circuit 403 may include a computer-readable program code that, when executed by the processor circuit 401, causes the processor circuit to perform operations according to the embodiments disclosed herein.

[0050] According to other embodiments, the processor circuit 401 can be defined to include a memory so that a separate memory circuit is not required. As discussed herein, the operations of the encoder 300 can be performed by the processor 401 and / or the network interface 405. For example, the processor 401 can control the network interface 405 to send communications to the decoder 302 and / or receive communications from one or more other network nodes / entities / servers (such as other encoder nodes, storage servers, etc.) through the network interface 405. In addition, modules can be stored in the memory 403, and these modules can provide instructions so that when the instructions of the modules are executed by the processor 401, the processor 401 performs the corresponding operations.

[0051] Figure 5 is a block diagram illustrating elements of a decoder 302 configured to encode audio frames according to some embodiments of the inventive concept. As shown, the decoder 302 may include a network interface circuit 505 (also referred to as a network interface) configured to provide communication with other devices / entities / functions, etc. The decoder 302 may also include a processor circuit 501 (also referred to as a processor) coupled to the network interface circuit 505 and a memory circuit 503 (also referred to as a memory) coupled to the processor circuit. The memory circuit 503 may include a computer-readable program code that, when executed by the processor circuit 501, causes the processor circuit to perform operations according to the embodiments disclosed herein.

[0052] According to other embodiments, the processor circuit 501 can be defined to include a memory so that a separate memory circuit is not required. As discussed herein, the operation of the decoder 302 can be performed by the processor 501 and / or the network interface 505. For example, the processor 501 can control the network interface 505 to send communications to the decoder 302 and / or receive communications from one or more other network nodes / entities / servers (such as other encoder nodes, storage servers, etc.) through the network interface 505. In addition, modules can be stored in the memory 503, and these modules can provide instructions so that when the instructions of the modules are executed by the processor 501, the processor 501 performs the corresponding operations.

[0053] Figure 6 It is a block diagram showing a virtualized environment 600, in which the functions implemented by some embodiments can be virtualized. In this context, virtualization means creating a virtual version of a device or equipment, which can include a virtualized hardware platform, storage device and network resources. As used herein, virtualization can be applied to any device described herein or its components, and relates to an implementation in which at least a portion of a function is implemented as one or more virtual components. Some or all of the functions described herein can be implemented as virtual components performed by one or more virtual machines (VMs), which are implemented in one or more virtual environments 600 hosted by one or more hardware nodes (such as hardware computing devices running as network nodes, user equipment (UE), core network nodes or mainframes). In addition, in embodiments where a virtual node does not require a radio connection (e.g., a core network node or mainframe), the node can be fully virtualized.

[0054] Application 602 (which may alternatively be referred to as a software instance, a virtual device, a network function, a virtual node, a virtual network function, etc.) runs in virtualization environment 600 to implement some of the features, functions and / or advantages of some embodiments disclosed herein.

[0055] Hardware 604 includes processing circuitry, memory storing software and / or instructions executable by the hardware processing circuitry, and / or other hardware devices described herein, such as network interfaces, input / output interfaces, and the like. The software may be executed by the processing circuitry to instantiate one or more virtualization layers 606 (also referred to as hypervisors or virtual machine monitors (VMMs)), provide VMs 608A and VMs 608B (one or more of which may be generally referred to as VMs 608), and / or perform any of the functions, features, and / or benefits described in connection with some embodiments described herein. Virtualization layer 606 may present a virtual operating platform that appears to be network hardware to VMs 608.

[0056] VM 608 includes virtual processing, virtual memory, virtual network or interface, and virtual storage, and can be run by corresponding virtualization layer 606. Different embodiments of instances of virtual device 602 can be implemented on one or more VMs in VM 608, and can be implemented in different ways. Virtualization of hardware is sometimes referred to as network function virtualization (NFV). NFV can be used to integrate many network device types onto industry standard high-volume server hardware, physical switches, and physical storage, which can be located in data centers and client devices.

[0057] In the context of NFV, VM 608 may be a software implementation of a physical machine running a program, in the same manner as if the program were executed on a physical, non-virtualized machine. Each of VM 608 and the portion of hardware 604 on which it executes (whether hardware dedicated to the VM and / or hardware shared by the VM with other VMs) form a separate virtual network element. Still in the context of NFV, a virtual network function is responsible for handling a specific network function running in one or more VMs 608 on hardware 604 and corresponding to an application 602.

[0058] Hardware 604 may be implemented in a standalone network node with general or specific components. Hardware 604 may implement some functions via virtualization. Alternatively, hardware 604 may be part of a larger hardware cluster (e.g., in a data center or customer premises equipment (CPE)), where many hardware nodes work together and are managed via management and orchestration 610, which, among other things, oversees the lifecycle management 602 of the application. In some embodiments, hardware 604 is coupled to one or more radio units, each of which includes one or more transmitters and one or more receivers that may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces, and may be used in conjunction with virtual components to provide radio capabilities to virtual nodes (such as radio access nodes or base stations). In some embodiments, a control system 612 may be used to provide some signaling, which may alternatively be used for communication between hardware nodes and radio units.

[0059] Returning to the parameter encoder 218, the side signal predictor may be implemented as a single predictor g that minimizes the mean square error (MSE) between the side signal and the predicted side signal. Furthermore, the prediction will preferably be applied to frequency bands and involve prediction parameters for each frequency band b.

[0060] If the coefficients with b are written as a column vector X S,b (m) and XM,b (m), the minimum MSE predictor can be derived as

[0061] However, this expression can be simplified to produce more stable prediction parameters. b (m) does not directly represent the level difference, but it can control the portion of the downmix signal routed to the left and right channels. Therefore, for the ILD parameter, it has a significant impact on the perceived sound location. For more background on prediction parameters, see PCT Application No. PCT / EP2019 / 058650 (Publication No. WO2019 / 193156). Prediction parameter g b (m) are encoded in turn using an inter-frame prediction coding scheme, in which the difference between frames m is taken into account. For each band b, the reconstruction parameters compared to the previous frame are calculated according to the following formula difference

[0062] The encoder 300 may select the encoding g b (m) or Δg b (m), depending on which of them produces the lowest bit consumption. In an embodiment, g b (m) and Δg b (m) can be quantized using a scalar quantizer followed by an entropy encoder on the quantizer index. Arithmetic coding, Huffman coding, and Golomb-Rice coding are examples of what can be used as entropy encoders. In a typical scenario, the entropy encoder will assign smaller codewords to small changes, i.e., Δg b This means that for stable audio scenes, Δg b (m) predictive coding may require a small number of bits nbits pred (m). For fast scene changes, this results in a larger Δg b (m), for g b The bit consumption of (m) encoding is nbits abs (m) may be lower. The encoding scheme has two modes, which are: 1) Absolute (ABSOLUTE): g b (m), and 2) Prediction (PREDICTIVE): Δg b (m) encoding.

[0063] Coding mode g mode(m)∈{ABSOLUTE,PREDICTIVE} needs to be encoded for each frame m so that the decoder 302 knows whether the encoded value is 1)ABSOLUTE:g b (m), or 2) PREDICTIVE:

[0064] Further variations of this encoding scheme are possible within the scope of the present disclosure. For example, if the prediction parameter g b If (m) shows a high correlation with another parameter (such as the residual codec energy or the corresponding representation), then it may be beneficial to jointly encode these parameters. The important part is that the coding scheme has a prediction mode and an absolute codec mode, and this decision is encoded and signaled to the decoder. Sequences of consecutive PREDICTIVE codec modes may be called "prediction codec stripes" or "prediction stripes" and are typically observed for scene-stable audio segments. If an audio frame at the beginning of a prediction stripe is lost, the parameters may suffer from error propagation during the entire stripe (see Figure 1 ).

[0065] The ABSOLUTE and PREDICTIVE codec modes may also each represent a subset of modes with a common property, i.e. they are either absolute codec modes or predictive codec modes, respectively. For example, the absolute codec modes may rely on intra prediction, entropy coding of parameters, or simply represent an enumeration of possible parameters with fixed-width indices. For the subset of predictive codec modes, they all rely on previously encoded and decoded parameters.

[0066] When packet loss is experienced between the encoder and decoder, in order to reduce g b The effect of error propagation in the decoding of (m) to form the reconstructed counterpart ABSOLUTE codec can be forced at regular intervals, effectively limiting the prediction stripes to a maximum time length. However, this may result in g for frames where ABSOLUTE codec is forced. b The coding efficiency of (m) is low. The coding scheme can be implemented according to the following formula: Where pred_allowed(m) is an optional flag indicating whether predictive coding and decoding is allowed. pred_allowed(m) is based at least on the number of frames since the last ABSOLUTE coded frame, so that PREDICTIVE mode is disabled when the maximum number of predicted frames is reached, e.g. where predcount(m) is the number of frames since the last ABSOLUTE encoded frame. allowed(m) The optional embodiment can be obtained in the following way: or Using "≤" or "=" will make the effective prediction stripe equal to MAX_PRED_STREAK_M1+1, or in other words: MAX_PRED_STREAK=MAX_PRED_STREAK_M1+1 pred_allowed(m) may further be modified to reflect situations where prediction modes should be allowed or not allowed, such as after a switch in codec mode or in other situations where prediction stripes should be broken.

[0067] The predcount(m) variable can be initialized to zero at startup and then updated according to the following formula:

[0068] Note that since predcount(m) depends on g of the current frame m mode (m), so pred_allowed(m) is typically based on predcount(m-1) of the previous frame m-1. This needs to be taken into account in the MAX_PRED_STREAK constant in order to achieve the number of frames required in the prediction stripe. A typical value of MAX_PRED_STREAK can be MAX_PRED_STREAK=10, which corresponds to 200ms in the case of a 20ms frame step. In the case of comfort noise, the frame rate can be reduced and frames may be transmitted, for example, every 160 milliseconds (corresponding to every 8 frames in the codec of the active signal). In this case, predcount(m) can be updated with a larger step size to have a similar recovery time during comfort noise. For example, predcount(m) can be updated with a larger step size according to the following formula:

[0069] A disadvantage of the above mode selection is that when absolute codec is forced, if MAX_PRED_STREAK is reached, the cost of selecting absolute codec instead of predictive codec may be significant.

[0070] Reference will now be made to some embodiments according to the present disclosure Figure 7 The flow chart of the encoder 300 (using Figure 4 and Figure 8 For example, the module can be stored in Figure 4 The modules are stored in the memory 403, and these modules can provide instructions so that when the instructions of the modules are executed by the corresponding encoder processing circuit 401, the encoder 300 performs the corresponding operations of the flowchart.

[0071] Which codec mode to choose (i.e., g mode (m)) is determined based on nbits abs (m) and nbits pred The estimated mean difference between (m). Figure 8 In the example, blocks 802 and 804 execute block 725 to obtain nbits for absolute codec. abs (M) and predictive codec nbits pred (m) bit rate. nbits pred The calculation of (m) is based on the parameter memory 806. abs (m), more than one absolute codec mode can be considered, and nbits abs (m) should be set to the minimum of these absolute codec modes. For example, entropy coding methods may in some cases result in a larger bit rate than directly encoding the index in binary format. In this case, direct index encoding is preferably selected as a fallback solution.

[0072] In block 730, the encoder 300 calculates a bit rate difference based on the obtained bit rate. Figure 8 In block 808, the bit rate difference bitdiff(m) is calculated. In some embodiments, the bit rate difference is calculated according to the following formula: bitdiff(m)=nbits abs (m)-nbits pred (m)

[0073] In block 735, encoder 300 low pass filters the bit rate difference. Figure 8 In some embodiments, the encoder 300 performs low pass filtering on the bit rate difference according to the following formula: bitdiff LP (m)=γbitdiff(m)+(1-γ)bitdiff LP (m-1)where γ is the low-pass filter coefficient, a suitable value for which may be γ=0.06 or within the range of γ∈[0.01,0.20].

[0074] Note that if the bit difference is constant (e.g., bitdiff(m) = constant D), then the average cost of periodically forcing absolute encoding and decoding at intervals of MAX_PRED_STREAK will be D / MAX_PRED_STREAK. If one wants to opportunistically encode and decode absolute frames after predcount(m) frames, this should incur a lower or equal cost, i.e. bitdiff(m) / predcount(m-1)≤D / MAX_PRED_STREAK Equivalently, this can be expressed as a condition for selecting a prediction mode only when the absolute encoding / decoding cost is strictly greater than the estimated cost at regular enforcement intervals, i.e. bitdiff(m) / predcount(m-1)>D / MAX_PRED_STREAK

[0075] In some embodiments, the constant D can be obtained by using the low-pass filtered bit rate difference bitdiff LP (m) to approximate, the equation becomes bitdiff(m)>bitdiff LP (m)·predcount(m-1) / MAX_PRED_STREAK

[0076] This corresponds to a decision threshold such as Fig. 9 If a more conservative approach is desired, a tuning constant C can be added. For example, the tuning constant C can be used together with the constant D according to the following formula: bitdiff(m)>C·bitdiffLP(m)·predcount(m-1) / MAX_PRED_STREAK where C∈[0,1]. Note that C=0 gives the value when the required bits are less than or equal to the predicted codec bits. abs (m)≤bit diff (m) only selects the original behavior of the absolute codec, or equivalently, in bits abs (m)>bit diff (m) selects predictive coding. Although C = 1 gives a constant bitdiff LP (m) = the theoretical optimal value of D, but it can be adjusted by setting C = 0.8 to adjust the actual bitdiff LP A slightly more conservative approach to (m) can be found. Note that, depending on how pred_allowed(m) is computed, the expression can be written as follows:

[0077] In block 740, the encoder 300 selects a codec mode based on the bit rate difference, the low pass filtered bit rate difference, and the prediction mode counter. In some embodiments, the codec mode selection is based on combining an expression with a tuning constant C with the codec mode selector 816 according to the following equation: mode (m) Choose to combine: If the pred_allowed(m) flag is not present, the expression can be written as follows:

[0078] Another embodiment attempts to consider possible options for absolute codecs before MAX_PRED_STREAK can be implemented according to: Where A is a constant, such as A = 0.4 or in the range of A∈(0,2]. Thus, if the cost (nbits abs (m)-nbits pred (m)) is found to be low enough that the prediction streaks are kept within the limit MAX_PRED_STREAK, while still allowing the codec mode to be switched to ABSOLUTE mode before that limit is reached. This can be seen as the urgency of selecting the absolute codec mode increasing as the number of frames since the last absolute codec mode increases. By setting the constant A appropriately, the average bitrate and average length of the prediction streaks can be reduced given a certain type of material and compared to forcing absolute codec after MAX_PRED_STREAK frames.

[0079] If the prediction mode is selected by the encoder 300, as shown in block 745, the encoder 300 sets the codec mode to the prediction mode in block 750 and increments the prediction mode counter (e.g., predcount(m)) in block 755. Figure 8 In block 750, the parameter encoder 818 sets the codec mode to the prediction mode. The prediction frame counter block 810 performs an increment of the prediction mode counter.

[0080] If the prediction mode is not selected by the encoder 300 as shown in block 745, the encoder 300 sets the codec mode to the absolute codec mode in block 715 and resets (or sets) the prediction mode counter to an initial value (e.g., predcount(m)) in block 720. The initial value may be 0, 1, or any other value. Figure 8 In block 750, the parameter encoder 818 sets the codec mode to absolute mode. The prediction frame counter block 810 performs a reset / set of the prediction mode counter.

[0081] In some other embodiments, predictive codec may not be allowed for a particular frame. Therefore, in these embodiments, encoder 300 determines whether predictive codec is allowed. Fig. 9 This is explained.

[0082] Steering Fig. 9 , Box 715-755 on top Figure 7 9 and will not be repeated here. In block 910, the encoder 300 determines whether predictive coding is allowed. One way to determine this is to check the pred_allowed(m) indicator. As described above, the pred_allowed(m) indicator is based on the prediction mode counter predcount(m) (e.g., predcount(m-1)). This determination is made at Figure 8 As described above, in some embodiments, predictive codec is allowed when predcount(m-1) is less than the maximum number of predicted frames allowed. In other words, predictive codec is not allowed when predcount(m-1) is greater than the maximum number of predicted frames allowed. In other embodiments, predictive codec is not allowed when predcount(m-1) is greater than or equal to the maximum number of predicted frames allowed. In other embodiments, predictive codec is not allowed when predcount(m-1) is greater than or equal to the maximum number of predicted frames allowed. In other embodiments, predictive codec is allowed when predcount(m-1) is less than or equal to the maximum number of predicted frames allowed. Since predcount(m) is incremented by one per frame, the condition can be evaluated by checking whether predcount(m-1) is equal to the maximum number of predicted frames allowed.

[0083] In some embodiments, the pred_allowed(m) indicator is a flag. Fig.10 An embodiment of setting a flag is described. Fig.10 In block 1000, the encoder 300 determines whether the prediction count is less than the maximum number of prediction frames. In response to determining that the prediction count is less than the maximum number of prediction frames, the method includes setting a flag in block 1010 to indicate that prediction codec is allowed. In response to determining that the prediction count is greater than the maximum number of prediction frames, the method includes setting a flag in block 1020 to indicate that prediction codec is not allowed. When the prediction count is equal to the maximum number of prediction frames, the flag may be set by default to indicate that prediction codec is allowed or to indicate that prediction codec is not allowed. When pred_allowed(m) is true, other settings may be used to set the flag and are described in the embodiments of this document.

[0084] Return to Fig. 9In response to predictive coding being not allowed, the encoder 300 sets the coding mode to absolute coding mode in block 715 and sets or resets the prediction mode counter to zero in block 720 as previously described.

[0085] In another embodiment, the selection between ABSOLUTE and PREDICTIVE decoding modes may be influenced by the encoding of the second parameter of the codec. The selection may be related to the potential error recovery of the second parameter, rather than just counting frames. Assuming that intra prediction C is used intra (m), inter-frame prediction C inter The second parameter C(m) is encoded by combining the prediction of (m) and the coded prediction residual r(m). Let the reconstruction parameter Reconstruction-based intra prediction Reconstructed Inter Prediction and the reconstructed prediction residuals For example, the reconstruction parameter In some embodiments, it can be determined according to the following formula:

[0086] The weighting parameter α(m) controls the proportion of intra prediction (which can be considered an absolute codec) compared to inter prediction (which depends on past frames). Since the parameters are generally stable, the weight α(m) balances fast error recovery (large α(m)) over low bitrate in residual codecs (small α(m)). In case of transmission errors, the decoder's memory will be out of sync with the encoder. This can be modeled as an error ε in the inter prediction term according to the following equation:

[0087] Subtracting the error-free decoding results in:

[0088] If a transmission error occurs in frame m, the residual error ε after M frames is res can be expressed as:

[0089] This can be expressed as a relative error:

[0090] Using a recursive definition, this can be expressed as:

[0091] Note that the sequence of α(m) is close to 1, which means that the weight for intra prediction is high, which will cause the relative error to drop to zero quickly. For α(m) close to zero, the weight for inter prediction of past frames will be larger and the relative error will decrease at a lower rate. Similarly, the recovery parameter can be defined as R(M) = (1-ε rel (M)). It can also be calculated according to the following formula:

[0092] By analyzing the error recovery rate of the coherent parameters, the side prediction parameter g b The absolute codec of (m) can be triggered so that the error resistance of the first parameter and the second parameter is balanced. One way to achieve this is to influence pred_allowed(m) according to the following formula:

[0093] where ε THR is the value of the threshold for relative error. THR = 0.01, if the potential relative error has dropped below 1%, the prediction will not be allowed. Note that pred_allowed(m) needs to depend on the relative error of the previous frame m-1, because the updated ε rel (m) depends on the mode selection of the current frame m. Further combinations are also possible, for example using a frame counter and recovery analysis according to the following formula:

[0094] Note that the mode selection can be further tuned to use ε rel (m-1), which can be adjusted as follows: Where B is a constant.

[0095] Calculate the relative error ε rel An alternative approach to (m) is to accumulate α(m) weighting factors:

[0096] Similarly, according to the following formula α acc (m) can be used to affect pred_allowed(m): where α acc,max is a constant corresponding to the level of error recovery, or affects the mode selection according to the following formula: where G is a constant that gives a suitable bit margin given the input material to be encoded and the observed value of α(m) for the material.

[0097] In the above description of various embodiments, it should be understood that the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the inventive concept. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meanings as those generally understood by those of ordinary skill in the art to which the inventive concept belongs. It should also be understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having meanings consistent with their meanings in the context of this specification and related technologies, and will not be interpreted in an idealized or excessive manner. Unless explicitly defined herein, all are formal meanings.

[0098] When an element is referred to as being "connected", "coupled", "responsive" or its variants to another element, it may be directly connected, coupled or responsive to another element, or there may be an intermediate element. In contrast, when an element is referred to as being "directly connected", "directly coupled", "directly responsive" or its variants to another element, there is no intermediate element. Similar numbers refer to similar elements throughout the text. In addition, "coupling", "connection", "responsiveness" or its variants used herein may include wireless coupling, connection or response. As used herein, the singular forms "one", "an" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. For the sake of brevity and / or clarity, well-known functions or configurations may not be described in detail. The term "and / or" (abbreviated as " / ") includes any and all combinations of one or more related listed items.

[0099] It should be understood that although the terms first, second, third, etc. can be used herein to describe various elements / operations, these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another element / operation. Therefore, without departing from the teaching of the present invention, the first element / operation in some embodiments can be referred to as the second element / operation in other embodiments. Throughout the specification, the same reference numerals or the same reference numerals represent the same or similar elements.

[0100] As used herein, the terms "comprises," "including," "having," or variations thereof are open ended and include one or more stated features, integers, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or groups. In addition, as used herein, the common abbreviation "eg," derived from the Latin phrase "exempligratia," may be used to introduce or designate one or more general examples of previously mentioned items, and is not intended to limit such items. The common abbreviation "ie," derived from the Latin phrase "id est," may be used to designate a specific item from a more general narrative.

[0101] Example embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, devices (systems and / or equipment), and / or computer program products. It should be understood that the blocks in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to processor circuits of general-purpose computer circuits, special-purpose computer circuits, and / or other programmable data processing circuits to produce a machine, so that instructions executed by a processor of a computer and / or other programmable data processing devices, transform and control transistors, values ​​stored in memory locations, and other hardware components within such circuits, to implement the functions / actions specified in the block diagrams and / or flowchart blocks or multiple blocks, and thereby create a device (function) and / or a structure for implementing the functions / actions specified in the block diagrams and / or (multiple) flowchart blocks.

[0102] These computer program instructions may also be stored in a tangible computer-readable medium, which may direct a computer or other programmable data processing device to operate in a particular manner so that the instructions stored in the computer-readable medium produce an article of manufacture including instructions for implementing the functions / actions specified in the block diagram and / or flowchart blocks. Thus, embodiments of the present inventive concept may be embodied in hardware and / or in software (including firmware, resident software, microcode, etc.) running on a processor such as a digital signal processor, which may be collectively referred to as a digital signal processor. As a "circuit," "module," or variations thereof.

[0103] It should also be noted that in some alternative embodiments, the function / action marked in the box may not occur in the order marked in the flow chart. For example, the two boxes shown in succession can actually be performed substantially simultaneously, or these boxes can sometimes be performed in reverse order, depending on the function / action involved. In addition, the function of a given box of a flow chart and / or block diagram can be divided into multiple boxes and / or the function of two or more boxes of a flow chart and / or block diagram can be integrated at least in part. Finally, without departing from the scope of the present invention, other boxes can be added / inserted between the boxes shown, and / or boxes / operations can be omitted. In addition, although some figures include arrows on the communication path to show the main direction of communication, it should be understood that communication can occur in the direction opposite to the arrows depicted.

[0104] In the case of not departing from the principle of the inventive concept in fact, many changes and modifications can be made to the embodiments.All such changes and modifications are intended to be included in the scope of the inventive concept.Therefore, the subject matter disclosed above should be considered to be illustrative rather than restrictive, and the examples of the embodiments are intended to cover all such modifications, enhancements and other embodiments that fall within the spirit and scope of the inventive concept.Therefore, to the maximum extent permitted by law, the scope of the inventive concept is determined by the most widely allowed interpretation of the present disclosure (including examples of embodiments and their equivalents), and should not be subject to the constraints or limitations of the foregoing.

Claims

1. A method for selecting a coding mode for a current frame of an audio signal, performed in an audio encoder (300), the method comprising: Obtaining (725) absolute codec and predicted codec bit rates for the current frame; calculating (730) a bit rate difference based on the acquired bit rate; According to bitdiff LP (m)=ybitdiff(m)+(1-γ)bitdiff LP (m-1) low pass filtering the bit rate difference (735), where γ is a low pass filter coefficient; as well as The codec mode is selected (740) based on the bit rate difference, the low pass filtered bit rate difference, and a prediction mode counter, the prediction mode counter being the number of frames since the last absolute codec frame.

2. The method according to claim 1, further comprising: In response to selecting a predictive codec mode: Setting (750) the codec mode to the predictive codec mode; as well as The prediction mode counter is incremented (755).

3. The method according to any one of claims 1 to 2, further comprising: In response to selecting the absolute codec mode: Setting (715) the codec mode to the absolute codec mode; as well as The prediction mode counter is reset (720) to an initial value.

4. The method according to any one of claims 1 to 3, further comprising determining (910) whether predictive coding is allowed. 5 . The method of claim 4 , wherein determining whether predictive codec is allowed comprises determining whether a flag indicates that predictive codec is allowed.

6. The method of claim 5, wherein determining whether the flag indicates that predictive coding is allowed comprises: determining (1000) whether the prediction count is less than a maximum number of prediction frames; In response to determining that the prediction count is less than the maximum number of prediction frames, setting (1010) the flag to indicate that predictive coding is enabled; as well as In response to determining that the prediction count is greater than the maximum number of prediction frames, the flag is set (1020) to indicate that predictive coding is not allowed.

7. The method according to claim 4, further comprising: In response to determining that predictive codec is not allowed: Setting (715) the codec mode to absolute codec mode; as well as The prediction mode counter is set (720) to an initial value.

8. The method according to any one of claims 1 to 7, wherein calculating the bit rate difference comprises calculating the bit rate difference according to the following formula: bitdiff(m)=nbits abs (m)-nbits pred (m), Where nbits abs (m) is the bit rate for the absolute codec, and nbits pred (m) is the bit rate for predictive codec.

9. The method according to any one of claims 1 to 8, wherein selecting the codec mode based on the bit rate difference, the low-pass filtered bit rate difference, and the prediction mode counter comprises selecting the codec mode according to the following formula: in bitdiff(m)=nbits abs (m)-nbits pred (m), and bitdiff LP (m)=γbitdiff(m)+(1-γ)bitdiff LP (m-1) where g mode (m) is the codec mode, predcount(m-1) is the number of frames since the last absolute codec frame, C is the tuning constant, bitdiff(m) is the bitrate difference, nbits pred (m) is the estimated number of bits that will be used for predictive coding, nbits abs (m) is the estimated number of bits that will be used for absolute encoding and decoding, bitdiff LP (m) is the low-pass filtered bit rate difference, γ is the low-pass filter coefficient, and MAX_PRED_STREAK is the maximum number of predicted consecutive frames allowed.

10. The method according to claim 9, wherein The SID frame is a silence insertion descriptor frame.

11. The method according to claim 9, wherein γ∈[0.01, 0.20] and C∈[0，1]。 12. The method according to any one of claims 9 to 11, wherein g mode (m) is further determined based on pred_allowed(m) according to the following formula: in 13. The method of any one of claims 1 to 8, wherein selecting the codec mode based on the bit rate difference, the low pass filtered bit rate difference, and the prediction mode counter comprises selecting a codec mode according to the following formula: where gm ode (m) is the codec mode, predcount(m) is the number of frames since the last absolute codec frame, nbits pred (m) is the estimated number of bits that will be used for predictive coding, nbits abs (m) is the estimated number of bits that will be used for absolute encoding, A is a constant, and MAX_PRED_STREAK is the maximum number of predicted consecutive frames allowed. The method according to claim 13 , wherein A∈(0,2].

15. The method of any one of claims 1 to 8, wherein selecting the codec mode based on the bit rate difference, the low pass filtered bit rate difference, and the prediction mode counter comprises selecting a codec mode according to the following formula: where g mode (m) is the encoding / decoding mode, nbits pred (m) is the estimated number of bits that will be used for predictive coding, nbits abs (m) is the estimated number of bits that will be used for absolute encoding and decoding, G is a constant, and α acc (m) is the weighting factor, ε rel (m) is the relative error, and ε THR is the value of the threshold for the relative error.

16. An audio encoder (300) for selecting a codec mode for a current frame of an audio signal, the encoder being adapted to perform the method according to any one of claims 1 to 15.

Images