Apparatus and method for improved hiding of adaptive codebook in ACELP-like hiding using improved pitch lag estimation

Abstract

An apparatus and method for improved concealment of an adaptive codebook in ACELP-like concealment using improved pitch lag estimation are provided. The apparatus comprises an input interface (110) for receiving a plurality of original pitch lag values, and a pitch lag estimator (120) for estimating the estimated pitch lag. The pitch lag estimator (120) is configured to estimate the estimated pitch lag based on the plurality of original pitch lag values ​​and based on a plurality of information values, wherein for each original pitch lag value in the plurality of original pitch lag values, an information value in the plurality of information values ​​is assigned to the original pitch lag value.

Description

[0001] The present application is a divisional application of Fraunhofer Gesellschaft, filed on June 16, 2014, with application number 201480035427.3 and invention name “Device and method for improved hiding of adaptive codebook in ACELP-like hiding using improved pitch lag estimation”. Technical Field

[0002] The present invention relates to audio signal processing, in particular to speech processing, and more particularly to an apparatus and method for improved concealment of an adaptive codebook in ACELP-like (Algebraic Code Excited Linear Prediction-like) concealment. Background Art

[0003] Audio signal processing is becoming more and more important. In the field of audio signal processing, concealment techniques play an important role. When frames are lost or damaged, the information lost due to the lost or damaged frames must be put back. In speech signal processing, especially, when considering ACELP or ACELP-like speech codecs, pitch information is very important. Pitch prediction techniques as well as pulse resynchronization techniques are required.

[0004] Regarding pitch reconstruction, there are different pitch extrapolation techniques in the prior art.

[0005] One of these techniques is the repetition-based technique. Most existing codecs apply a concealment method based on simple repetition, which means that the last correctly received pitch period before the packet loss is repeated until a good frame arrives and new pitch information can be decoded from the bitstream. Alternatively, a pitch stability logic is applied, according to which a pitch value that has been received some time before the packet loss is selected. Codecs that follow the repetition-based approach are, for example, G.719 (see [ITU08b, 8.6]), G.729 (see [ITU12, 4.4]), AMR (see [3GP12a, 6.2.3.1], [ITU03]), AMR-WB (see [3GP12b, 6.2.3.4.2]) and AMR-WB+ (ACELP and TCX20 (ACELP-like) concealment) (see [3GP09]); (AMR = Adaptive Multi-Rate; AMR-WB = Adaptive Multi-Rate Wideband).

[0006] Another prior art pitch reconstruction technique is pitch derivation from the time domain. For some codecs, pitch is necessary for concealment, but is not embedded in the bitstream. Therefore, the pitch is calculated based on the time domain signal of the previous frame in order to calculate the pitch period, which is then kept constant during concealment. Codecs that follow this approach are, for example, G.722, see, in particular, G.722 Annex 3 (see [ITU06a, III.6.6 and III.6.7]) and G.722 Annex 4 (see [ITU07, IV.6.1.2.5]).

[0007] Yet another prior art pitch reconstruction technique is based on extrapolation. Some existing codecs apply pitch extrapolation methods and implement specific algorithms to change pitch based on the extrapolated pitch estimate when packets are lost. These methods will be described in more detail below with reference to G.718 and G.729.1.

[0008] First, G.718 is considered (see [ITU08a]). An estimation of the future pitch is performed by extrapolation to support the glottal pulse resynchronization module. This information of possible future pitch values ​​is used to synchronize the glottal pulses of the hidden excitation.

[0009] Pitch extrapolation is performed only when the last good frame is not silent. G.718's pitch extrapolation is based on the assumption that the encoder has a smooth pitch contour. The extrapolation is based on the pitch lag of the last seven subframes before silence. And was carried out.

[0010] In G.718, the history update of the floating pitch value is performed after each correctly received frame. For this purpose, the pitch value is updated only when the core mode is not silent. In the case of a lost frame, the difference between the floating pitch lags is calculated according to formula (1):

[0011]

[0012] In formula (1), represents the pitch lag of the last (i.e., 4th) subframe of the previous frame; Indicates the pitch lag of the 3rd subframe of the previous frame; and so on.

[0013] According to G.718, the difference The sum of is calculated as formula (2):

[0014]

[0015] Due to the value Can be positive or negative. The number of sign reversals is added and the position of the first reversal is indicated by a parameter stored in memory.

[0016] The parameter f is obtained by formula (3): corr :

[0017]

[0018] where d max =231 is the maximum value of the pitch lag considered.

[0019] In G.718, the position i indicating the maximum absolute difference is obtained according to the following definition: max :

[0020]

[0021] And the ratio for this maximum difference is calculated as follows:

[0022]

[0023] If this ratio is greater than or equal to 5, the pitch of the 4th subframe of the last correctly received frame is used for all subframes to be concealed. If this ratio is greater than or equal to 5, this means that the algorithm is not reliable enough to extrapolate the pitch and the glottal pulse resynchronization will not be performed.

[0024] If r max If is less than 5, then additional processing is performed to achieve the best possible extrapolation. Three different methods are used to extrapolate the future pitch. To choose between possible pitch extrapolation algorithms, the deviation parameter f is calculated corr2 , which depends on the factor f corr and the position i of the maximum pitch change max . But first, modify the average float pitch differences to remove pitch differences that are too large from the average:

[0025] If f corr <0.98 and if i max =3, then the average partial pitch difference is determined according to formula (5):

[0026]

[0027] to remove the pitch difference on the transition between two frames.

[0028] If f corr <≥0.98 or if i max ≠3, then calculate the average partial pitch difference according to formula (6):

[0029] And the maximum floating pitch difference is replaced by this new average:

[0030]

[0031] Using this new mean of floating pitch differences, the standard deviation f is calculated as follows corr2 :

[0032]

[0033] Among them I sf In the first case it is equal to 4 and in the second case it is equal to 6.

[0034] Based on this new parameter, choose between three methods of extrapolating future pitches:

[0035] if changes sign more than twice (which indicates a high pitch change), the first sign reversal is in the last good frame (for i < 3), and f corr2 >0.945, then the extrapolated pitch d is calculated as follows ext (The extrapolated pitch is also denoted as T ext ):

[0036]

[0037]

[0038]

[0039] If 0.945<f corr2 <0.99 and If the sign changes at least once, the pitch is extrapolated using the weighted average of the partial pitch differences. The weight f of the average difference w With standard deviation f corr2 The first symbol is related to the position of the reverse as follows:

[0040]

[0041] The parameter i in the formula mem depending on The position of the first sign reversal, such that if the first sign reversal occurs between the last two subframes of the past frame, then i mem = 0, so that if the first symbol reversal occurs between the 2nd and 3rd subframes of the past frame, then i mem =1, and so on. If the first symbol is reversed close to the end of the last frame, this means that the pitch change was not stable just before the lost frame. Therefore the weighting factor applied to the average will be close to 0 and the extrapolated pitch d extwill be the tone of the 4th subframe close to the last good frame:

[0042]

[0043] Otherwise, the pitch evolution is considered stable and the extrapolated pitch d is determined as follows ext :

[0044]

[0045] After this processing, the pitch lag is limited between 34 and 231 (the values ​​indicate the minimum and maximum allowed pitch lag).

[0046] Now, to illustrate another example of a pitch reconstruction technique based on extrapolation, G.729.1 is considered (see [ITU06b]).

[0047] G.729.1 features a pitch extrapolation method in cases where no forward error concealment information (e.g. phase information) is decodable (see [Gao]). This situation occurs, for example, if two consecutive frames are lost (a superframe consists of four frames and can be either ACELP or TCX20). TCX40 or TCX80 frames and almost all combinations thereof are also possible.

[0048] When one or more frames are lost in a voiced region, previous pitch information is typically used to reconstruct the currently lost frame. The accuracy of the currently estimated pitch may directly affect the phase alignment with the original signal, and it is critical to the reconstruction quality of the currently lost frame and the frames received after the lost frame. Using multiple past pitch lags instead of just copying the previous pitch lag will result in statistically better pitch estimates. In the G.729.1 encoder, the pitch extrapolation used for FEC (forward error correction) consists of a linear extrapolation based on the past five pitch values. The past five pitch values ​​are P(i), for i=0, 1, 2, 3, 4, where P(4) is the most recent pitch value. The extrapolation model is defined according to formula (9):

[0049] P′(i)=a+i·b (9)

[0050] Then, the extrapolated pitch value for the first subframe in the lost frame is defined as equation (10):

[0051] P′(5)=a+5·b (10)

[0052] To determine the coefficients a and b, the error E is minimized, where the error E is defined according to formula (11):

[0053]

[0054] By setting

[0055]

[0056] Get a and b:

[0057]

[0058] Below, the existing frame erasure concealment concept for AMR-WB codec as proposed in [MCZ11] is described. This frame erasure concealment concept is based on pitch and gain linear prediction. The article proposes a linear pitch interpolation / extrapolation method in the case of frame loss based on the minimum mean square error criterion.

[0059] According to this frame deletion concealment concept, at the decoder side, when the type of the last valid frame before the deleted frame (past frame) is the same as the type of the first frame after the deleted frame (future frame), a pitch P(i) is defined, where i=-N, -N+1,...,0,1,...,N+4,N+5, and where N is the number of past and future subframes of the deleted frame. P(1), P(2), P(3), P(4) are the four tones of the four subframes in the deleted frame, P(0), P(-1),..., P(-N) are the tones of the past subframe, and P(5), P(6),..., P(N+5) are the tones of the future subframe. A linear prediction model P'(i)=a+b·i is adopted. For i=1, 2, 3, 4; P'(1), P'(2), P'(3), P'(4) are the predicted tones for the deleted frame. The MMS criterion (MMS = Least Mean Square) is taken into account to derive the values ​​of the two prediction coefficients a and b according to an interpolation method. According to this method, the error E is defined as:

[0060]

[0061] Then, the coefficients a and b can be calculated by the following formula:

[0062]

[0063]

[0064]

[0065] The pitch lag for the last four subframes of the deleted frame can be calculated according to formula (14e):

[0066] P′(1)=a+b·1: P′(2)=a+b·2

[0067] P′(3)=a+b·3; P′(4)=a+b·4 (14e)

[0068] It was found that N = 4 provided the best results. N = 4 means that 5 past subframes and 5 future subframes are used for interpolation.

[0069] However, when the type of the past frame is different from the type of the future frame, for example, when the past frame is voiced but the future frame is unvoiced, only the voiced pitch of the past frame or the future frame is used to predict the pitch of the deleted frame using the above extrapolation method.

[0070] Now, consider the prior art of pulse resynchronization, with particular reference to G.718 and G.729.1. The method of pulse resynchronization is described in [VJGS12].

[0071] First, the cyclical part of constructing incentives is explained.

[0072] For concealment of deleted frames after correct reception of frames other than silence, the periodic part of the excitation is constructed by repeating the last low-pass filtered pitch period of the previous frame.

[0073] Construction of the periodic portion is done using a simple copy of a low pass filtered segment of the excitation signal from the end of the previous frame.

[0074] The pitch period length is rounded to the nearest integer:

[0075] T c =round(last note) (15a)

[0076] Consider the final pitch cycle length is T p , then the length of the copied segment is T r It can be defined, for example, according to formula (15b):

[0077]

[0078] A periodic part is constructed for one frame and one additional subframe.

[0079] For example, there are M subframes in a frame, and the subframe length is

[0080] Where L is the frame length, also denoted as L frame :L=L frame .

[0081] Figure 3 The build-up period portion of the speech signal is shown.

[0082] T[0] is the position of the first largest pulse in the build-up period portion of the excitation. The positions of the other pulses are given by:

[0083] T[i]=T[0]+iT c(16a)

[0084] Corresponds to

[0085] T[i]=T[0]+iT r (16b)

[0086] After construction of the periodic portion of the excitation, glottal pulse resynchronization is performed to correct for the difference between the estimated target position (P) of the last pulse of the lost frame and its actual position (T[k]) in the construction periodic portion of the excitation.

[0087] The pitch lag evolution is extrapolated based on the pitch lag of the last seven subframes before the lost frame. The evolving pitch lag in each subframe is:

[0088] p[i]=round(T c +(i+1)δ), 0≤i<M (17a)

[0089] in

[0090]

[0091] And T ext (Also denoted as d ext ) is the extrapolated pitch, as above for d ext Description.

[0092] Find the pitch period with constant pitch within the frame length (T c ) and the difference between the total number of samples and the total number of samples in the pitch period with evolving pitch p[i] (denoted d). The literature does not explain how to find d.

[0093] In the source code of G.718 (see [ITU08a]), d is found using the following algorithm (where M is the number of subframes in a frame):

[0094]

[0095] The number of pulses in the build-up period part of the frame length plus the first pulse in the future frame is N. How to find N is not described in the literature.

[0096] In the source code of G.718 (see [ITU08a]), N is found according to the following formula:

[0097]

[0098] The position T[n] of the last pulse in the build-up period part of the excitation belonging to the lost frame is determined by the following formula:

[0099]

[0100] The estimated final pulse position P is:

[0101] P=T[n]+d (19a)

[0102] The actual position T[k] of the last pulse position is the position of the pulse in the build-up period part of the excitation that is closest to the estimated target position P (contained in the search for the first pulse after the current frame):

[0103]

[0104] Glottal pulse resynchronization is performed by adding or removing samples in the minimum energy region of the entire pitch period. The number of samples added or removed is determined by the difference between:

[0105] diff=PT[k] (19c)

[0106] A sliding 5-sample window is used to determine the minimum energy region. The minimum energy position is set in the middle of the window, where the energy is minimum. c / 8 to T[i+1]-T c / 4 to search. min =n-1 minimum energy regions.

[0107] If N min =1, there is only one minimum energy region and diff samples are inserted or deleted at this position.

[0108] For N min >1, fewer samples are added or removed at the beginning, and more samples are added or removed towards the end of the frame. Use the following recursive relationship to find the number of samples removed or added between pulses T[i] and T[i+1]:

[0109]

[0110] If R[i]<R[i-1], the values ​​of R[i] and R[i-1] are swapped. Summary of the invention

[0111] It is an object of the present invention to provide improved concepts for audio signal processing, in particular to provide improved concepts for speech processing, and more particularly to provide improved hiding concepts.

[0112] The object of the present invention is achieved by the apparatus, method and computer program described below.

[0113] An apparatus for determining an estimated pitch lag is provided. The apparatus comprises an input interface for receiving a plurality of raw pitch lag values, and a pitch lag estimator for estimating the estimated pitch lag. The pitch lag estimator is configured to estimate the estimated pitch lag based on the plurality of raw pitch lag values ​​and based on a plurality of information values, wherein for each raw pitch lag value in the plurality of raw pitch lag values, an information value in the plurality of information values ​​is assigned to the raw pitch lag value.

[0114] According to an embodiment, the pitch lag estimator, for example, can be used to estimate the estimated pitch lag based on the multiple original pitch lag values ​​and based on a plurality of pitch gain values ​​as the multiple information values, wherein for each original pitch lag value of the multiple original pitch lag values, a pitch gain value of the multiple pitch gain values ​​is assigned to the original pitch lag value.

[0115] In certain embodiments, each of the plurality of pitch gain values ​​may, for example, be an adaptive codebook gain.

[0116] In an embodiment, the pitch lag estimator may, for example, be configured to estimate the estimated pitch lag by minimizing an error function.

[0117] According to an embodiment, the pitch lag estimator may be configured, for example, to estimate the estimated pitch lag by determining two parameters a, b by minimizing the following error function,

[0118]

[0119] where a is a real number, where b is a real number, where k is an integer with k ≥ 2, and where P(i) is the i-th original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th pitch lag value P(i).

[0120] In an embodiment, the pitch lag estimator may be configured to estimate the estimated pitch lag by determining two parameters a, b, for example, by minimizing the following error function:

[0121]

[0122] where a is a real number, where b is a real number, where P(i) is the ith original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th pitch lag value P(i).

[0123] According to an embodiment, the pitch lag estimator may, for example, be configured to determine the estimated pitch lag p according to p=a·i+b.

[0124] In an embodiment, the pitch lag estimator, for example, may be configured to estimate the estimated pitch lag based on the plurality of original pitch lag values ​​based on a plurality of time values ​​serving as the plurality of information values, wherein for each of the plurality of original pitch lag values, a time value from the plurality of time values ​​is assigned to the original pitch lag value.

[0125] According to an embodiment, the pitch lag estimator may, for example, be configured to estimate the estimated pitch lag by minimizing an error function.

[0126] In an embodiment, the pitch lag estimator may be configured, for example, to estimate the estimated pitch lag by determining two parameters a, b by minimizing the following error function,

[0127]

[0128] where a is a real number, where b is a real number, where k is an integer with k ≥ 2, and where P(i) is the i-th original pitch lag value, where time passed (i) is the i-th time value assigned to the i-th pitch lag value P(i).

[0129] According to an embodiment, the pitch lag estimator may be configured, for example, to estimate the estimated pitch lag by determining two parameters a, b by minimizing the following error function,

[0130]

[0131] where a is a real number, where b is a real number, where P(i) is the ith original pitch lag value, where time passed (i) is the i-th time value assigned to the i-th pitch lag value P(i).

[0132] In an embodiment, the pitch lag estimator may, for example, be configured to determine the estimated pitch lag p according to p=a·i+b.

[0133] Furthermore, a method for determining an estimated pitch lag is provided. The method comprises:

[0134] receiving a plurality of raw pitch lag values; and

[0135] The estimated pitch lag is estimated.

[0136] The estimated pitch lag is estimated based on a plurality of original pitch lag values ​​and based on a plurality of information values, wherein for each original pitch lag value of the plurality of original pitch lag values, an information value of the plurality of information values ​​is assigned to the original pitch lag value.

[0137] Furthermore, a computer program is provided, which, when executed on a computer or a signal processor, is used to implement the above method.

[0138] In addition, a device for reconstructing a frame including a speech signal as a reconstructed frame is provided, the reconstructed frame being associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as one or more available pitch periods. The device comprises: a determination unit for determining a sample number difference, the sample number difference indicating a difference between the number of samples of one of the one or more available pitch periods and the number of samples of a first pitch period to be reconstructed. And, the device comprises a frame reconstructor for reconstructing the reconstructed frame by reconstructing the first pitch period to be reconstructed as a first reconstructed pitch period according to the sample number difference and according to samples of one of the one or more available pitch periods. The frame reconstructor is for reconstructing the reconstructed frame so that the reconstructed frame completely or partially includes the first reconstructed pitch period, so that the reconstructed frame completely or partially includes the second reconstructed pitch period, and so that the number of samples of the first reconstructed pitch period is different from the number of samples of the second reconstructed pitch period.

[0139] According to an embodiment, the determining unit, for example, may be configured to determine a sample number difference for each of a plurality of pitch periods to be reconstructed, such that the sample number difference for each of the pitch periods indicates a difference between a sample number of one of the one or more available pitch periods and a sample number of the pitch period to be reconstructed. The frame reconstructor, for example, may be configured to reconstruct each of the plurality of pitch periods to be reconstructed based on the sample number difference of the pitch period to be reconstructed and based on samples of one of the one or more available pitch periods, thereby reconstructing the reconstructed frame.

[0140] In an embodiment, the frame reconstructor may, for example, be configured to generate an intermediate frame according to one of the one or more available pitch periods. The frame reconstructor may, for example, be configured to modify the intermediate frame to obtain the reconstructed frame.

[0141] According to an embodiment, the determining unit, for example, may be used to determine a frame difference value (d; s) indicating how many samples are to be removed from the intermediate frame or how many samples are to be added to the intermediate frame. In addition, the frame reconstructor, for example, may be used to remove the first sample from the intermediate frame to obtain the reconstructed frame when the frame difference value indicates that the first sample should be removed from the frame. In addition, the frame reconstructor, for example, may be used to add the second sample to the intermediate frame to obtain the reconstructed frame when the frame difference value (d; s) indicates that the second sample should be added to the frame.

[0142] In an embodiment, the frame reconstructor, for example, may be configured to remove the first sample from the intermediate frame when the frame difference value indicates that the first sample should be removed from the frame, whereby the number of the first sample removed from the intermediate frame is indicated by the frame difference value. In addition, the frame reconstructor, for example, may be configured to add the second sample to the intermediate frame when the frame difference value indicates that the second sample should be added to the frame, whereby the number of the second sample added to the intermediate frame is indicated by the frame difference value.

[0143] According to an embodiment, the determining unit, for example, may be used to determine the frame difference s, and thus the following formula is applicable:

[0144]

[0145] Where L indicates the number of samples of the reconstructed frame, Where M indicates the number of subframes of the reconstructed frame, Where T r indicates a rounded pitch period length of one of the one or more available pitch periods, and wherein p[i] indicates the pitch period length of the reconstructed pitch period of the i-th subframe of the reconstructed frame.

[0146] In an embodiment, the frame reconstructor may, for example, be adapted to generate an intermediate frame in dependence on one of the one or more available pitch periods. Furthermore, the frame reconstructor may, for example, be adapted to generate the intermediate frame such that the intermediate frame comprises a first partial intermediate pitch period, one or more other intermediate pitch periods, and a second partial intermediate pitch period. Furthermore, the first partial intermediate pitch period may, for example, depend on one or more of the samples of one of the one or more available pitch periods, wherein each of the one or more other intermediate pitch periods depends on all samples of the one of the one or more available pitch periods, and wherein the second partial intermediate pitch period depends on one or more of the samples of the one of the one or more available pitch periods. Furthermore, the determining unit may, for example, be adapted to determine a starting partial delta indicating how many samples are to be removed from the first partial intermediate pitch period or added to the first partial intermediate pitch period, and wherein the frame reconstructor is adapted to remove one or more first samples from the first partial intermediate pitch period, or to add one or more first samples to the first partial intermediate pitch period, in dependence on the starting partial delta. Furthermore, the determination unit, for example, may be configured to determine a pitch period difference for each of the other intermediate pitch periods, the pitch period difference indicating how many samples are to be removed from or added to the one of the other intermediate pitch periods. Furthermore, the frame reconstructor, for example, may be configured to remove one or more second samples from or add one or more second samples to the one of the other intermediate pitch periods in accordance with the pitch period difference. Furthermore, the determination unit, for example, may be configured to determine an end portion difference indicating how many samples are to be removed from or added to the second portion intermediate pitch period, and wherein the frame reconstructor is configured to remove one or more third samples from or add one or more third samples to the second portion intermediate pitch period in accordance with the end portion difference.

[0147] According to an embodiment, the frame reconstructor, for example, may be used to generate an intermediate frame based on the one of the one or more available pitch periods. In addition, the determination unit, for example, may be adapted to determine one or more low-energy signal portions of the speech signal included in the intermediate frame, wherein each of the one or more low-energy signal portions is a first signal portion of the speech signal within the intermediate frame, wherein the energy of the speech signal is lower than the energy in the second signal portion of the speech signal included in the intermediate frame. In addition, the frame reconstructor, for example, may be used to remove one or more samples from at least one of the one or more low-energy signal portions of the speech signal, or add one or more samples to at least one of the one or more low-energy signal portions of the speech signal to obtain the reconstructed frame.

[0148] In a particular embodiment, the frame reconstructor, for example, may be configured to generate the intermediate frame such that the intermediate frame comprises one or more reconstructed pitch periods such that each of the one or more reconstructed pitch periods depends on the one of the one or more available pitch periods. Furthermore, the determination unit, for example, may be configured to determine the number of samples that should be removed from each of the one or more reconstructed pitch periods. Furthermore, the determination unit, for example, may be configured to determine each of the one or more low-energy signal portions such that for each of the one or more low-energy signal portions, the number of samples of the low-energy signal portion depends on the number of samples that should be removed from one of the one or more reconstructed pitch periods, wherein the low-energy signal portion is located within one of the one or more reconstructed pitch periods.

[0149] In an embodiment, the determining unit, for example, may be used to determine the position of one or more pulses of the speech signal of the frame to be reconstructed as the reconstructed frame. In addition, the frame reconstructor, for example, may be used to reconstruct the reconstructed frame based on the position of the one or more pulses of the speech signal.

[0150] According to an embodiment, the determining unit, for example, may be used to determine the positions of two or more pulses of the speech signal of the frame to be reconstructed as a reconstructed frame, wherein T[0] is the position of one of the two or more pulses of the speech signal with the frame to be reconstructed as a reconstructed frame, and wherein the determining unit is used to determine the positions (T[i]) of the other pulses of the two or more pulses of the speech signal according to the following formula:

[0151] T[i]=T[0]+iT r

[0152] Where T rIndicates the rounded length of the one of the one or more available pitch periods and wherein i is an integer.

[0153] According to an embodiment, the determining unit may, for example, be configured to determine the index k of the last pulse of the tone signal of the frame to be reconstructed as the reconstructed frame, such that

[0154]

[0155] wherein L indicates the number of samples of the reconstructed frame, wherein s indicates the frame difference, wherein T[0] indicates the position of a pulse of the speech signal of the frame to be reconstructed as the reconstructed frame (other than the last pulse of the speech signal), which is, and wherein T r Indicates a rounded length of the one of the one or more available pitch periods.

[0156] In an embodiment, the determining unit, for example, may be configured to reconstruct the frame to be reconstructed as the reconstructed frame by determining a parameter δ, wherein the parameter δ is defined according to the following formula:

[0157]

[0158] The frame to be reconstructed as the reconstructed frame comprises M subframes, wherein T p indicates the length of the one of the one or more available pitch periods, and wherein T ext Indicating the length of one of the to-be-reconstructed pitch periods of the frame to be reconstructed as the reconstructed frame.

[0159] According to an embodiment, the determining unit may be configured to determine the rounded length T of the one of the one or more available pitch periods based on the following formula: r , to reconstruct the reconstructed frame:

[0160]

[0161] Where T p Indicating a length of the one of the one or more available pitch periods.

[0162] In an embodiment, the determining unit may, for example, be configured to reconstruct the reconstructed frame by applying the following formula:

[0163]

[0164] Where T p indicates the length of the one of the one or more available pitch periods, wherein T rindicating a rounded length of said one of said one or more available pitch periods, wherein said frame to be reconstructed as said reconstructed frame comprises M subframes, wherein said frame to be reconstructed as said reconstructed frame comprises L samples, and wherein δ is a real number indicating a difference between the number of samples of said one of said one or more available pitch periods and the number of samples of one of said one or more pitch periods to be reconstructed.

[0165] Furthermore, a method for reconstructing a frame including a speech signal as a reconstructed frame is provided, the reconstructed frame being associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as one or more available pitch periods. The method comprises:

[0166] - Determine sample size difference The sample size difference indicating a difference between a number of samples of one of the one or more available pitch periods and a number of samples of a first pitch period to be reconstructed; and

[0167] - by the difference in sample size and reconstructing the reconstructed frame by reconstructing the first pitch period to be reconstructed as a first reconstructed pitch period based on samples of the one of the one or more available pitch periods.

[0168] Reconstructing the reconstructed frame is performed so that the reconstructed frame completely or partially includes the first reconstructed pitch period, so that the reconstructed frame completely or partially includes the second reconstructed pitch period, and so that the number of samples of the first reconstructed pitch period is different from the number of samples of the second reconstructed pitch period.

[0169] Furthermore, a computer program is provided, which, when executed on a computer or a signal processor, is used to implement the above method.

[0170] In addition, a system for reconstructing a frame comprising a speech signal is provided. The system comprises a device for determining an estimated pitch lag according to one of the above or below embodiments, and a device for reconstructing the frame, wherein the device for reconstructing the frame is used to reconstruct the frame according to the estimated pitch lag. The estimated pitch lag is the pitch lag of the speech signal.

[0171] In an embodiment, the reconstructed frame is, for example, associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as one or more available pitch periods. The device for reconstructing the frame may be, for example, a device for reconstructing a frame according to one of the above or below embodiments.

[0172] The present invention is based on the discovery that the prior art has important shortcomings. Both G.718 (see [ITU08a]) and G.729.1 (see [ITU06b]) use pitch extrapolation techniques in the event of frame loss. This is necessary because in the event of frame loss, the pitch lag is also lost. According to G.718 and G.729.1, the pitch is extrapolated by taking into account the pitch evolution in the last two frames. However, the pitch lag reconstructed by G.718 and G.729.1 is not very accurate, for example, and often produces a reconstructed pitch lag that is significantly different from the real pitch lag.

[0173] Embodiments of the invention provide for a more accurate pitch lag reconstruction.For this purpose, some embodiments take into account information about the reliability of the pitch information compared to G.718 and G.729.1.

[0174] According to the prior art, the pitch information on which the extrapolation technique is based includes the last eight correctly received pitch lags, for which the encoding mode is different from the silent case. However, in the prior art, the voiced characteristics may be weak, which is indicated by a low pitch gain (corresponding to a low prediction gain). In the prior art, when the extrapolation is based on pitch lags with different pitch gains, the extrapolation will not be able to output reasonable results or even fail at all and will fall back to a simple pitch lag repetition method.

[0175] The embodiments are based on the discovery that the reason for these prior art shortcomings is that on the encoder side, the pitch lag is selected with respect to maximizing the pitch gain in order to maximize the coding gain of the adaptive codebook, but, in case of weak speech characteristics, the pitch lag may not accurately indicate the fundamental frequency because noise in the speech signal causes the pitch lag estimation to become inaccurate.

[0176] Therefore, during the concealment process, according to an embodiment, the application of the pitch lag extrapolation is weighted according to the reliability of the previously received lags used for this extrapolation.

[0177] According to some embodiments, past adaptive codebook gains (pitch gains) may be employed as reliability measure.

[0178] According to some other embodiments of the present invention, a weight according to how far in the past the tone lag was received is used as a reliability measure. For example, a high weight is applied to more recent lags and a low weight is applied to lags received a long time ago.

[0179] According to an embodiment, a weighted pitch prediction concept is provided. Compared to the prior art, the pitch prediction provided by the embodiments of the present invention uses a reliability measure for each pitch lag on which it is based, making the prediction result more effective and stable. In particular, the pitch gain can be used as an indicator of reliability. Optionally or additionally, according to some embodiments, the time that has passed after the correct reception of the pitch lag, for example, can be used as an indicator.

[0180] Concerning pulse resynchronization, the present invention is based on the finding that one of the drawbacks of the prior art regarding glottal pulse resynchronization is that the pitch extrapolation does not take into account how many pulses (pitch periods) should be constructed in the concealed frame.

[0181] According to prior art, pitch extrapolation is performed such that changes in pitch are only expected at subframe boundaries.

[0182] According to an embodiment, pitch changes other than continuous pitch changes may be considered when performing glottal pulse resynchronization.

[0183] The embodiments of the present invention are based on the discovery that G.718 and G.729.1 have the following disadvantages:

[0184] First, in the prior art, when calculating d, it is assumed that there are an integer number of pitch periods in a frame. Because d defines the position of the last pulse in the concealed frame, when there are a non-integer number of pitch periods in the frame, the position of the last pulse will be incorrect. This is Figure 6 and Figure 7 Shown in. Figure 6 Shown is the speech signal before sample removal. Figure 7 The speech signal after the sample removal is shown. In addition, the algorithm used in the prior art to calculate d is inefficient.

[0185] Furthermore, the prior art calculations require the number of pulses N in the build-up period portion of the excitation. This adds unnecessary computational complexity.

[0186] Furthermore, in the prior art, the calculation of the number of pulses N in the build-up period portion of the excitation does not take into account the position of the first pulse.

[0187] Figure 4 and Figure 5 The signals represented in have the same pitch period length T c .

[0188] Figure 4A speech signal with 3 pulses in a frame is shown.

[0189] In contrast, Figure 5 A speech signal with only 2 pulses in a frame is shown.

[0190] Figure 4 and 5 The examples shown show that the number of pulses depends on the first pulse position.

[0191] Furthermore, according to the prior art, it is checked whether the position T[N-1] of the Nth pulse in the build-up period part of the excitation is within the frame length, although N is defined to include the first pulse in the next frame.

[0192] Furthermore, according to the prior art, no samples are added or removed before the first pulse and after the last pulse. The present invention is based on the finding that this leads to the disadvantage that there may be a sudden change in the length of the first complete pitch period, which further leads to the disadvantage that the length of the pitch period after the last pulse may be longer than the length of the last complete pitch period before the last pulse, even when the pitch lag is reduced (see Figure 6 and 7 ).

[0193] The embodiment is based on the discovery that the pulses T[k]=P-diff and T[n]=Pd are not equal when:

[0194] - In this case, diff = T c -d and the number of removed samples will be diff instead of d.

[0195] - T[k] is in the future frame and only after removing d samples, it moves to the current frame.

[0196] - After adding -d (d < 0) samples, T[n] moves to the future frame.

[0197] This will result in wrong positions of pulses in the concealed frame.

[0198] Furthermore, embodiments are based on the discovery that in the prior art, the maximum value of d is limited by the minimum allowed value for encoding pitch lag. This is a constraint that limits the occurrence of other problems, but it also limits the possible changes in pitch and therefore pulse resynchronization.

[0199] Furthermore, the embodiments are based on the discovery that in the prior art, the periodic part is constructed using an integer number of pitch lags and this results in a significant degradation in the concealment of the tonal signal with a constant pitch and the frequency shift of the harmonics. This degradation can be seen in Figure 8 ,in Figure 8Shown is a time-frequency representation of a speech signal that is resynchronized when a rounded pitch lag is used.

[0200] The embodiment is also based on the discovery that most of the problems of the prior art occur when Figure 6 and 7 The example shown describes the case where d samples are removed. No limit is considered here for the maximum value of d, in order to make the problem easily visible. The problem also occurs when there is a limit on d, but not so clearly visible. Instead of increasing the pitch continuously, a sudden increase in the pitch is followed by a sudden decrease. The embodiment is based on the discovery that this occurs because no samples are removed before and after the last pulse, which is indirectly also caused by not taking into account that the pulse T[2] moves within the frame after removing d samples. In this example, an incorrect calculation of N also occurs.

[0201] According to an embodiment, an improved pulse resynchronization concept is provided. The embodiment provides an improved concealment of monophonic signals (including speech), which is advantageous compared to the prior art described in standards G.718 (see [ITU08a]) and G.729.1 (see [ITU06b]). The provided embodiments are applicable to signals with constant pitch as well as to signals with varying pitch.

[0202] In addition, according to the embodiment, three technologies are provided:

[0203] According to a first technique provided by an embodiment, a search concept for pulses is provided which, in contrast to G.718 and G.729.1, takes into account the position of the first pulse in the calculation of the number of pulses (denoted N) in the construction period part.

[0204] According to a second technique provided in accordance with another embodiment, an algorithm for searching pulses is provided which, compared to G.718 and G.729.1, does not require constructing the number of pulses in a periodic portion (denoted as N), considers the position of the first pulse, and directly calculates the last pulse index (denoted as k) in the hidden frame.

[0205] According to a third technique provided by yet another embodiment, pulse searching is not required. According to this third technique, the construction of the periodic part is combined with the removal or addition of samples, thereby achieving less complexity compared to the prior art.

[0206] Additionally or alternatively, some embodiments provide the following changes to the above techniques and the techniques of G.718 and G.729.1:

[0207] - Fractional parts of the pitch lag, which can be used, for example, to construct periodic parts of a signal with constant pitch.

[0208] - For a non-integer number of pitch periods within a frame, for example, an offset from the expected position of the last pulse in the concealed frame can be calculated.

[0209] - For example, samples can also be added or removed before the first pulse and after the last pulse.

[0210] - If there is only one pulse, samples can also be added or removed, for example.

[0211] - The number of samples removed or added may also vary linearly, for example, with a predicted linear change in pitch. BRIEF DESCRIPTION OF THE DRAWINGS

[0212] Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings, in which:

[0213] Figure 1 An apparatus for determining an estimated pitch lag according to an embodiment is shown,

[0214] FIG. 2a shows an apparatus for reconstructing a frame including a speech signal as a reconstructed frame according to an embodiment,

[0215] FIG2b shows a speech signal comprising a plurality of pulses,

[0216] FIG. 2c shows a system for reconstructing a frame including a speech signal according to an embodiment,

[0217] Figure 3 shows the construction period part of the speech signal,

[0218] Figure 4 shows a speech signal with three pulses in a frame,

[0219] Figure 5 shows a speech signal with two pulses in a frame,

[0220] Figure 6 shows the speech signal before sample removal,

[0221] Figure 7 Shows the sample after removal Figure 6 The speech signal,

[0222] Figure 8 shows a time-frequency representation of a speech signal resynchronized using rounded pitch lags,

[0223] Fig. 9 shows a time-frequency representation of a speech signal resynchronized using an unrounded pitch lag with a fractional part,

[0224] Fig.10 shows a pitch lag diagram, where the pitch lag is reconstructed using prior art techniques,

[0225] Fig.11 shows a pitch lag diagram, wherein the pitch lag is reconstructed according to an embodiment,

[0226] Fig.12 shows the speech signal before removing the sample, and

[0227] Fig.13 Show Fig.12 , and Δ0 to Δ3 are also shown. DETAILED DESCRIPTION

[0228] Figure 1 An apparatus for determining an estimated pitch lag according to an embodiment is shown. The apparatus comprises an input interface 110 for receiving a plurality of original pitch lag values, and a pitch lag estimator 120 for estimating an estimated pitch lag. The pitch lag estimator 120 is configured to estimate the estimated pitch lag based on the plurality of original pitch lag values ​​and based on a plurality of information values, wherein for each original pitch lag value in the plurality of original pitch lag values, an information value in the plurality of information values ​​is assigned to the original pitch lag value.

[0229] According to an embodiment, the pitch lag estimator 120, for example, may be used to estimate the pitch lag based on a plurality of original pitch lag values ​​and based on a plurality of pitch gain values ​​as a plurality of information values, wherein for each original pitch lag value among the plurality of original pitch lag values, a pitch gain value among the plurality of pitch gain values ​​is assigned to the original pitch lag value.

[0230] In a particular embodiment, each of the plurality of pitch gain values ​​is an adaptive codebook gain.

[0231] In an embodiment, the pitch lag estimator 120 may, for example, be configured to estimate the pitch lag by minimizing an error function.

[0232] According to an embodiment, the pitch lag estimator 120 may be configured to estimate the pitch lag by determining two parameters a, b, for example, by minimizing the following error function:

[0233]

[0234] where a is a real number, where b is a real number, where k is an integer with k ≥ 2, and where P(i) is the i-th original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th pitch lag value P(i).

[0235] In an embodiment, the pitch lag estimator 120 may be configured to estimate the pitch lag by determining two parameters a, b, for example, by minimizing the following error function:

[0236]

[0237] where a is a real number, where b is a real number, where P(i) is the ith original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th pitch lag value P(i).

[0238] According to an embodiment, the pitch lag estimator 120 may, for example, be configured to determine an estimated pitch lag p according to p=a·i+b.

[0239] In an embodiment, the pitch lag estimator 120, for example, may be used to estimate the pitch lag based on a plurality of original pitch lag values ​​and based on a plurality of time values ​​as a plurality of information values, wherein for each original pitch lag value in the plurality of original pitch lag values, a time value in the plurality of time values ​​is assigned to the original pitch lag value.

[0240] According to an embodiment, the pitch lag estimator 120 may be configured to estimate the pitch lag by minimizing an error function, for example.

[0241] In an embodiment, the pitch lag estimator 120 may be configured to estimate the pitch lag by determining two parameters a, b, for example, by minimizing the following error function:

[0242]

[0243] where a is a real number, where b is a real number, where k is an integer with k ≥ 2, and where P(i) is the i-th original pitch lag value, where time passed (i) is the i-th time value assigned to the i-th pitch lag value P(i).

[0244] According to an embodiment, the pitch lag estimator 120 may be configured to estimate the pitch lag by determining two parameters a, b, for example, by minimizing the following error function:

[0245]

[0246] where a is a real number, where b is a real number, where P(i) is the ith original pitch lag value, where time passed (i) is the i-th time value assigned to the i-th pitch lag value P(i).

[0247] In an embodiment, the pitch lag estimator 120 is configured to determine an estimated pitch lag p according to p=a·i+b.

[0248]

[0066] Below, an embodiment for providing weighted pitch prediction is described with respect to equations (20)-(24b).

[0249] First, referring to equations (20)-(22c), embodiments of weighted pitch prediction using weighting according to pitch gain are described. According to some of these embodiments, in order to overcome the shortcomings of the prior art, pitch gain is used to weight the pitch lag for pitch prediction.

[0250] In some embodiments, the pitch gain may be the adaptive codebook gain gp defined in standard G.729 (see [ITU12], in particular Section 3.7.3, more particularly Formula (43)). In G.729, the adaptive codebook gain is determined according to the following formula:

[0251] 0≤g p ≤1.2 is the boundary,

[0252] Where x(n) is the target signal and y(n) is obtained by convolving v(n) with h(n) according to the following formula:

[0253]

[0254] where v(n) is the adaptive codebook vector, where y(n) is the filtered adaptive codebook vector, and where h(ni) is the impulse response of the weighted synthesis filter, as defined in G.729 (see [ITU12]).

[0255] Similarly, in some embodiments, the pitch gain may be the adaptive codebook gain g defined in standard G.718. p (See [ITU08a], especially Section 6.8.4.1.4.1, and more especially equation (170)). In G.718, the adaptive codebook gain is determined according to the following equation:

[0256]

[0257] where x(n) is the target signal and y k (n) is the past filtered excitation at delay k.

[0258] For example, see [ITU08a], section 6.8.4.1.4.1, equation (171), which defines how to define y k (n).

[0259] Similarly, in some embodiments, the pitch gain may be the adaptive codebook gain g defined in the AMR standard. p (See [3GP12b]), where the adaptive gain g is defined as the pitch gain according to p :

[0260] 0≤g p ≤1.2 is the boundary,

[0261] Where y(n) is the filtered adaptive codebook vector.

[0262] In some specific embodiments, for example, the pitch lag may be weighted using a pitch gain, for example, before pitch prediction is performed.

[0263] For this purpose, according to an embodiment, for example, a second buffer of length 8 may be introduced to store the pitch gain, which is taken at the same subframe as the pitch lag. In an embodiment, for example, the buffer may be updated using exactly the same rules as the update of the pitch lag. One possible implementation is to update both buffers (storing the pitch lag and pitch gain of the last eight subframes) at the end of each frame, regardless of whether this frame is error-free or error-prone.

[0264] There are two different prediction strategies known from the prior art, which can be enhanced to use weighted pitch prediction:

[0265] Some embodiments provide a significant inventive improvement over the prediction strategy of the G.718 standard. In G.718, in the case of packet loss, the buffers can be element-wise multiplied with each other so that the tone is weighted with a high coefficient if the associated tone gain is high and with a low coefficient if the associated tone gain is low. Then, according to G.718, tone prediction is performed as usual (see [ITU08a, section 7.11.1.3] for a description of G.718).

[0266] Some embodiments provide a significant inventive improvement of the prediction strategy of the G.729.1 standard. According to an embodiment, the algorithm used for predicting tones of G.729.1 (see [ITU06b] for a description of G.729.1) is modified to use weighted prediction.

[0267] According to some embodiments, the goal is to minimize the error function:

[0268]

[0269] where g p (i) holds the pitch gain of the past subframe and P(i) holds the corresponding pitch lag.

[0270] In the inventive formula (20), g p (i) represents the weighting coefficient. In the above example, each g p (i) represents the pitch gain from one of the past subframes.

[0271] Below, equations are provided according to an embodiment that illustrate how coefficients a and b are derived that can be used to predict the pitch lag according to a+i·b, where i is the subframe number of the subframe to be predicted.

[0272] For example, to obtain the first predicted subframe based on the prediction of the last five subframes P(0), ..., P(4), the predicted pitch value P(5) will be:

[0273] P(5)=a+5·b.

[0274] To derive the coefficients a and b, the error function, for example, can be derived (derivative) and can be set to zero:

[0275]

[0276] The prior art does not disclose the weighting technique of the present invention provided by the embodiment. In particular, the prior art does not use the weighting coefficient g p (i).

[0277] Therefore, in the prior art, the weighting coefficient g is not used. p (i) Deriving the error function and setting its derivative to zero results in:

[0278]

[0279] (See [ITU06b, 7.6.5]).

[0280] In contrast, when the weighted prediction method of the provided embodiment is used, for example, with a weighting coefficient g p (i) The weighted prediction method of formula (20), a and b become:

[0281]

[0282]

[0283] According to a specific embodiment, A, B, C, D; E, F, G, H, I, J and K, for example, may have the following values:

[0284]

[0285]

[0286]

[0287]

[0288]

[0289]

[0290]

[0291]

[0292]

[0293]

[0294]

[0295] Fig.10 and Fig.11 The superior performance of the proposed pitch extrapolation is shown.

[0296] in, Fig.10 A pitch lag diagram is shown, in which the pitch lag is reconstructed using the concepts of the prior art. Fig.11 A pitch lag diagram is shown, wherein the pitch lag is reconstructed according to an embodiment.

[0297] In particular, Fig.10 The performance of the prior art standards G.718 and G.729.1 is shown. Fig.11 The performance of the concepts provided by the embodiments is shown.

[0298] The horizontal axis indicates the number of subframes. The solid line 1010 shows the encoder pitch lag, which is embedded in the bitstream and which is lost in the area 1030 of the gray segment. The left vertical axis represents the pitch lag axis. The right vertical axis represents the pitch gain axis. The solid line 1010 shows the pitch lag, while the dotted lines 1021, 1022, 1023 show the pitch gain.

[0299] A frame loss is indicated by a grey rectangle 1030. Because the frame loss occurs in the region 1030 of the grey segment, information about the pitch lag and pitch gain in this region is not available at the decoder side and must be reconstructed.

[0300] Fig.10 In FIG. 1 , the pitch lag hidden using the G.718 standard is shown by the dotted line portion 1011. The pitch lag hidden using the G.729.1 standard is shown by the solid line portion 1012. It can be clearly seen that using the provided pitch prediction ( Fig.11 , the solid line portion 1013) is basically consistent with the lost encoder tone lag, and is therefore superior to G.718 and G.729.1 technologies.

[0301] Below, an embodiment using weighting depending on the past time is described with reference to formulas (23a)-(24b).

[0302] To overcome the shortcomings of the prior art, some embodiments apply time weighting to the pitch lag before performing pitch prediction. Applying time weighting can be achieved by minimizing this error function:

[0303]

[0304] where time passed (i) represents the inverse of the amount of time that has elapsed after the pitch lag is correctly received, and P(i) holds the corresponding pitch lag.

[0305] Some embodiments, for example, may apply high weights to more recent lags and lower weights to lags received more recently.

[0306] According to some embodiments, a and b may then be derived using formula (21a).

[0307] To obtain the first predicted subframe, some embodiments, for example, may perform prediction based on the last five subframes P(0), ..., P(4). For example, the predicted pitch value P(5) may be obtained according to the following formula:

[0308] P(5)=a+5·b (23b)

[0309] For example, if

[0310] time passed =[1 / 5 1 / 4 1 / 3 1 / 2 1]

[0311] (weighted by the time of the subframe delays), which results in:

[0312]

[0313]

[0314] Next, embodiments providing pulse resynchronization are described.

[0315] Fig. 2a shows an apparatus for reconstructing a frame including a speech signal as a reconstructed frame of an embodiment. The reconstructed frame is associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as one or more available pitch periods.

[0316] The apparatus comprises a determination unit 210 for determining a sample quantity difference The sample size difference Indicates a difference between a number of samples of one of the one or more available pitch periods and a number of samples of a first pitch period to be reconstructed.

[0317] Furthermore, the apparatus comprises a frame reconstructor (220) for reconstructing the frame by and reconstructing a reconstructed frame based on samples of one of the one or more available pitch periods to reconstruct a first pitch period to be reconstructed as a first reconstructed pitch period.

[0318] The frame reconstructor (220) is used to reconstruct a reconstructed frame so that the reconstructed frame completely or partially includes a first reconstructed pitch period, so that the reconstructed frame completely or partially includes a second reconstructed pitch period, and so that the number of samples of the first reconstructed pitch period is different from the number of samples of the second reconstructed pitch period.

[0319] Reconstructing the pitch period is performed by reconstructing some or all of the samples of the pitch period to be reconstructed. If the pitch period to be reconstructed consists entirely of lost frames, then all samples of the pitch period, for example, must be reconstructed. If the pitch period to be reconstructed consists only partially of lost frames, and if some samples of the pitch period are available, for example, because they consist of another frame, then, for example, reconstructing only the samples of the pitch period comprised by the lost frame is sufficient for reconstructing the pitch period.

[0320] Fig. 2b illustrates the functioning of the apparatus of Fig. 2a. In particular, Fig. 2b illustrates a speech signal 222 comprising pulses 211, 212, 213, 214, 215, 216, 217.

[0321] The first part of the speech signal 222 consists of frame n-1. The second part of the speech signal 222 consists of frame n. The third part of the speech signal 222 consists of frame n+1.

[0322] In Figure 2b, frame n-1 precedes frame n and frame n+1 follows frame n. This means that frame n-1 includes a portion of the speech signal that occurs earlier in time than the portion of the speech signal in frame n, and frame n+1 includes a portion of the speech signal that occurs later in time than the portion of the speech signal in frame n.

[0323] In the example of FIG. 2b, it is assumed that frame n is lost or destroyed, and therefore, only the frame preceding frame n ("previous frame") and the frame following frame n ("following frame") are available ("available frames").

[0324] For example, a pitch period may be defined as follows: the pitch period begins at one of pulses 211, 212, 213, etc. and ends at the immediately subsequent pulse in the speech signal. For example, pulses 211 and 212 define pitch period 201. Pulses 212 and 213 define pitch period 202. Pulses 213 and 214 define pitch period 203, and so on.

[0325] Other definitions of the pitch period known to those skilled in the art are also contemplated, for example using other start and end points of the pitch period.

[0326] In the example of Figure 2b, frame n is unavailable or damaged at the receiver. Therefore, the receiver knows pulses 211 and 212 of frame n-1 and pitch period 201. In addition, the receiver knows pulses 216 and 217 of frame n+1 and pitch period 206. However, frame n including pulses 213, 214 and 215, which completely includes pitch periods 203 and 204 and which partially includes pitch periods 202 and 205, must be reconstructed.

[0327] According to some embodiments, frame n may be reconstructed from samples of at least one pitch period ("available pitch period") of an available frame (e.g., previous frame n-1 or subsequent frame n+1). For example, samples of pitch period 201 of frame n-1 may be periodically repeated to reconstruct samples of a lost or damaged frame. By periodically repeating the samples of the pitch period, the pitch period itself is replicated, e.g., if the pitch period is c, then

[0328] Sample(x+i·c)=Sample(x); i is an integer.

[0329] In an embodiment, samples from the end portion of frame n-1 are copied. The length of the portion of the copied frame n-1 is equal to (or almost equal to) the length of the pitch period 201. However, samples from 201 and 202 are used for the copying. This may require special consideration when the frame n-1 has only one pulse.

[0330] In some embodiments, the copied sample is modified.

[0331] The present invention is also based on the discovery that, by periodically repeating the replication of samples of pitch periods, when the size of the pitch periods (pitch periods 202, 203, 204 and 205) comprised by the (completely or partially) lost frame (n) is different from the size of the available pitch period being replicated (here: pitch period 201), the pulses 213, 214, 215 of the lost frame n are moved to incorrect positions.

[0332] For example, in Figure 2b, the difference between the pitch period 201 and the pitch period 202 is indicated by Δ1, the difference between the pitch period 201 and the pitch period 203 is indicated by Δ2, the difference between the pitch period 201 and the pitch period 204 is indicated by Δ3, and the difference between the pitch period 201 and the pitch period 205 is indicated by Δ4.

[0333] In FIG. 2b , it can be seen that the pitch period 201 of frame n-1 is significantly larger than the pitch period 206. In addition, each of the pitch periods 202, 203, 204, and 205 (partially or completely) included in frame n is smaller than the pitch period 201 and larger than the pitch period 206. In addition, the pitch period closer to the large pitch period 201 (e.g., pitch period 202) is larger than the pitch period closer to the small pitch period 206 (e.g., pitch period 205).

[0334] Based on these findings of the present invention, according to an embodiment, the frame reconstructor 220 is configured to reconstruct a reconstructed frame such that the number of samples of a first reconstructed pitch period is different from the number of samples of a second reconstructed pitch period partially or completely included by the reconstructed frame.

[0335] For example, according to some embodiments, reconstruction of a frame depends on a sample number difference indicating the difference between the number of samples of one of one or more available pitch periods (e.g., pitch period 201) and the number of samples of a first pitch period to be reconstructed (e.g., pitch periods 202, 203, 204, 205).

[0336] For example, according to an embodiment, samples of the pitch period 201, for example, may be replicated periodically.

[0337] The sample number difference then indicates how many samples should be deleted from or added to the periodically repeating copy corresponding to the first pitch period to be reconstructed.

[0338] In FIG. 2 b, each sample number indicates how many samples should be deleted from the periodically repeated copy. However, in other examples, the sample number may indicate how many samples should be added to the periodically repeated copy. For example, in some embodiments, samples may be added by adding samples with zero amplitude to the corresponding pitch period. In other embodiments, samples may be added to the pitch period by copying other samples of the pitch period, for example, by copying samples adjacent to the position of the sample to be added.

[0339] Although an embodiment in which samples of the pitch period of a frame before a lost frame or a damaged frame have been periodically repeated and copied has been described above, in other embodiments, samples of the pitch period of a frame after a lost frame or a damaged frame are periodically repeated and copied to reconstruct the lost frame. The same principles as above and as described below similarly apply.

[0340] This sample number difference can be determined for each pitch period to be reconstructed. The sample number difference for each pitch period then indicates how many samples should be deleted from the periodically repeated copy corresponding to the corresponding pitch period to be reconstructed, or how many samples should be added to the periodically repeated copy corresponding to the corresponding pitch period to be reconstructed.

[0341] According to an embodiment, the determination unit 210, for example, may be configured to determine a sample number difference for each of the plurality of pitch periods to be reconstructed, such that the sample number difference for each of the pitch periods indicates a difference between the sample number of the one of the one or more available pitch periods and the sample number of the pitch period to be reconstructed. The frame reconstructor 220, for example, may be configured to reconstruct each of the plurality of pitch periods to be reconstructed based on the sample number difference for the pitch period to be reconstructed and based on the samples of the one of the one or more available pitch periods, thereby reconstructing a reconstructed frame.

[0342] In an embodiment, the frame reconstructor 220, for example, may be configured to generate an intermediate frame according to the one of the one or more available pitch periods. The frame reconstructor 220, for example, may be configured to modify the intermediate frame to obtain the reconstructed frame.

[0343] According to an embodiment, the determination unit 210, for example, may be used to determine a frame difference value (d; s) indicating how many samples are to be removed from the intermediate frame or how many samples are to be added to the intermediate frame. In addition, the frame reconstructor 220, for example, may be used to remove the first sample from the intermediate frame to obtain the reconstructed frame when the frame difference value indicates that the first sample should be removed from the frame. In addition, the frame reconstructor 220, for example, may be used to add the second sample to the intermediate frame to obtain the reconstructed frame when the frame difference value (d; s) indicates that the second sample should be added to the frame.

[0344] In an embodiment, the frame reconstructor 220, for example, may be configured to remove the first sample from the intermediate frame when the frame difference value indicates that the first sample should be removed from the frame, whereby the number of the first sample removed from the intermediate frame is indicated by the frame difference value. In addition, the frame reconstructor 220, for example, may be configured to add the second sample to the intermediate frame when the frame difference value indicates that the second sample should be added to the frame, whereby the number of the second sample added to the intermediate frame is indicated by the frame difference value.

[0345] According to an embodiment, the determination unit 210, for example, may be configured to determine the frame difference s, and thus the following formula applies:

[0346]

[0347] Where L indicates the number of samples of the reconstructed frame, Where M indicates the number of subframes of the reconstructed frame, Where T rIndicates a rounded pitch period length of the one of the one or more available pitch periods, and wherein p[i] indicates the pitch period length of the reconstructed pitch period of the i-th subframe of the reconstructed frame.

[0348] In an embodiment, the frame reconstructor 220, for example, may be adapted to generate an intermediate frame in dependence on said one of the one or more available pitch periods. Furthermore, the frame reconstructor 220, for example, may be adapted to generate an intermediate frame so that the intermediate frame comprises a first partial intermediate pitch period, one or more other intermediate pitch periods, and a second partial intermediate pitch period. Furthermore, the first partial intermediate pitch period, for example, depends on one or more of the samples of said one of the one or more available pitch periods, wherein each of said one or more other intermediate pitch periods depends on all samples of said one of the one or more available pitch periods, and wherein the second partial intermediate pitch period depends on one or more of the samples of said one of the one or more available pitch periods. Furthermore, the determination unit 210, for example, may be adapted to determine a starting partial delta indicating how many samples are to be removed from or added to the first partial intermediate pitch period, and wherein the frame reconstructor 220 is adapted to remove one or more first samples from the first partial intermediate pitch period, or to add one or more first samples to the first partial intermediate pitch period, in dependence on the starting partial delta. Furthermore, the determination unit 210, for example, may be configured to determine a pitch period difference for each of the other intermediate pitch periods, the pitch period difference indicating how many samples are to be removed from or added to the one of the other intermediate pitch periods. Furthermore, the frame reconstructor 220, for example, may be configured to remove one or more second samples from the one of the other intermediate pitch periods, or to add one or more second samples to the one of the other intermediate pitch periods, based on the pitch period difference. Furthermore, the determination unit 210, for example, may be configured to determine an ending portion difference indicating how many samples are to be removed from or added to the second portion of the intermediate pitch period, and wherein the frame reconstructor 220 is configured to remove one or more third samples from the second portion of the intermediate pitch period, or to add one or more third samples to the second portion of the intermediate pitch period, based on the ending portion difference.

[0349] According to an embodiment, the frame reconstructor 220, for example, may be used to generate an intermediate frame based on the one of the one or more available pitch periods. In addition, the determination unit 210, for example, may be adapted to determine one or more low-energy signal portions of the speech signal included by the intermediate frame, wherein each of the one or more low-energy signal portions is a first signal portion of the speech signal within the intermediate frame, wherein the energy of the speech signal is lower than the energy in the second signal portion of the speech signal included by the intermediate frame. In addition, the frame reconstructor 220, for example, may be used to remove one or more samples from at least one of the one or more low-energy signal portions of the speech signal, or add one or more samples to at least one of the one or more low-energy signal portions of the speech signal to obtain a reconstructed frame.

[0350] In a particular embodiment, the frame reconstructor 220, for example, may be configured to generate an intermediate frame such that the intermediate frame includes one or more reconstructed pitch periods such that each of the one or more reconstructed pitch periods depends on the one of the one or more available pitch periods. Furthermore, the determination unit 210, for example, may be configured to determine the number of samples that should be removed from each of the one or more reconstructed pitch periods. Furthermore, the determination unit 210, for example, may be configured to determine each of the one or more low-energy signal portions such that for each of the one or more low-energy signal portions, the number of samples of the low-energy signal portion depends on the number of samples that should be removed from one of the one or more reconstructed pitch periods, wherein the low-energy signal portion is located within the one of the one or more reconstructed pitch periods.

[0351] In an embodiment, the determining unit 210, for example, may be configured to determine the position of one or more pulses of the speech signal to be reconstructed as a reconstructed frame. In addition, the frame reconstructor 220, for example, may be configured to reconstruct the reconstructed frame according to the position of one or more pulses of the speech signal.

[0352] According to an embodiment, the determination unit 210, for example, may be used to determine the positions of two or more pulses of the speech signal to be reconstructed as a frame of the reconstructed frame, wherein T[0] is the position of one of the two or more pulses of the speech signal to be reconstructed as a frame of the reconstructed frame, and wherein the determination unit 210 is used to determine the positions (T[i]) of the other pulses of the two or more pulses of the speech signal according to the following formula:

[0353] T[i]=T[0]+iT r

[0354] Where T r Indicates the rounded length of the one of one or more available pitch periods, and wherein i is an integer.

[0355] According to an embodiment, the determining unit 210 may, for example, be configured to determine the index k of the last pulse of the speech signal of the frame to be reconstructed as the reconstructed frame, such that

[0356]

[0357] wherein L indicates the number of samples of the reconstructed frame, wherein s indicates the frame difference, wherein T[0] indicates the position of a pulse of the speech signal of the frame to be reconstructed as the reconstructed frame, which is different from the last pulse of the speech signal, and wherein T r Indicates the rounded length of the one of one or more available pitch periods.

[0358] In an embodiment, the determining unit 210, for example, may be configured to reconstruct a frame to be reconstructed as a reconstructed frame by determining a parameter δ, wherein the parameter δ is defined according to the following formula:

[0359]

[0360] The frame to be reconstructed as a reconstructed frame includes M subframes, where T p indicates the length of the one of the one or more available pitch periods, and wherein T ext Indicates the length of one of the to-be-reconstructed pitch periods of a frame to be reconstructed as a reconstructed frame.

[0361] According to an embodiment, the determining unit 210, for example, may be configured to determine the rounded length T of the one of the one or more available pitch periods based on the following formula: r And reconstruct the reconstructed frame:

[0362]

[0363] Where T p Indicating a length of the one of the one or more available pitch periods.

[0364] In an embodiment, the determining unit 210, for example, may be configured to reconstruct the reconstructed frame by applying the following formula:

[0365]

[0366] Where T p indicates the length of the one of the one or more available pitch periods, where T rindicating a rounded length of said one of one or more available pitch periods, wherein a frame to be reconstructed as a reconstructed frame comprises M subframes, wherein the frame to be reconstructed as a reconstructed frame comprises L samples, and wherein δ is a real number indicating the difference between the number of samples of said one of the one or more available pitch periods and the number of samples of one of the one or more pitch periods to be reconstructed.

[0367] Next, examples are described in more detail.

[0368] Below, the first set of pulse resynchronization embodiments is described with reference to equations (25)-(63).

[0369] In these embodiments, if there is no pitch change, the last pitch lag is used without rounding, preserving the fractional part. The periodic part is constructed using non-integer pitches and interpolation (see, for example, [MTTA90]). This will reduce the frequency shift of harmonics compared to using rounded pitch lags, and thus significantly improve the concealment of tones or voiced signals with constant pitch.

[0370] pass Figure 8 and Fig. 9 The advantages are shown for hiding a signal representing a pitch pipe with frame loss using rounded pitch lag and unrounded fractional pitch lag, respectively. Here, Figure 8 shows a time-frequency representation of a resynchronized speech signal using a rounded pitch lag. In contrast, Fig. 9 A time-frequency representation of a resynchronized speech signal using an unrounded pitch lag with a fractional part is shown.

[0371] There will be increased computational complexity when using fractional parts of the pitch. This should not affect the worst case complexity since no glottal pulse resynchronization is required.

[0372] If no pitch change is predicted, then the processing described below is not required.

[0373] If pitch changes are predicted, the embodiments described with reference to equations (25)-(63) provide a concept for determining d, which is the pitch period (T c ) and the difference between the total number of samples within the pitch period p[i] with evolving pitch.

[0374] Next, T is defined as in formula (15a) c :T c =Rounded (last pitch).

[0375] According to an embodiment, the difference d may be determined using a faster and more accurate algorithm (a fast algorithm of a method for determining d), as described below.

[0376] For example, such an algorithm may be based on the following principle:

[0377] - In each subframe i: for (length T c ) for each tone cycle, T c -p[i] samples (or if T c -p[i]<0, increase T c -p[i] samples).

[0378] - Each subframe has A tone cycle.

[0379] - Therefore, for each subframe, samples should be removed.

[0380] According to some embodiments, no rounding is performed and fractional pitch is used. Then:

[0381] -p[i]=T c +(i+1)δ.

[0382] - Therefore, for each subframe i, if δ < 0, samples should be removed (or added if δ>0).

[0383] -therefore, (where M is the number of subframes in a frame).

[0384] According to some other embodiments, rounding is performed. For integer tones (M is the number of subframes in a frame), d is defined as follows:

[0385]

[0386] According to an embodiment, an algorithm for calculating d is provided accordingly:

[0387]

[0388] In another embodiment, the last line of the algorithm is replaced by the following:

[0389] d=(short)floor(L_frame-ftmp*(float)L_subfr / T_c+0.5);

[0390] According to an embodiment, the last pulse T[n] is found according to the following formula:

[0391] n=i|T[0]+iT c <L_frame∧T[0]+(i+1)T c ≥L_frame (26)

[0392] According to the embodiment, a formula for calculating N is used. This formula is obtained according to formula (26):

[0393]

[0394] And then, the last pulse has index N-1.

[0395] According to this formula, for Figure 4 as well as Figure 5 The example shown calculates N.

[0396] In the following, the concept of not explicitly searching for the last pulse but taking into account the pulse position is described. This concept does not require N (the index of the last pulse in the periodic part).

[0397] The actual last pulse position in the build-up period portion of the excitation (T[k]) determines the number k of full pitch periods in which samples are removed (or added).

[0398] Fig.12 The position of the last pulse T[2] before d samples are removed is shown. With respect to the embodiments described with reference to equations (25)-(63), reference numeral 1210 indicates d.

[0399] exist Fig.12 In the example of , the index k of the last pulse is 2 and there are 2 complete pitch periods from which samples are to be removed.

[0400] After removing d samples from a signal of length L_frame+d, there are no samples from the original signal outside L_frame+d samples. Therefore T[k] is within L_frame+d samples, and k is therefore determined by formula (28):

[0401] k=i|T[i]<L frame +d≤T[i+1] (28)

[0402] From formula (17) and formula (28), we get:

[0403] T[0]+kT c <L frame +d≤T[0]+(k+1)T c (29)

[0404] Right now,

[0405]

[0406] From formula (30), we get:

[0407]

[0408] In a codec (eg using frames of at least 20 ms and in which the lowest fundamental frequency of speech is, eg, at least 40 Hz), at least one pulse is present in most cases in a concealed frame other than UNVOICED.

[0409] Next, the case with at least two pulses (k ≥ 1) is described with reference to equations (32)-(46).

[0410] Assume that, for each complete ith tone period between pulses, Δ i samples, where Δ i is defined as:

[0411] Δ i =Δ+(i-1)a, 1≤i≤k, (32)

[0412] where a is the unknown variable that needs to be expressed in terms of the known variables.

[0413] Assume that Δ0 samples should be removed before the first pulse, where Δ0 is defined as:

[0414]

[0415] Assume that Δ should be removed after the last pulse k+1 samples, where Δ k+1 is defined as:

[0416]

[0417] The last two assumptions are consistent with formula (32) which takes into account the lengths of the first and last pitch periods of the parts.

[0418] Δ i Each of the values ​​is the sample size difference. Moreover, Δ0 is the sample size difference. In addition, Δ k+1 is the sample size difference.

[0419] Fig.13 Show Fig.12 The speech signal is shown in FIG. 1 , and Δ0 to Δ3 are additionally described. Fig.13 In the example of , the number of samples to be removed in each pitch period is schematically represented, where k = 2. With respect to the embodiments described with reference to formulas (25)-(63), reference numeral 1210 indicates d.

[0420] Therefore, the total number of samples to be removed d is related to Δ according to the following formula i Related:

[0421]

[0422] From formulas (32)-(35), we can get d as:

[0423]

[0424] Formula (36) is equivalent to:

[0425]

[0426] Assume that the last complete pitch period in the hidden frame has a length of p[M-1], that is:

[0427] Δ k =T c -p[M-1] (38)

[0428] From formula (32) and formula (38), we get:

[0429] Δ=T c -p[M-1]-(k-1)a (39)

[0430] In addition, from formula (37) and formula (39), we get:

[0431]

[0432] Formula (40) is equivalent to:

[0433]

[0434] From formula (17) and formula (41), we get:

[0435]

[0436] Formula (42) is equivalent to:

[0437]

[0438] In addition, from formula (43), we get:

[0439]

[0440] Formula (44) is equivalent to:

[0441]

[0442] In addition, formula (45) is equivalent to:

[0443]

[0444] According to an embodiment, it is now calculated based on formulas (32)-(34), (39) and (46) how many samples are to be removed or added before the first pulse, and / or between pulses and / or after the last pulse.

[0445] In an embodiment, samples are removed or added in the minimum energy region.

[0446] According to an embodiment, for example, the number of samples to be removed is rounded as follows:

[0447]

[0448]

[0449]

[0450] Next, the case with one pulse (k=0) is explained with reference to formulas (47)-(55).

[0451] If there is only one pulse in the hidden frame, Δ0 samples are removed before the pulse:

[0452]

[0453] Where Δ and a are unknown variables that need to be expressed in terms of known variables. After the pulse, Δ1 samples will be removed, where:

[0454]

[0455] So, giving the total number of samples that will be removed:

[0456] d=Δ0+Δ1 (49)

[0457] From formulas (47)-(49), we get:

[0458]

[0459] Formula (50) is equivalent to:

[0460] dT c =Δ(L+d)-aT[0] (51)

[0461] Assume that the ratio of the pitch period before the pulse to the pitch period after the pulse is the same as the ratio between the pitch lag in the last subframe to the pitch lag in the first subframe in the previously received frame:

[0462]

[0463] From formula (52), we get:

[0464]

[0465] In addition, from formula (51) and formula (53), we get:

[0466]

[0467] Formula (54) is equivalent to:

[0468]

[0469] Before the pulse, there is samples will be removed or added to the minimum energy region, after the pulse, samples will be removed or added to the minimum energy region.

[0470] Below, the simplified concept of pulse-free (position) search according to an embodiment is explained with reference to formulas (56)-(63).

[0471] t[i] indicates the length of the ith pitch period. After removing d samples from the signal, there are k full pitch periods and 1 partial (to full) pitch period.

[0472] therefore:

[0473]

[0474] Since after removing some samples from the length T c The pitch period of length t[i] is obtained, and since the total number of removed samples is d, we get:

[0475] kT c <L+d≤(k+1)T c (57)

[0476] Then we get:

[0477]

[0478] And, we get:

[0479]

[0480] According to an embodiment, a linear change of pitch lag may be assumed as:

[0481] t[i]=T c -(i+1)Δ, 0≤i≤k

[0482] In an embodiment, (k+1)Δ samples are removed in the kth pitch period.

[0483] According to an embodiment, in the portion of the k-th pitch period, the samples remaining in the frame after removal are samples were removed.

[0484] Therefore, the total number of removed samples is:

[0485]

[0486] Formula (60) is equivalent to:

[0487]

[0488] Moreover, formula (61) is equivalent to:

[0489]

[0490] In addition, formula (62) is equivalent to:

[0491]

[0492] According to an embodiment, (i+1)Δ samples are removed at the position of minimum energy. The position of the pulse does not need to be known, since the search for the position of minimum energy is done in a circular buffer that holds one pitch period.

[0493] If the minimum energy position is after the first pulse and if the samples before the first pulse are not removed, a pitch lag may occur as (T c +Δ), T c , T c , (T c -Δ), (T c -2Δ) (2 pitch periods in the last received frame and 3 pitch periods in the hidden frame). Therefore, there will be a discontinuity. A similar discontinuity may occur after the last pulse, but it occurs at a different time than the discontinuity before the first pulse.

[0494] On the other hand, if the pulse is closer to the beginning of the hidden frame, the minimum energy region will be more likely to occur after the first pulse. If the first pulse is closer to the beginning of the hidden frame, it will be more likely that the last pitch period in the last received frame is greater than T c In order to reduce the likelihood of pitch change interruptions, weighting should be used to provide the advantage of having the smallest area closer to the beginning or end of the pitch period.

[0495] According to an embodiment, an implementation of the provided concept is described, wherein one or more or all of the following method steps are performed:

[0496] 1. In temporary buffer B, store the low-pass filtered T from the end of the last received frame csamples, the minimum energy region is searched in parallel. When searching for the minimum energy region, the temporary buffer is considered a circular buffer. (This may mean that the minimum energy region may consist of some samples from the beginning and some samples from the end of the pitch period.) The minimum energy region, for example, may be of length For example, weighting may be used, for example, to provide an advantage that the minimum region is closer to the start of the pitch period.

[0497] 2. Copy samples from temporary buffer B to the frame, skipping those in the minimum energy region. samples. Therefore, a pitch period of length t[0] is generated. Set

[0498] 3. For the i-th pitch period (0<i<k), copy samples from the (i-1)-th pitch period, skipping the samples in the minimum energy region. Samples. Set Repeat this step k-1 times.

[0499] 4. For the kth pitch period, use a weighted search for a new minimum region in the (k-1)th pitch period, providing the advantage that the minimum region is closer to the end of the pitch period. Then copy samples from the (k-1)th pitch period, skipping the minimum energy region at the end of the pitch period. samples.

[0500] If additional samples are needed, an equivalent procedure can be used by considering d < 0 and Δ < 0, adding a total of |d| samples, and adding (k+1) |Δ| samples in the kth cycle at the minimum energy position.

[0501] As described above with respect to the "Fast Algorithm for Determining d", fractional pitch can be used at the sub-frame level to derive d since an approximate pitch period length is used anyway.

[0502] Next, the second group of pulse resynchronization embodiments is described with reference to equations (64)-(113). These embodiments of the first group adopt the definition of equation (15b),

[0503]

[0504] Among them, the final tone cycle length is T p , and the length of the copied segment is T r .

[0505] If some parameters used by the second set of pulse resynchronization embodiments are not defined below, embodiments of the present invention may adopt the definitions of these parameters provided with respect to the first set of pulse resynchronization embodiments defined above (see equations (25)-(63)).

[0506] Some of the equations (64)-(113) of the second set of pulse resynchronization embodiments may redefine some parameters that have been previously used with respect to the first set of pulse resynchronization embodiments. In this case, the provided redefined definitions apply to the second pulse resynchronization embodiments.

[0507] As described above, according to some embodiments, for example, a periodic portion may be constructed for a frame and an additional subframe, wherein the frame length is represented by L=L frame .

[0508] For example, there are M subframes in a frame, and the subframe length is

[0509] As previously stated, T[0] is the position of the first largest pulse in the build-up period portion of the excitation. The positions of the other pulses are given by:

[0510] T[i]=T[0]+iT r .

[0511] According to an embodiment, based on the construction of the periodic portion of the excitation, for example, after the construction of the periodic portion of the excitation, glottal pulse resynchronization is performed to correct the difference between the estimated target position (P) of the last pulse in the lost frame and its actual position (T[k]) in the constructed periodic portion of the excitation.

[0512] For example, the estimated target position (P) of the last pulse in the lost frame can be determined indirectly by estimating the pitch lag evolution. For example, the pitch lag evolution is obtained by extrapolating the pitch lag of the last seven subframes before the lost frame. The evolving pitch lag in each subframe is:

[0513] p[i]=T p +(i+1)δ, 0≤i<M (64)

[0514] in

[0515]

[0516] And, T ext is the extrapolated pitch and i is the subframe index. For example, the pitch extrapolation may be performed using a weighted linear fit or a method from G.718 or a method from G.729.1 or any other method for pitch interpolation (e.g., taking into account one or more pitches of future frames). The pitch extrapolation may also be nonlinear. In an embodiment, T may be determined as above ext T is determined in the same way as t ext .

[0517] The total number of samples within a pitch period with evolving pitch (p[i]) and the total number of samples within a pitch period with constant pitch (T p ) and the frame length difference between them is denoted as s.

[0518] According to an embodiment, if T ext >T p , then s samples should be added to the frame if T ext <T p , then -s samples should be removed from the frame. After adding or removing |s| samples, the last pulse in the hidden frame will be at the estimated target position (P).

[0519] If T ext =T p , no sample addition or removal within the frame is required.

[0520] According to some embodiments, glottal pulse resynchronization is accomplished by adding or removing samples in the minimum energy region of all pitch cycles.

[0521] Below, the calculation parameter s according to the embodiment is explained with reference to formulas (66)-(69).

[0522] According to some embodiments, for example, the difference s may be calculated based on the following principle:

[0523] - In each subframe i, (if p[i] - T r >0), for each pitch period (length T r ), p[i]-T should be increased r samples; (or, if p[i] - T r <0, p[i]-T should be removed r samples).

[0524] - Each subframe has A tone cycle.

[0525] -Therefore, in the i-th subframe, the samples.

[0526] Therefore, according to the embodiment, in accordance with formula (64), for example, s can be calculated according to formula (66):

[0527]

[0528]

[0529] Formula (66) is equivalent to:

[0530]

[0531] Where formula (67) is equivalent to:

[0532]

[0533] And formula (68) is equivalent to:

[0534]

[0535] Note that if T ext >T p , then s is positive and the pseudo-copy should be increased if T ext <T p , then s is negative and the sample should be removed. Therefore, the number of samples removed or added can be expressed as |s|.

[0536] The calculation of the index of the last pulse according to the embodiment is described below with reference to formulas (70)-(73).

[0537] The actual last pulse position (T[k]) in the build-up period portion of the excitation determines the number k of complete pitch periods in which samples are removed (or added).

[0538] Fig.12 Shown is the speech signal before samples are removed.

[0539] exist Fig.12 In the example shown, the index k of the last pulse is 2 and there are two complete pitch periods from which samples should be removed. With respect to the embodiments described with reference to equations (64)-(113), reference numeral 1210 indicates |s|.

[0540] After removing |s| samples from a signal of length Ls, where L=L_frame, or after adding |s| samples to a signal of length Ls, there are no samples from the original signal beyond Ls samples. Note that s is positive if samples are added and negative if samples are removed. Thus Ls<L if samples are added and Ls>L if samples are removed. Thus T[k] must be within Ls samples, so k is determined by:

[0541] k=i|T[i]<Ls≤T[i+1] (70)

[0542] From formula (15b) and formula (70), we get:

[0543] T[0]+kT r <Ls≤T[0]+(k+1)T r (71)

[0544] Right now:

[0545]

[0546] According to an embodiment, for example, k may be determined based on formula (72):

[0547]

[0548] For example, in a codec using frames of, for example, at least 20 milliseconds, and using a minimum fundamental frequency of speech of at least 40 Hz, in most cases at least one pulse is present in a concealed frame other than silence.

[0549] Below, referring to formulas (74)-(99), the calculation of the number of samples removed in the minimum area according to an embodiment is explained.

[0550] For example, it may be assumed that Δ i samples, where Δ i is defined as follows:

[0551] Δ i =Δ+(i-1)a, 1≤i≤k (74)

[0552] And where a is an unknown variable, for example, it can be represented by known variables.

[0553] Furthermore, for example, it can be assumed that before the first pulse, samples, of which is defined as:

[0554]

[0555] Furthermore, for example, it can be assumed that after the last pulse, samples, of which is defined as:

[0556]

[0557] The last two assumptions are consistent with formula (74) which takes into account the lengths of the first and last pitch periods of the sections.

[0558] Fig.13 The number of samples to be removed (or added) in each pitch period is schematically represented in the example of , where k=2. Fig.13 A schematic representation of the samples removed in each pitch period is shown. With respect to the embodiments described with reference to equations (64)-(113), reference numeral 1210 indicates |s|.

[0559] The total number of samples to be removed (or added) s is calculated according to the following formula with Δ i Related:

[0560]

[0561] From formulas (74)-(77), we get:

[0562]

[0563] Formula (78) is equivalent to:

[0564]

[0565] In addition, formula (79) is equivalent to:

[0566]

[0567] In addition, formula (80) is equivalent to:

[0568]

[0569] In addition, considering formula (16b), formula (81) is equivalent to:

[0570]

[0571] According to an embodiment, it may be assumed that the number of samples removed (or added) in a full pitch period after the last pulse is given by:

[0572] Δ k+1 =|T r -p[M-1]|=|T r -T ext | (83)

[0573] From formula (74) and formula (83), we get:

[0574] Δ=|T r -T ext |-ka (84)

[0575] From formula (82) and formula (84), we get:

[0576]

[0577] Formula (85) is equivalent to:

[0578]

[0579] In addition, formula (86) is equivalent to:

[0580]

[0581] In addition, formula (87) is equivalent to:

[0582]

[0583] From formula (16b) and formula (88), we get:

[0584]

[0585] Formula (89) is equivalent to:

[0586]

[0587] In addition, formula (90) is equivalent to:

[0588]

[0589] In addition, formula (91) is equivalent to:

[0590]

[0591] In addition, formula (92) is equivalent to:

[0592]

[0593] From formula (93), we get:

[0594]

[0595] Therefore, for example, based on formula (94), according to an embodiment:

[0596] - calculate how many samples to remove and / or add before the first pulse, and / or

[0597] - calculate how many samples to remove and / or add between pulses, and / or

[0598] - Calculate how many samples will be removed and / or added after the last pulse.

[0599] According to some embodiments, for example, samples may be removed or added in the minimum energy region.

[0600] From formula (85) and formula (94), we get:

[0601]

[0602] Formula (95) is equivalent to:

[0603]

[0604] In addition, from formula (84) and formula (94), we get:

[0605] Δ i=Δ+(i-1)a=|T r -T ext |-ka+(i-1)a, 1≤i≤k (97)

[0606] Formula (97) is equivalent to:

[0607] Δ i =|T r -T ext |-(k+1-i)a, 1≤i≤k (98)

[0608] According to an embodiment, the number of samples removed after the last pulse may be calculated based on formula (97):

[0609]

[0610] It should be noted that, according to the embodiment, Δ i and is positive and the sign of s determines whether samples will be removed or added.

[0611] For complexity reasons, in some embodiments, it is required to add or remove an integer number of samples and therefore, in such embodiments, Δ i and For example, it may be rounded. In other embodiments, alternatively or additionally, other concepts such as using waveform interpolation may be used to avoid rounding, but with increased complexity.

[0612] Next, an algorithm for pulse resynchronization according to an embodiment is described with reference to equations (100)-(113).

[0613] According to an embodiment, the input parameters of this algorithm may be, for example:

[0614] L - frame length

[0615] M - number of subframes

[0616] T p -The length of the pitch period at the end of the last received frame

[0617] T ext - The length of the pitch period at the end of the concealed frame

[0618] src_exc - the input excitation signal generated by copying the low-pass filtered last pitch period of the excitation signal from the end of the last received frame, as described above

[0619] dst_exc - For pulse resynchronization, generates the output excitation signal from src_exc using the algorithm described here

[0620] Depending on the embodiment, this algorithm may include one or more or all of the following steps:

[0621] - Based on formula (65), calculate the pitch change per subframe:

[0622]

[0623] - Based on formula (15b), calculate the rounded starting pitch:

[0624]

[0625] - Based on formula (69), calculate the number of samples to be added (or removed if negative):

[0626]

[0627] - Find the first T in the build cycle part of the excitation src_exc r The position T[0] of the first maximum pulse in samples.

[0628] - Based on formula (73), the index of the last pulse in the resynchronization frame dst_exc is obtained:

[0629]

[0630] - Based on formula (94), calculate a - the increment of samples to be added or removed between consecutive cycles:

[0631]

[0632] - Based on formula (96), calculate the number of samples to be added or removed before the first pulse:

[0633]

[0634] - Round down the number of samples added or removed before the first pulse and save the fractional part in memory:

[0635]

[0636]

[0637] - Based on formula (98), for each region between 2 pulses, calculate the number of samples added or removed:

[0638] Δ i =|T r -T ext|-(k+1-i)a, 1≤i≤k (108)

[0639] -Round down the number of samples added or removed between 2 pulses, taking into account the fractional part left over from the previous rounding:

[0640]

[0641] F=Δ i -Δ′ i (110)

[0642] - If for some i, due to the increase in F, Δ′ i >Δ′ i-1 , then exchange Δ′ i and Δ′ i-1 The value of .

[0643] - Based on formula (99), calculate the number of samples to be added or removed after the last pulse:

[0644]

[0645] - Next, calculate the maximum number of samples that will be added or removed between minimum energy regions:

[0646]

[0647] - Find the position P of the minimum energy segment between the first two pulses in src_exc min [1], which has Δ max For each consecutive minimum energy segment between two pulses, the position is calculated by:

[0648] P min [i]=P min [1]+(i-1)T r , 1<i≤k (113)

[0649] - If P min [1]>T r , then use P min [0] = P min [1]-T r Calculate the position of the minimum energy segment before the first pulse in src_exc. Otherwise, find the position P of the minimum energy segment before the first pulse in src_exc. min [0], which has a length of Δ′0.

[0650] - If P min [1]+kT r <Ls, then use P min[k+1]=P min [1]+kT r Calculate the position of the minimum energy segment after the last pulse in src_exc. Otherwise, find the position P of the minimum energy segment after the last pulse in src_exc. min [k+1], which has Δ′ k+1 length.

[0651] - If there is only one pulse in the hidden excitation signal dst_exc, that is, if k is equal to 0, the limit for P min [1] to Ls, then P min [1] Indicates the position of the minimum energy segment after the last pulse in src_exc.

[0652] - If s>0, at position P min [i] (0≤i≤k+1) increases Δ′ i samples to signal src_exc and store it in dst_exc, otherwise if s < 0, at position P min [i] (0≤i≤k+1) remove Δ′ from the signal src_exc i samples and store them in dst_exc. There are k+2 regions where samples are removed or added.

[0653] Fig. 2c shows a system for reconstructing a frame including a speech signal according to an embodiment. The system comprises an apparatus 100 for determining an estimated pitch lag according to one of the above embodiments, and an apparatus 200 for reconstructing a frame, wherein the apparatus for reconstructing a frame is used to reconstruct the frame according to the estimated pitch lag. The estimated pitch lag is the pitch lag of the speech signal.

[0654] In an embodiment, the reconstructed frame, for example, may be associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as one or more available pitch periods. The apparatus 200 for reconstructing a frame, for example, may be an apparatus for reconstructing a frame according to one of the above-described embodiments.

[0655] Although some aspects have been described in the context of a system, it should be clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding device.

[0656] The decomposed signal of the present invention can be stored on a digital storage medium or can be transmitted on a transmission medium (eg, a wireless transmission medium or a wired transmission medium such as the Internet).

[0657] According to certain implementation needs, the embodiments of the present invention can be implemented by hardware or software. It can be implemented using a digital storage medium with electronically readable control signals thereon and cooperating with a programmable computer system to perform each method, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory.

[0658] Some embodiments according to the invention comprise a non-transitory data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0659] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code can, for example, be stored on a machine readable carrier.

[0660] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0661] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0662] A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein.

[0663] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may, for example, be configured to be transferred via a data communication connection, for example via the Internet.

[0664] A further embodiment comprises a processing element, for example a computer or a programmable logic device, configured or adapted to perform one of the methods described herein.

[0665] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0666] In some embodiments, a programmable logic device (e.g., a field programmable gate array) can be used to perform some or all functions of the method described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Typically, the method is preferably performed by a hardware device.

[0667] The embodiments described above are merely examples of the principles of the present invention. It should be understood that modifications and variations of the configurations and details described herein will be apparent to other persons skilled in the art. Therefore, the present invention is limited only by the scope of the appended claims and not by the specific details represented by the description and illustration of the embodiments herein.

[0668] References

[0669] [3GP09] 3GPP; Technical Specification Group Services and System Aspects, Extended Adaptive Multi-Rate FastBand (AMR-WB+) Codec, 3GPP TS 26.290, 3rd Generation Partnership Project, 2009.

[0670] [3GP12a], Adaptive Multi-Rate (AMR) Speech Codec; Error Concealment for Lost Frames (Release 11), 3GPP TS26.091, 3rd Generation Partnership Project, September 2012.

[0671] [3GP12b], Speech codec speech processing functions; Adaptive Multi-Rate Wideband (AMRWB) speech codec; Error concealment of errored or lost frames, 3GPP TS 26.191, 3rd Generation Partnership Project, September 2012.

[0672] [Gao] Yang Gao, Pitch prediction for packet loss concealment, European Patent 2002 427 B1.

[0673] [ITU03] ITU-T, Wideband coding of speech at around 16 kbit / s using adaptive multi-rate wideband (amr-wb), Recommendation ITU-T G.722.2, ITU Telecommunication Standardization Sector, June 2003.

[0674] [ITU06a], G.722 Annex III: A high-complexity algorithm for packet lossconcealment for G.722, ITU-T Recommendation, ITU-T, November 2006.

[0675] [ITU06b], G.729.1: G.729-based embedded variable bit-rate coder: An 8-32kbit / s scalable wideband coder bitstream interoperable with g.729, Recommendation ITU-TG.729.1, ITU Telecommunication Standardization Sector, May 2006.

[0676] [ITU07], G.722 Annex IV: A low-complexity algorithm for packet lossconcealment with G.722, ITU-T Recommendation, ITU-T, August 2007.

[0677] [ITU08a], G.718: Frame error robust narrow-band and wideband embedded variable bit-rate coding of speech and audio from 8-32 kbit / s, Recommendation ITU-TG.718, ITU Telecommunication Standardization Sector, June 2008.

[0678] [ITU08b], G.719: Low-complexity, full-band audio coding for high-quality, conversational applications, Recommendation ITU-T G.719, ITU Telecommunication Standardization Sector, June 2008.

[0679] [ITU12], G.729: Coding of speech at 8 kbit / s using conjugate-structure algebraic-code-excited linear prediction (cs-acelp), Recommendation ITU-T G.729, ITU Telecommunication Standardization Sector, June 2012.

[0680] [MCZ11] Xinwen Mu, Hexin Chen, and Yan Zhao, A frame erasure concealment method based on pitch and gain linear prediction fbr AMR-WB codec, In Consumer Electronics (ICCE), 2011 IEEE International Conference on, January 2011, pp. 815-816.

[0681] [MTTA90] J. S. Marques, I. Trancoso, J. M. Tribolet, and L. B. Almeida, Improved pitch prediction with fractional delays in cel coding, Acoustics, Speech, and Signal Processing, 1990. ICASSP-90., 1990 International Conference on Computer Aided Design, 1990, pp. 665-668, Vol. 2.

[0682] [VJGS12] Tommy Vaillancourt, Milan Jelinek, Philippe Gournay, and Redwan Salami, Method and device for efficient frame erasure concealment in speechcodecs, US 8,255,207 B2, 2012.

Claims

1. An apparatus for determining an estimated pitch lag, comprising: An input interface (110) for receiving a plurality of original pitch lag values, and a pitch lag estimator (120) for estimating the estimated pitch lag, wherein the pitch lag estimator (120) is configured to estimate the estimated pitch lag based on a plurality of original pitch lag values ​​and based on a plurality of pitch gain values, wherein for each original pitch lag value in the plurality of original pitch lag values, a pitch gain value in the plurality of pitch gain values ​​is assigned to the original pitch lag value; wherein the pitch lag estimator (120) is configured to estimate the estimated pitch lag by minimizing an error function, the error function comprising two or more of the plurality of original pitch lag values ​​and comprising two or more of the plurality of pitch gain values; Wherein, the error function is: where a is a real number, Where b is a real number, where k is an integer with k ≥ 1, and where P(i) is the ith original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th original pitch lag value P(i).

2. The apparatus of claim 1, wherein each of the plurality of pitch gain values ​​is an adaptive codebook gain.

3. The apparatus according to claim 2, wherein the pitch lag estimator is used to estimate the estimated pitch lag by determining two parameters a, b while minimizing the error function.

4. The apparatus according to claim 1, wherein the pitch lag estimator is configured to estimate the estimated pitch lag by determining two parameters a, b by minimizing the following error function, 5. The apparatus of claim 1, wherein the pitch lag estimator is configured to determine the estimated pitch lag p according to p=a·i+b.

6. A system for reconstructing a frame comprising a speech signal, wherein the system comprises: The apparatus for determining an estimated pitch lag according to claim 1, and means for reconstructing the frame, wherein the means for reconstructing the frame is configured to reconstruct the frame based on the estimated pitch lag, Wherein the estimated pitch lag is the pitch lag of the speech signal.

7. A system for reconstructing a frame comprising a speech signal according to claim 6, wherein the reconstructed frame is associated with one or more available frames, the one or more available frames being at least one of one or more previous frames of the reconstructed frame and one or more subsequent frames of the reconstructed frame, wherein the one or more available frames include one or more pitch periods as the one or more available pitch periods, and The device for reconstructing the frame comprises: A determination unit (210) is used to determine the sample quantity difference The sample size difference indicating a difference between a number of samples of one of the one or more available pitch periods and a number of samples of a first pitch period to be reconstructed, and A frame reconstructor (220) is used to reconstruct the frame according to the difference in the number of samples. and reconstructing the reconstructed frame by reconstructing the first pitch period to be reconstructed as a first reconstructed pitch period based on samples of one of the one or more available pitch periods, wherein the frame reconstructor (220) is configured to reconstruct the reconstructed frame so that the reconstructed frame completely or partially includes the first reconstructed pitch period, so that the reconstructed frame completely or partially includes the second reconstructed pitch period, and so that the number of samples of the first reconstructed pitch period is different from the number of samples of the second reconstructed pitch period, The determining unit (210) is used to determine the sample quantity difference according to the estimated pitch lag.

8. A method for determining an estimated pitch lag, comprising: receiving a plurality of raw pitch lag values, and estimating the estimated pitch lag, wherein the estimated pitch lag is estimated based on a plurality of original pitch lag values ​​and based on a plurality of pitch gain values, wherein for each original pitch lag value of the plurality of original pitch lag values, a pitch gain value of the plurality of pitch gain values ​​is assigned to the original pitch lag value, wherein the estimated pitch lag is estimated by minimizing an error function including two or more of the plurality of original pitch lag values ​​and including two or more of the plurality of pitch gain values; Wherein, the error function is: where a is a real number, Where b is a real number, where k is an integer with k ≥ 1, and where P(i) is the ith original pitch lag value, where g p (i) is the i-th pitch gain value assigned to the i-th original pitch lag value P(i).

9. A computer program for implementing the method according to claim 8 when it is executed on a computer or a signal processor.

Images