A method and device for calculating pitch cycle in packet loss concealment and a readable medium

Abstract

The application discloses a method and device for calculating pitch period in packet loss concealment and a readable medium, wherein the method obtains the pitch period through a correlation coefficient method, and in the calculation process of the correlation coefficient, the calculation of the total energy of a template window and x2 is removed, the calculation process of the total energy of a sliding window and y2 is adjusted, and a Taylor expansion method is used to obtain an approximation equation, the operation of 1 / sqrt(z) is approximately replaced by the approximation equation, which is equivalent to removing the division and square root (sqrt) operation in the previous algorithm and replacing them only with multiplication, shifting and addition and subtraction, so that the total calculation amount is greatly reduced, even if the hardware acceleration circuit is implemented in this way, the hardware area is greatly reduced, and cost and power consumption can be effectively saved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of network communication technology, and in particular, to a method and device for calculating pitch period in packet loss concealment and a readable medium. BACKGROUND

[0002] In the process of network transmission, such as wireless communication, when interference is received or the distance between devices is relatively far, it is easy to cause loss of voice packets, resulting in a decline in voice communication quality. In addition to using channel coding with strong error correction capability, various packet loss concealment technologies have become the main means for dealing with unstable wireless networks in real-time voice services.

[0003] Packet loss concealment (PLC) is a technology that compensates for lost packets by synthesizing data packets to eliminate or reduce the adverse effects of packet loss on call quality. The pitch waveform replacement-based packet loss concealment method is the most commonly used method, which first needs to find the pitch period, and the determination of the pitch period is crucial to obtaining the correlation coefficient corresponding to the pitch period.

[0004] Correlation coefficient analysis is a commonly used speech time-domain waveform analysis method defined by a correlation function. The correlation function can be used to measure the time-domain similarity between signals. When the two signals being correlated are completely different, the value of the correlation function is close to zero; when the waveforms of the two signals being correlated are the same, a peak value appears at the leading or lagging position. Therefore, the correlation function can be used to study the signal itself, such as waveform synchronization, periodicity, etc.

[0005] The prior art method for calculating the correlation coefficient corresponding to the pitch period in packet loss concealment has a process as shown in Figure 1

[0006] S101: Determine the template window x, the initial sliding window y, and the sliding range;

[0007] S102: Update the sliding window y, and calculate the dot product num of the two arrays of the template window and the sliding window;

[0008] where M is the length of the template window x and the initial sliding window y;

[0009] S103: Calculate the total energy of the template window x2;

[0010] S104: Calculate the total energy of the sliding window y2;

[0011] S105: Calculate the correlation coefficient denominator den = sqrt(x2*y2);

[0012] S106: Calculate the correlation coefficient cn = num / den;​

[0013] S107: save the maximum correlation coefficient maxcn and the corresponding sliding position bestmatch;

[0014] S108: determine whether the sliding range of the sliding window is completed, if not, jump to S102 for continuous execution; if yes, end.

[0015] In the above calculation method, if the sampling rate is 16k, the template window length M = 120, and the sliding is calculated 480 times, at least 173280 multiplications, 171360 additions, 480 divisions, and 480 square root operations (i.e. sqrt operation) are required. In low-power and low-cost chips, such as RISC-V CPU or ARM M3 / M4, most of the time, the CPU is a fixed-point CPU without floating-point unit and the frequency of the CPU is not too high. The sqrt operation cannot be supported. For a fixed-point CPU, one fixed-point sqrt operation needs to achieve 16-bit calculation precision, which is estimated to require 10 multiplications and 5 additions, which is a time-consuming calculation process. Usually, a dedicated hardware acceleration circuit is needed to realize it, otherwise the operation amount is too large; if the CPU does not support the fixed-point division hardware acceleration circuit, the fixed-point division is also more time-consuming than the fixed-point multiplication. It can be seen that the calculation amount of the mode matching correlation coefficient method in the existing packet loss concealment algorithm is very large, which may be very difficult to run in low-power and low-cost chips, resulting in the need to increase the frequency and increase the power consumption, and if a hardware acceleration circuit is added, the cost is also increased.

[0016] Therefore, whether the calculation method of the pitch period corresponding correlation coefficient in the packet loss concealment can be optimized to reduce the operation amount has become a problem that people urgently need to solve. SUMMARY

[0017] In view of this, the present application provides a calculation method of a pitch period in a packet loss concealment, a calculation device and a readable medium to solve the problems in the background art.

[0018] In one aspect, the present application provides a calculation method of a pitch period in a packet loss concealment, which comprises the following steps:

[0019] S1: determining a template window x, an initial sliding window y and a sliding range based on historical voice data;

[0020] S2: calculating the initial sliding window energy y2a;

[0021] S3: after sliding the sliding window y by a unit length in a preset direction, obtaining an updated sliding window y, and calculating the dot product num of the two arrays of the template window and the sliding window;

[0022] S4: calculating a sliding window total energy sum y2 by using a formula y2=y2a+y(M-1), wherein y(M-1) is an energy of a last value in the sliding window;

[0023] S5: performing Taylor expansion on a function f(z)=1 / sqrt(z) to obtain an approximation equation, and calculating a reciprocal of a correlation coefficient denominator value den2 according to the approximation equation and the sliding window total energy sum y2;

[0024] S6: calculating a correlation coefficient cn according to a dot product num of the two arrays of the template window and the sliding window and the reciprocal of the correlation coefficient denominator value den2;

[0025] S7: saving a maximum correlation coefficient maxcn and a corresponding sliding position bestmach;

[0026] S8: updating the initial sliding window energy sum y2a=y2-y(0), wherein y(0) is an energy of a first value in the sliding window;

[0027] S9: judging whether a sliding range of the sliding window is completed, if not, sequentially executing S3-S8;

[0028] S10: if the sliding range of the sliding window is completed, obtaining the sliding position bestmach saved in step S7, calculating a distance between the sliding position bestmach and an initial sliding window y position, and obtaining a pitch period.

[0029] Preferably, in step S2, a specific formula for calculating the initial sliding window energy sum y2a is:

[0030] wherein M is a length of the initial sliding window y.

[0031] Further preferably, in step S5, the approximation equation is:

[0032] Further preferably, in step S5, the approximation equation is:

[0033] Further preferably, in step S6, a specific formula for calculating the correlation coefficient cn according to the dot product num of the two arrays of the template window and the sliding window and the reciprocal of the correlation coefficient denominator value den2 is:

[0034] cn=num*den2.

[0035] On the other hand, the present application further provides a computing device, which comprises:

[0036] one or more processors;

[0037] a memory device for storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement any of the above computing methods.

[0038] Further, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement any of the above computing methods.

[0039] In the pitch period calculation method for the packet loss concealment provided by the present application, the pitch period is obtained by the correlation coefficient method, and in the calculation process of the correlation coefficient, the calculation of the total energy of the template window and x2 is removed, the calculation process of the total energy of the sliding window and y2 is adjusted, the approximation equation is obtained by the Taylor expansion method, the operation of 1 / sqrt(z) is approximately replaced by the approximation equation, which is equivalent to removing the division and square root (sqrt) operation in the background technology, and only multiplication, shift and addition and subtraction are used to replace, so that the total calculation amount is greatly reduced, even if the hardware acceleration circuit is realized in this way, the hardware area is greatly reduced, and the cost and power consumption can be effectively saved.

[0040] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the disclosure of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below, and obviously, other drawings can also be obtained by those skilled in the art without creative labor.

[0043] Figure 1 The flowchart of the method for calculating the correlation coefficient corresponding to the pitch period in the packet loss concealment in the background art of the present application;

[0044] Figure 2 The flowchart of the pitch period calculation method for the packet loss concealment provided by the present application;

[0045] Figure 3 The error analysis diagram of Taylor expansion in the pitch period calculation method for the packet loss concealment provided by the present application;

[0046] Figure 4A comparison diagram of calculation results of a pitch cycle calculation method and a post-packet loss concealment algorithm provided by the disclosed embodiments. DETAILED DESCRIPTION

[0047] The exemplary embodiments will be described in detail hereinbelow with reference to the drawings. In the following description, the same numbers in different drawings represent the same or similar elements unless otherwise represented. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] In the packet loss concealment technology, the calculation of the pitch cycle is a key step, and currently the correlation coefficient method is mostly used for the calculation of the pitch cycle. However, in the calculation of the correlation coefficient in the past, as described in the background art, since the operations of multiplication, addition, division and square root need to be performed multiple times, the amount of calculation is large, and especially for the operation of square root, a hardware acceleration circuit needs to be additionally provided to improve the operation speed, which has the problems of high cost and large energy consumption. Therefore, the embodiments of the present application attempt to provide a new calculation method to optimize the calculation process of the correlation coefficient to reduce the overall amount of calculation.

[0049] Specifically, referring to Figure 1 The present embodiments provide a calculation method of a pitch cycle in a packet loss concealment, which comprises the following steps:

[0050] S1: determining a template window x, an initial sliding window y and a sliding range based on historical voice data;

[0051] S2: calculating an initial sliding window energy y2a;

[0052] S3: after the sliding window y is slid by a unit length in a preset direction, an updated sliding window y is obtained, and a dot product num of the two arrays of the template window and the sliding window is calculated;

[0053] S4: using the formula y2=y2a+y(M-1), the total energy y2 of the sliding window is calculated, wherein y(M-1) is the energy of the last value in the sliding window;

[0054] S5: performing Taylor expansion on the function f(z)=1 / sqrt(z) to obtain an approximation equation, and according to the approximation equation and the total energy y2 of the sliding window, the reciprocal den2 of the denominator value of the correlation coefficient is calculated;

[0055] S6: according to the dot product num of the two arrays of the template window and the sliding window and the reciprocal den2 of the denominator value of the correlation coefficient, the correlation coefficient cn is calculated;

[0056] S7: save the maximum correlation coefficient maxcn and the corresponding sliding position bestmach;

[0057] S8: update the initial sliding window energy and y2a=y2-y(0), wherein y(0) is the energy of the first value in the sliding window;

[0058] S9: determine whether the sliding range of the sliding window is completed, if not, sequentially execute S3-S8;

[0059] S10: if completed, obtain the sliding position bestmach saved in step S7, calculate the distance between the sliding position bestmach and the initial sliding window y position, and obtain the pitch period.

[0060] The pitch period obtained by the method provided in the above embodiment is near an integer multiple of the real period, and in actual implementation process, the pitch period can be directly used, of course, in order to further optimize the accuracy effect, the pitch period obtained in the embodiment can be corrected to obtain the final pitch period.

[0061] In step S1, the length of the template window x and the length of the initial sliding window y are the same, both are M, and the sliding window y slides in the set sliding range of the historical voice data. Generally, the sliding range is above 30 ms, and the sliding length is above 480 times calculated by using the actual general 16k sampling rate.

[0062] In step S2, the specific formula of the initial sliding window energy y2a is:

[0063] Wherein, M is the length of the initial sliding window y.

[0064] Here, the initial sliding window energy y2a is only counted to M-2, so that in the subsequent calculation of the total energy y2 of the sliding window, only the energy changes of the first value and the last value after the sliding of the sliding window need to be considered, and multiple repeated addition calculations of the energy of multiple values in the overlapping area are not needed, thereby reducing the calculation amount.

[0065] In step S3, the sliding direction of the sliding window y can be from left to right or from right to left, and the calculation result is not affected, only the positions of the first value and the last value in the sliding window y are changed in the subsequent steps. When the sliding window y slides from left to right, the first value is the first value on the left side of the sliding window y, and the last value is the first value on the right side of the sliding window y. When the sliding window y slides from right to left, the first value is the first value on the right side of the sliding window y, and the last value is the first value on the left side of the sliding window y. The moving length of the sliding window y is 1 unit length, and the specific formula of the dot product num of the two arrays of the template window and the sliding window is as follows:

[0066] Wherein, M is the length of the template window x and the initial sliding window y.

[0067] In step S4, the total energy sum y2 of the sliding window is calculated by using the formula y2=y2a+y(M-1), wherein y(M-1) is the energy of the last value in the sliding window. Since the initial sliding window energy sum y2a calculated in step S2 only counts M-2, the energy of the last value in the sliding window is added in step S4.

[0068] In step S5, the Taylor expansion is used:

[0069]

[0070] The function f(z)=1 / sqrt(z) is expanded according to the above Taylor expansion formula near z0=1 to obtain the first-order approximation equation of the original equation The total energy sum y2 of the sliding window calculated in step S4 is substituted into the approximation equation to calculate the reciprocal den2 of the denominator value of the correlation coefficient.

[0071] In order to further improve the calculation accuracy, the second-order approximation equation of the original equation The total energy sum y2 of the sliding window calculated in step S4 is substituted into the approximation equation to calculate the reciprocal den2 of the denominator value of the correlation coefficient.

[0072] In this step, the approximation equation f2(z) is used to approximately calculate 1 / sqrt(z). Since the main purpose of the calculation method is to find the corresponding sliding position by the maximum correlation coefficient, it is not necessary to accurately calculate the maximum correlation coefficient, but only to compare the sizes of the correlation coefficients. Therefore, 1 / sqrt(z) can be approximately calculated by the approximation equation, and the square root and division in the background technology are replaced by the Taylor expansion approximation method, so that the calculation amount is greatly reduced.

[0073] The error analysis of the first-order approximation equation f2(z) is performed below, and the precision error of the first-order approximation equation f2(z) is fixed at 16 bits, as shown in the following table. Figure 3 As shown in the table, the horizontal coordinate z = [0.1 -1], and the vertical coordinate "*" is the standard correct value, and the "+" point is the approximate approximation value. From the table, it can be seen that although there is an error, the size of f2(z) values of different z values is consistent, and in actual application, only the position of the maximum correlation coefficient in the entire sliding range needs to be known, and the correlation coefficient value does not need to be known correctly, only the size needs to be correctly distinguished, so the approximate approximation equation is effective. Figure 3

[0074] In step S6, the specific formula for calculating the correlation coefficient cn is obtained according to the point multiplication of the template window and the sliding window, num, and the reciprocal den2 of the correlation coefficient denominator value.

[0075] cn=num*den2.

[0076] In step S7, the correlation coefficient cn calculated in step S6 is compared with the previously stored correlation coefficient. If it is greater than the previously stored correlation coefficient, the correlation coefficient cn calculated in step S6 is stored as the maximum correlation coefficient maxcn and the sliding position bestmach corresponding to the maximum correlation coefficient maxcn is saved. If it is less than or equal to the previously stored correlation coefficient, the original correlation coefficient and the corresponding sliding position are still saved.

[0077] In step S8, since the sliding window y needs to be slid next time, the first value in the sliding window y will be removed, and therefore, the total energy of the current sliding window and the energy of the first value in the sliding window y in y2 are removed as the initial sliding window energy y2.

[0078] In step S9, it is judged whether the sliding window has traversed the entire sliding range. If not, steps S3-S8 are sequentially executed. If the entire sliding range has been traversed, the sliding position bestmach saved in step S7 is obtained in step S10, the distance between the sliding position bestmach and the initial sliding window y position is calculated, and the pitch period is obtained.

[0079] In order to verify the effectiveness of the pitch period calculation method provided by the embodiment, the method provided by the embodiment is used to calculate the pitch period in the packet loss concealment, and the effect of the audio after the packet loss concealment is observed.

[0080] As shown in the following table, the horizontal coordinate z = [0.1 -1], and the vertical coordinate "*" is the standard correct value, and the "+" point is the approximate approximation value. From the table, it can be seen that although there is an error, the size of f2(z) values of different z values is consistent, and in actual application, only the position of the maximum correlation coefficient in the entire sliding range needs to be known, and the correlation coefficient value does not need to be known correctly, only the size needs to be correctly distinguished, so the approximate approximation equation is effective. Figure 4 ​As shown, the upper waveform in the figure is the audio with 2 frames lost, and the lower waveform in the figure is the audio processed by the packet loss concealment method, and the processed audio is noise-free. In the packet loss concealment method, the pitch period is calculated by the method provided in the above embodiment, indicating that the pitch period calculated by the method provided in the above embodiment is accurate. In addition, actual tests of multiple audio files are also conducted, and the final processed audio is all noise-free.

[0081] The method for calculating the pitch period in the packet loss concealment provided in the above embodiment removes the calculation of the template window total energy and x2 in the prior art, adjusts the sliding window total energy and y2 calculation process, and approximately realizes the operation of 1 / sqrt(z) by using the Taylor expansion method, which is equivalent to removing the division and sqrt operation, and only using multiplication, shifting and addition and subtraction instead, so that the total calculation amount is greatly reduced. Even if the hardware acceleration circuit is realized in this way, the hardware area is also greatly reduced.

[0082] According to the calculation in the background art, the operations with similar shift and other calculation amounts are replaced by addition, and the total calculation amount after optimization can be estimated to be 60119 multiplications and 60118 additions. Compared with the prior art, 113161 multiplications, 111242 additions, 480 divisions and 480 square root operations (i.e. sqrt operation) are reduced. The read and write data times not counted will also be greatly reduced.

[0083] The embodiment also provides a computing device, which comprises:

[0084] one or more processors;

[0085] a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement the method of any one of the above embodiments.

[0086] In addition, the embodiment also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of any one of the above embodiments.

[0087] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0088] It should be understood that the application is not limited to what has been described above and that various modifications and changes can be made without departing from its scope. The scope of the application is limited only by the claims that follow.

Claims

1. A method for calculating pitch period in packet loss concealment, characterized in that: The steps include: S1: Determine the template window x, the initial sliding window y, and the sliding range based on historical speech data; S2: Calculate the initial sliding window energy and y2a; S3: After sliding the sliding window y along the preset direction by the unit length, an updated sliding window y is obtained, and the dot product of the template window and the sliding window arrays and num are calculated; S4: Calculate the total energy of the sliding window and y2 using the formula y2=y2a+y(M-1), where y(M-1) is the energy of the last value in the sliding window; S5: Performing Taylor expansion on the function f(z)=1 / sqrt(z) to obtain an approximation equation, and calculating the inverse of the denominator value of the correlation coefficient den2 based on the approximation equation, the total energy of the sliding window, and y2; S6: Calculate the correlation coefficient cn based on the dot product of the template window and the sliding window arrays, num, and the inverse of the correlation coefficient denominator value den2; S7: Save the maximum correlation coefficient maxcn and the corresponding sliding position bestmach; S8: Update the initial sliding window energy sum y2a=y2-y(0), where y(0) is the energy of the first value in the sliding window; S9: Determine whether the sliding window sliding range is completed. If not, execute S3-S8 in sequence. S10: If completed, obtain the sliding position bestmach saved in step S7, calculate the distance between the sliding position bestmach and the initial sliding window y position, and obtain the fundamental pitch period.

2. The method for calculating the pitch period in packet loss concealment according to claim 1, wherein: In step S2, the specific formula for calculating the initial sliding window energy and y2a is: Where M is the length of the initial sliding window y.

3. The method for calculating the pitch period in packet loss concealment according to claim 1, wherein: In step S5, the approximation equation is:

4. The method for calculating the pitch period in packet loss concealment according to claim 1, wherein: In step S5, the approximation equation is:

5. The method for calculating the pitch period in packet loss concealment according to claim 1, wherein: In step S6, the specific formula for calculating the correlation coefficient cn based on the dot product of the template window and the sliding window arrays, num, and the inverse of the correlation coefficient denominator value den2 is: cn=num*den2.

6. A computing device, characterized in that: include: one or more processors; A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that The readable storage medium stores a computer program, which implements the method according to any one of claims 1 to 5 when executed by a processor.

Images