Voice packet loss hiding method and device, electronic equipment and storage medium

Abstract

The invention provides a voice packet loss hiding method and device, electronic equipment and a storage medium, and the method comprises the steps: carrying out the framing processing of a received voice signal, and obtaining a current frame signal and a historical reference frame signal; based on the current frame signal, calculating an LPC coefficient through linear predictive coding LPC analysis; performing echo detection on the current frame signal and the historical reference frame signal, and judging whether an echo exists in the current frame signal; if it is judged that the echo exists, echo cancellation processing is started, and an LPC coefficient is updated based on an echo cancellation result to obtain an updated LPC coefficient; the updated LPC coefficient is used for frame reconstruction when subsequent voice packet loss occurs. According to the invention, through real-time echo detection and elimination of echo components in the reconstructed voice, the core problem of echo generation due to limitation of an LPC model in a transient phase is solved in a targeted manner, so that the reconstructed voice is purer and more natural, and the voice quality is remarkably improved.

Description

Technical Field

[0001] This disclosure relates to the field of voice communication technology, and more specifically, to a method and apparatus for hiding voice packet loss. Background Technology

[0002] In real-time voice communication systems, such as multi-person voice conferencing and in-vehicle voice interaction, network packet loss is one of the main causes of voice quality degradation. To address this problem, Packet Loss Concealment (PLC) technology has emerged. Its core is to use algorithms at the receiving end to recover lost voice data packets in order to maintain the continuity of communication.

[0003] Traditional packet loss concealment methods are typically based on Linear Predictive Coding (LPC). The process includes: first, distinguishing between voiced and unvoiced sounds using Voice Activity Detection (VAD); then performing pitch analysis to obtain the pitch period; next, performing Long-Term Prediction (LTP) analysis on the voiced signal to remove periodic components and obtain a residual signal; finally, performing LPC analysis on the residual signal to obtain linear prediction coefficients, which are used for frame reconstruction in case of packet loss.

[0004] However, the aforementioned traditional methods have significant limitations when processing rapidly changing segments of speech (such as transitions from consonants to vowels, the onset of plosives, and other transient signals). Because the LPC model struggles to accurately characterize the complex spectral characteristics of transient speech segments, echoes and other problems easily occur in the reconstructed speech frames during packet loss recovery, severely impacting speech clarity and becoming a technical bottleneck for improving voice communication quality. Summary of the Invention

[0005] This disclosure provides at least one method and apparatus for hiding voice packet loss to solve the above-mentioned technical problems.

[0006] In a first aspect, embodiments of this disclosure provide a method for hiding voice packet loss, including: The received audio signal is processed by framing to obtain the current frame signal and the historical reference frame signal; Based on the current frame signal, LPC coefficients are calculated using linear predictive coding (LPC) analysis. Echo detection is performed between the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal. If an echo is detected, echo cancellation processing is initiated, and the LPC coefficients are updated based on the echo cancellation result to obtain the updated LPC coefficients. The updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

[0007] In one possible implementation, the method further includes: If it is determined that there is no echo, the calculated LPC coefficients are output.

[0008] In one possible implementation, the echo detection of the current frame signal and the historical reference frame signal includes: Calculate the correlation peaks between the current frame signal and the multiple historical reference frame signals; The presence of an echo is determined based on the peak value of the relevant peak.

[0009] In one possible implementation, calculating the correlation peak between the current frame signal and the plurality of historical reference frame signals includes: Calculate the normalized cross-correlation function between the current frame signal and each historical reference frame signal respectively; The local maximum peak value is extracted from each of the normalized cross-correlation functions as the correlation peak of the corresponding historical reference frame.

[0010] In one possible implementation, determining whether an echo exists based on the peak value of the relevant peak includes: Determine the global maximum peak value from all relevant peaks; If the global maximum peak value is greater than the preset peak value threshold, it is determined that an echo exists, and the historical reference frame signal corresponding to the global maximum peak value is determined as the echo reference signal.

[0011] In one possible implementation, upon detecting the presence of an echo, echo cancellation processing is initiated in one of the following ways: The echo cancellation process is performed using a traditional adaptive filtering algorithm to update the LPC coefficients; The echo cancellation process is performed using a neural network model to update the LPC coefficients.

[0012] In one possible implementation, the echo cancellation process performed using a conventional adaptive filtering algorithm to update the LPC coefficients includes: Based on the echo reference signal and the current frame signal, the adaptive filter coefficients are iteratively updated using the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm. The current frame signal is filtered using the updated filter coefficients to obtain the de-echo signal; Based on the de-echo signal, LPC analysis is performed again to generate the updated LPC coefficients.

[0013] In one possible implementation, the step of using a neural network model to perform the echo cancellation process to update the LPC coefficients includes: The LPC coefficients calculated based on the current frame signal are input into a pre-trained neural network model; the neural network model is used to learn the mapping relationship between echo-containing LPC coefficients and de-echoed LPC coefficients. The updated LPC coefficients are directly output by the neural network model.

[0014] In one possible implementation, the method is executed cyclically in units of frames.

[0015] Secondly, this disclosure also provides a voice packet loss hiding device, comprising: The framing module is used to perform framing processing on the received voice signal to obtain the current frame signal and the historical reference frame signal. The calculation module is used to calculate LPC coefficients based on the current frame signal through linear predictive coding (LPC) analysis. The detection module is used to perform echo detection on the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal; An update module is used to initiate echo cancellation processing if an echo is detected, and update the LPC coefficients based on the echo cancellation result to obtain updated LPC coefficients; the updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

[0016] Thirdly, this disclosure also provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the voice packet loss hiding method as described in any one of the first aspects and various embodiments thereof is performed.

[0017] Fourthly, this disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the voice packet loss concealment method as described in any one of the first aspects and its various embodiments.

[0018] The aforementioned speech packet loss concealment method, apparatus, electronic device, and storage medium first perform frame segmentation on the received speech signal to obtain the current frame signal and historical reference frame signals. Then, LPC coefficients are calculated using Linear Predictive Coding (LPC) analysis. Echo detection is performed between the current frame signal and the historical reference frame signals to determine if an echo exists in the current frame signal. If an echo is detected, echo cancellation processing is initiated, and the LPC coefficients are updated based on the echo cancellation results. The updated LPC coefficients are then used for frame reconstruction during packet loss. This disclosure addresses the core problem of echo generation in transient segments due to the limitations of the LPC model by detecting and eliminating echo components in the reconstructed speech in real time. This results in cleaner and more natural reconstructed speech, significantly improving speech quality.

[0019] Other advantages of this disclosure will be explained in more detail in conjunction with the following description and accompanying drawings.

[0020] It should be understood that the above description is merely an overview of the technical solution of this disclosure, so as to enable a general understanding of the technical means of this disclosure and to implement it in accordance with the contents of the specification. In order to make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. The accompanying drawings are incorporated in and constitute a part of this specification. These drawings illustrate embodiments conforming to this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure. It should be understood that the drawings only illustrate certain embodiments of this disclosure and should not be considered as a limitation on the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. Furthermore, the same reference numerals denote the same components throughout the drawings. In the drawings: Figure 1 A flowchart of a voice packet loss concealment method provided in an embodiment of this disclosure is shown; Figure 2 This diagram illustrates an application flowchart of the voice packet loss concealment method provided in an embodiment of the present disclosure; Figure 3 A schematic diagram of a voice packet loss concealment device provided in an embodiment of this disclosure is shown; Figure 4 A schematic diagram of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0022] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0023] In the description of embodiments disclosed herein, it should be understood that terms such as “comprising” or “having” are intended to indicate the presence of the disclosed features, figures, steps, behaviors, components, portions or combinations thereof in this specification, and do not exclude the possibility of the presence of one or more other features, figures, steps, behaviors, components, portions or combinations thereof.

[0024] Unless otherwise stated, " / " means "or". For example, A / B can mean A or B. In this article, "and / or" is merely a way of describing the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A alone, A and B at the same time, and B alone.

[0025] The terms "first," "second," etc., are used only for ease of description to distinguish identical or similar technical features and should not be construed as indicating or implying the relative importance or number of these technical features. Therefore, a feature defined by "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, the term "multiple" means two or more.

[0026] Research has revealed that traditional packet loss concealment methods have significant limitations when handling rapidly changing segments of speech (such as transitions from consonants to vowels, the onset of plosives, and other transient signals). Because LPC models struggle to accurately characterize the complex spectral characteristics of transient speech segments, reconstructed speech frames are prone to echoes during packet loss recovery, severely impacting speech clarity and becoming a technical bottleneck for improving voice communication quality.

[0027] In order to at least partially solve one or more of the above-mentioned problems and other potential problems, this disclosure provides a speech packet loss concealment method, apparatus, electronic device and storage medium capable of dynamically detecting and eliminating such echoes, thereby significantly improving the speech quality after packet loss recovery.

[0028] To facilitate understanding of this embodiment, a detailed description of the voice packet loss concealment method disclosed in this disclosure is provided first. The execution entity of the voice packet loss concealment method provided in this disclosure is generally an electronic device with a certain computing power. This electronic device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a personal digital assistant (PDA), an in-vehicle device, a wearable device, etc. In some possible implementations, the voice packet loss concealment method can be implemented by a processor calling computer-readable instructions stored in memory.

[0029] See Figure 1 The diagram illustrates a flowchart of a voice packet loss concealment method provided in an embodiment of this disclosure, the method comprising the following steps S101-S104: S101: Perform frame segmentation on the received voice signal to obtain the current frame signal and the historical reference frame signal; S102: Calculate the LPC coefficients based on the current frame signal through linear predictive coding (LPC) analysis; S103: Perform echo detection on the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal; S104: If an echo is detected, echo cancellation processing is initiated, and the LPC coefficients are updated based on the echo cancellation result to obtain the updated LPC coefficients; the updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

[0030] To facilitate understanding of the voice packet loss hiding method provided in this disclosure, the application scenarios of the method will be briefly described first. This voice packet loss hiding method can be mainly applied to various real-time voice communication systems, such as multi-person voice conferencing and in-vehicle voice interaction. Considering the widespread application of multi-person voice conferencing, this scenario will be used as the primary example in the following examples.

[0031] In real-world scenarios, network packet loss can significantly degrade speech quality. Existing packet loss concealment methods are limited by the characteristics of the LPC model, leading to issues such as echoes in the reconstructed speech frames. Therefore, this disclosure provides a speech packet loss concealment method that dynamically detects and eliminates relevant echoes. This method, by introducing an echo detection and elimination mechanism, specifically addresses the core problem of echo generation in the transient segment by traditional PLCs, resulting in cleaner and more natural reconstructed speech and significantly improved speech quality.

[0032] like Figure 2As shown, the speech packet loss concealment method provided in this disclosure embeds an intelligent echo detection and cancellation step in the LPC coefficient calculation process. The overall process is a frame-by-frame loop, that is: each time a new frame of speech signal is acquired, the steps of frame segmentation, LPC calculation, echo detection, condition cancellation, and coefficient update are executed in sequence. The updated coefficients are stored and used for future packet loss reconstruction.

[0033] For a single loop, the real-time speech signal received from the network is first segmented into frames to obtain the current frame signal to be processed and the historical reference frame signal read from the buffer. Next, based on the current frame signal, LPC (Low-Time Perception) technology is used for analysis to calculate a set of LPC coefficients, which describe the short-time spectral envelope of the speech frame. Then, echo detection is performed on the current frame signal and the historical reference frame signal to determine whether the current frame contains inappropriate repetitive components (i.e., echoes) originating from historical frames.

[0034] If an echo is detected, echo cancellation is initiated. Based on the echo cancellation result, the previously calculated LPC coefficients are updated, resulting in a set of "cleaned" updated LPC coefficients. These updated LPC coefficients (regardless of whether they have been updated before) are stored in a coefficient memory. Their core purpose is to allow the system to use these latest and optimal LPC coefficients, combined with other parameters (such as pitch period), to reconstruct the lost speech frames when a network packet loss event is detected. This effectively suppresses echoes in the reconstructed speech while concealing the packet loss.

[0035] After echo detection, if the result indicates that there is no significant echo in the current frame signal, the system will not initiate echo cancellation processing. At this point, the process will jump directly to the next step, outputting the LPC coefficients calculated in step S102 based on the original current frame signal (i.e., coefficients without echo cancellation processing), and save them for subsequent packet loss reconstruction. This design ensures that the algorithm performs additional calculations only when necessary, improving overall processing efficiency.

[0036] The voice packet loss concealment method disclosed herein is a real-time, continuous processing loop. For example... Figure 2 As shown in the overall flow diagram, the entire method is executed cyclically, using speech frames as the basic processing unit. Specifically, when the processing of the first... l After all processing of the frame signal (including possible coefficient updates and saving), the process does not terminate, but returns to the starting point, waiting for or reading the next frame. l The process involves adding one frame of audio signal and then repeating all the steps described above. This cyclic execution mechanism ensures that the present disclosure can perform uninterrupted real-time echo monitoring and suppression of continuous audio streams.

[0037] In the specific echo detection process, this embodiment of the disclosure is based on correlation peak analysis. First, the cross-correlation between the current frame signal and multiple historical reference frame signals is calculated, and correlation peaks representing their similarity are extracted. Specifically, the system selects a continuous set of historical frames as a reference set and calculates the correlation peaks between the current frame and each of those frames. Then, by analyzing the peak values ​​of these correlation peaks (e.g., finding the maximum value), it objectively determines whether there is a segment in the current frame that is highly similar to a historical frame, i.e., an echo.

[0038] In a preferred embodiment, the process of calculating the correlation peak between the current frame signal and a certain historical reference frame signal includes: First, calculating the normalized cross-correlation function between the two. To improve computational efficiency, this operation can be performed in the frequency domain, i.e., first performing a Fast Fourier Transform (FFT) on both frame signals respectively, calculating their normalized cross-power spectra, and then performing an Inverse Fast Fourier Transform (IFFT). Then, extracting the local maximum peak value from the calculated normalized cross-correlation function, which is the correlation peak value between the current frame and this specific historical frame.

[0039] In this embodiment, the automatic echo detection logic is as follows: The system finds the global maximum peak value among all calculated relevant peak values. This global maximum peak value is compared with a pre-set peak threshold. If the global maximum peak value > the peak threshold, an echo is determined to exist. Simultaneously, the frame number of the historical reference frame corresponding to the global maximum peak value is recorded, and this frame signal is determined as the echo reference signal for subsequent echo cancellation processing. If the global maximum peak value ≤ the peak threshold, no echo is detected.

[0040] When an echo is detected, this embodiment provides two optional paths to perform echo cancellation processing and update the LPC coefficients. During implementation, one of these paths can be selected based on system resources and performance requirements.

[0041] Path A: Echo cancellation is performed using a traditional adaptive filtering algorithm. This type of algorithm estimates and cancels echo components by iteratively adjusting the filter coefficients. Specifically, it can be implemented through the following steps: Step 1: Based on the echo reference signal and the current frame signal, iteratively update the adaptive filter coefficients using the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm. Step 2: Filter the current frame signal using the updated filter coefficients to obtain the de-echo signal; Step 3: Perform LPC analysis again based on the de-echo signal to generate updated LPC coefficients.

[0042] The following is an example of a specific implementation method: Using a determined echo reference signal and the current frame signal An adaptive filter is iteratively updated using either the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm. The coefficient.

[0043] The following is the calculation and update process of the LMS algorithm for each frame:

[0044]

[0045]

[0046] in, For the estimated first denominator used for deecho Frame filter, where, filter Length is . After removing the echo, the first Frame signal, For error signals, μ This is the step size factor. Error signal. That is, the signal after echo removal. Then, this echo signal... As input, the LPC analysis is re-executed (e.g., using the Levinson-Durbin algorithm) to calculate a new, cleaner set of LPC coefficients, i.e., the updated LPC coefficients. .

[0047] Path B: Echo cancellation using a neural network model. This type of data-driven model can learn the complex mapping from echo-containing features to clean features. Specifically, it can be implemented through the following steps: Step 1: Input the LPC coefficients calculated based on the current frame signal into the pre-trained neural network model; the neural network model is used to learn the mapping relationship between the echo-containing LPC coefficients and the de-echoed LPC coefficients. Step 2: The updated LPC coefficients are directly output by the neural network model.

[0048] The following is an example of a specific implementation method: A neural network model is pre-trained using a large amount of echo-tagged speech data. The training objective of this model is to learn a non-linear mapping from "echo-tagged LPC coefficient vectors" to "clean LPC coefficient vectors". Available network architectures include, but are not limited to, convolutional neural networks, long short-term memory networks, or Transformers.

[0049] Once an echo is detected, no signal-level filtering is required. The LPC coefficients calculated based on the original current frame (i.e., the detected "echo-containing coefficients") are directly input into the pre-trained neural network model.

[0050] The echo cancellation is completed during the forward propagation process of the neural network model, and its output layer directly provides the estimated "de-echo LPC coefficients". This method eliminates the intermediate signal recovery step and is highly efficient.

[0051] To facilitate a further understanding of the voice packet loss hiding method provided in the embodiments of this disclosure, the implementation steps of the method will be described below in conjunction with specific formulas.

[0052] Step 1: Perform frame segmentation on the signal. This is the audio signal of the current frame. Historical reference signals, among which It is the current frame number. This is the historical reference signal frame number. Different values ​​represent a series of frame signals in the historical buffer. The sample point number, The range of values ​​is , in, This represents the number of sample points for each frame of the signal.

[0053] Step 2: Calculate the LPC coefficients of the current frame using the following formula. The method is as follows: =

[0054] in, It is the order of the filter. These are the coefficients of the filter. It refers to prediction error. The goal of LPC is to find an optimal set of prediction coefficients. This leads to prediction error To minimize energy consumption, the Levinson-Durbin recursive algorithm is typically used in traditional PLC calculation methods to solve for the prediction coefficients. .

[0055] Step 3: Activate the echo detection canceller and import the saved previous... Frame history reference signal LPC coefficients calculated using traditional PLC methods .

[0056] Step 4: The echo detection canceller can be used to determine the current frame. Does an echo exist?

[0057] The method for detecting echoes using correlation peak analysis is as follows: remember It is a reference signal used for echo detection. The range of values ​​is .in It should be greater than the long-term prediction window duration. The width of the echo signal detection. , These values ​​are usually taken from empirical values ​​or actual measurement results.

[0058] In echo signal detection width In the middle, analyze and calculate the current frame signal With each historical reference signal frame Normalized cross-correlation function The cross-correlation function describes the similarity between two signals, typically corresponding to a value in the range [0,1], where 0 indicates no correlation and 1 indicates the highest correlation. The calculation method is as follows:

[0059]

[0060]

[0061]

[0062] Find each cross-correlation function Local maximum peak value , This is used to measure the similarity between the current frame and any historical frame; then, it is used to measure the similarity between each local maximum peak value. Select the global maximum peak value and global maximum peak value Corresponding frame number The specific calculation method is as follows:

[0063]

[0064] Determine the validity of correlated frames based on peak value: When It is assumed that a signal with a high correlation to a historical frame has appeared in the current frame signal, i.e., an echo, and the echo reference frame signal is... Otherwise, it is assumed that no echo was detected. These values ​​are usually taken from empirical values ​​or actual measurement results.

[0065] Step 5: If there is an echo, the coefficients can be updated using the echo cancellation method described above or the neural network echo cancellation method; if there is no echo, the LPC coefficients calculated using the traditional PLC method are output. The specific implementation process is described above and will not be repeated here.

[0066] Step 6: Save the updated LPC coefficients This is used for the next packet loss reconstruction.

[0067] Step 7: The current frame ends. Repeat the entire process from step 1 to process the next frame of data.

[0068] Based on the aforementioned speech packet loss concealment method, an echo detection and cancellation mechanism is introduced to specifically address the core problem of echo generation in the transient segment of traditional PLCs, resulting in cleaner and more natural reconstructed speech and significantly improved speech quality. Simultaneously, echo detection using correlation peak analysis adaptively determines the state of each frame of signal, initiating the cancellation process only when necessary, avoiding unnecessary processing of non-echo segments and ensuring algorithm efficiency. This method also integrates with and optimizes the traditional PLC process, providing two optional cancellation paths: traditional adaptive filtering and modern neural networks. The choice can be flexibly made according to different system requirements for performance, complexity, and real-time performance, making it widely applicable. Furthermore, by continuously looping and updating LPC coefficients, the packet loss concealment system can dynamically adapt to changing speech content and acoustic environment, maintaining stable high-quality output in continuous communication.

[0069] In the description of this specification, references to terms such as "some possible implementations," "some implementations," "example," "specific example," or "some examples" indicate that a specific feature, structure, material, or characteristic described in connection with that implementation or example is included in at least one implementation or example of this disclosure, and the aforementioned terms do not necessarily refer to the same implementation or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more implementations or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different implementations or examples described in this specification, as well as the features of different implementations or examples.

[0070] Regarding the method flowcharts of embodiments of this disclosure, certain operations are described as different steps performed in a certain order. Such flowcharts are illustrative and not restrictive. Some steps described herein may be grouped together and performed in a single operation, or some steps may be divided into multiple sub-steps, and some steps may be performed in an order different from that shown herein. The various steps shown in the flowcharts may be implemented in any way by any circuit structure and / or tangible mechanism (e.g., software running on a computer device, hardware (e.g., logic functions implemented by a processor or chip), and / or any combination thereof).

[0071] Those skilled in the art will understand that in the methods described in the above specific embodiments, the order in which the steps are written does not imply a strict execution order, and the specific execution order of each step should be determined by its function and possible internal logic.

[0072] Based on the same inventive concept, this disclosure also provides a voice packet loss hiding device corresponding to the voice packet loss hiding method. Since the principle of the device in this disclosure is similar to the voice packet loss hiding method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0073] Reference Figure 3 The diagram shown is a schematic of a voice packet loss concealment device provided in an embodiment of this disclosure. The device includes: a framing module 201, a calculation module 202, a detection module 203, and an update module 204; wherein, The framing module 201 is used to perform framing processing on the received voice signal to obtain the current frame signal and the historical reference frame signal. Calculation module 202 is used to calculate LPC coefficients based on the current frame signal through linear predictive coding (LPC) analysis. The detection module 203 is used to perform echo detection between the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal; The update module 204 is used to start echo cancellation processing if an echo is detected, and update the LPC coefficients based on the echo cancellation result to obtain the updated LPC coefficients; the updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

[0074] The aforementioned speech packet loss concealment device first performs frame segmentation on the received speech signal to obtain the current frame signal and historical reference frame signals. Then, it calculates the LPC coefficients through Linear Predictive Coding (LPC) analysis. Furthermore, it performs echo detection on the current frame signal and the historical reference frame signals to determine if an echo exists in the current frame signal. If an echo is detected, echo cancellation processing is initiated, and the LPC coefficients are updated based on the echo cancellation results. The updated LPC coefficients are then used for frame reconstruction during packet loss. This disclosure addresses the core problem of echo generation in transient segments due to the limitations of the LPC model by detecting and eliminating echo components in the reconstructed speech in real time. This results in cleaner and more natural reconstructed speech, significantly improving speech quality.

[0075] In one possible implementation, the device further includes: Output module 205 is used to output the calculated LPC coefficients if it is determined that there is no echo.

[0076] In one possible implementation, the detection module 203 is specifically configured to perform echo detection between the current frame signal and the historical reference frame signal according to the following steps: Calculate the correlation peaks between the current frame signal and multiple historical reference frame signals; The presence of an echo is determined based on the peak value of the relevant peak.

[0077] In one possible implementation, the detection module 203 is specifically configured to calculate the correlation peak between the current frame signal and multiple historical reference frame signals according to the following steps: Calculate the normalized cross-correlation function between the current frame signal and each historical reference frame signal; The local maximum peak value is extracted from each normalized cross-correlation function as the correlation peak of the corresponding historical reference frame.

[0078] In one possible implementation, the detection module 203 is specifically configured to determine whether an echo exists based on the peak value of the relevant peak according to the following steps: Determine the global maximum peak value from all relevant peaks; If the global maximum peak value is greater than the preset peak value threshold, it is determined that an echo exists, and the historical reference frame signal corresponding to the global maximum peak value is determined as the echo reference signal.

[0079] In one possible implementation, upon determining that an echo is present, the update module 204 is used to initiate echo cancellation processing in one of the following ways: A traditional adaptive filtering algorithm is used for echo cancellation to update the LPC coefficients; A neural network model is used for echo cancellation to update the LPC coefficients.

[0080] In one possible implementation, the update module 204 is specifically configured to update the LPC coefficients by performing echo cancellation processing using a conventional adaptive filtering algorithm according to the following steps: Based on the echo reference signal and the current frame signal, the adaptive filter coefficients are iteratively updated using the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm. The current frame signal is filtered using the updated filter coefficients to obtain the de-echo signal; Based on the de-echo signal, LPC analysis is performed again to generate updated LPC coefficients.

[0081] In one possible implementation, the update module 204 is specifically configured to update the LPC coefficients by performing echo cancellation processing using a neural network model according to the following steps: The LPC coefficients calculated based on the current frame signal are input into the pre-trained neural network model; the neural network model is used to learn the mapping relationship between the echo-containing LPC coefficients and the de-echoed LPC coefficients. The updated LPC coefficients are directly output by the neural network model.

[0082] In one possible implementation, the device executes in a loop, frame by frame.

[0083] It should be noted that the apparatus in this embodiment can implement the various processes of the aforementioned method and achieve the same effects and functions, which will not be elaborated here.

[0084] This disclosure also provides an electronic device, such as... Figure 4 The diagram shown is a schematic representation of an electronic device structure provided in this embodiment of the present disclosure, including: a processor 301, a memory 302, and a bus 303. The memory 302 stores machine-readable instructions executable by the processor 301 (e.g., ...). Figure 3 The device includes the framing module 201, calculation module 202, detection module 203, and update module 204 (and their corresponding execution instructions). When the electronic device is running, the processor 301 communicates with the memory 302 via the bus 303. When a machine-readable instruction is executed by the processor 301, the following processing is performed: The received audio signal is processed by framing to obtain the current frame signal and the historical reference frame signal; Based on the current frame signal, LPC coefficients are calculated through linear predictive coding (LPC) analysis. Perform echo detection on the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal; If an echo is detected, echo cancellation is initiated, and the LPC coefficients are updated based on the echo cancellation results to obtain the updated LPC coefficients. The updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

[0085] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the voice packet loss concealment method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0086] This disclosure also provides a computer program product carrying program code. The program code includes instructions that can be used to execute the steps of the voice packet loss hiding method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0087] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0088] The various embodiments in this disclosure are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments. In particular, the description of the apparatus, device, and computer-readable storage medium embodiments is simplified because they are basically similar to the method embodiments, and the relevant parts can be referred to the description of the method embodiments.

[0089] The apparatus, device, and computer-readable storage medium provided in this disclosure correspond one-to-one with the method. Therefore, the apparatus, device, and computer-readable storage medium also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the apparatus, device, and computer-readable storage medium will not be repeated here.

[0090] Those skilled in the art will understand that embodiments of this disclosure can be implemented as methods and apparatus (devices or systems), or as computer-readable storage media. Therefore, this disclosure can be implemented entirely in hardware, entirely in software, or in a combination of software and hardware. Furthermore, this disclosure can be implemented as a computer-readable storage medium on one or more computer-readable storage media containing computer-usable program code (including, but not limited to, disk storage, read-only optical disc storage (CD-ROM), optical storage, etc.).

[0091] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices or systems), and computer-readable storage media according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or block diagrams.

[0092] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article including instruction means, wherein the instruction means implement the functions specified in one or more flowcharts and / or one or more blocks in a block diagram.

[0093] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more blocks in the block diagram.

[0094] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0095] Memory can include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0096] Computer-readable media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory, read-only memory, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. Furthermore, although the operations of the methods of this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally, certain steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple sub-steps.

[0097] While the spirit and principles of this disclosure have been described above with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A method for hiding voice packet loss, characterized in that, include: The received audio signal is processed by framing to obtain the current frame signal and the historical reference frame signal; Based on the current frame signal, LPC coefficients are calculated using linear predictive coding (LPC) analysis. Echo detection is performed between the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal. If an echo is detected, echo cancellation processing is initiated, and the LPC coefficients are updated based on the echo cancellation result to obtain the updated LPC coefficients. The updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

2. The method according to claim 1, characterized in that, The method further includes: If it is determined that there is no echo, the calculated LPC coefficients are output.

3. The method according to claim 1, characterized in that, The step of performing echo detection on the current frame signal and the historical reference frame signal includes: Calculate the correlation peaks between the current frame signal and the multiple historical reference frame signals; The presence of an echo is determined based on the peak value of the relevant peak.

4. The method according to claim 3, characterized in that, The calculation of the correlation peak between the current frame signal and the plurality of historical reference frame signals includes: Calculate the normalized cross-correlation function between the current frame signal and each historical reference frame signal respectively; The local maximum peak value is extracted from each of the normalized cross-correlation functions as the correlation peak of the corresponding historical reference frame.

5. The method according to claim 4, characterized in that, The determination of whether an echo exists based on the peak value of the relevant peaks includes: Determine the global maximum peak value from all relevant peaks; If the global maximum peak value is greater than the preset peak value threshold, it is determined that an echo exists, and the historical reference frame signal corresponding to the global maximum peak value is determined as the echo reference signal.

6. The method according to claim 5, characterized in that, If an echo is detected, initiate echo cancellation processing using one of the following methods: The echo cancellation process is performed using a traditional adaptive filtering algorithm to update the LPC coefficients; The echo cancellation process is performed using a neural network model to update the LPC coefficients.

7. The method according to claim 6, characterized in that, The process of using a traditional adaptive filtering algorithm to perform echo cancellation processing to update the LPC coefficients includes: Based on the echo reference signal and the current frame signal, the adaptive filter coefficients are iteratively updated using the Least Mean Square (LMS) algorithm or the Normalized Least Mean Square (NLMS) algorithm. The current frame signal is filtered using the updated filter coefficients to obtain the de-echo signal; Based on the de-echo signal, LPC analysis is performed again to generate the updated LPC coefficients.

8. The method according to claim 6, characterized in that, The step of using a neural network model to perform echo cancellation processing to update the LPC coefficients includes: The LPC coefficients calculated based on the current frame signal are input into a pre-trained neural network model; the neural network model is used to learn the mapping relationship between echo-containing LPC coefficients and de-echoed LPC coefficients. The updated LPC coefficients are directly output by the neural network model.

9. The method according to any one of claims 1 to 8, characterized in that, The method is executed cyclically in units of frames.

10. A voice packet loss concealment device, characterized in that, include: The framing module is used to perform framing processing on the received voice signal to obtain the current frame signal and the historical reference frame signal. The calculation module is used to calculate LPC coefficients based on the current frame signal through linear predictive coding (LPC) analysis. The detection module is used to perform echo detection on the current frame signal and the historical reference frame signal to determine whether there is an echo in the current frame signal; An update module is used to initiate echo cancellation processing if an echo is detected, and update the LPC coefficients based on the echo cancellation result to obtain updated LPC coefficients; the updated LPC coefficients are used for frame reconstruction when subsequent speech packet loss occurs.

11. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the voice packet loss concealment method as described in any one of claims 1 to 9 is performed.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the voice packet loss concealment method as described in any one of claims 1 to 9.

Images