A Packet Loss Compensation Method and Device for Adapting Window Switching in Advanced Audio Coding

Abstract

The present invention relates to the technical field of audio encoding and transmission, and particularly relates to a packet loss compensation method and device for adapting window switching in advanced audio encoding. The method includes: determining a packet loss frame; when the packet loss frame is a single data frame, selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame; determining the window sequence of the packet loss frame according to the window sequence of the previous frame determined by decoding and the window sequence of the next frame, where the next frame is the next frame of the packet loss frame; determining the overlapping data of the packet loss frame transmitted to the next frame according to the window sequence of the packet loss frame and the predicted data of the packet loss frame, completing the packet loss compensation, and the overlapping data is used for determining the output data of the next frame. Thus, this packet loss compensation method compensates the packet loss frame through the previous frame and the next frame, has a very smooth effect during window switching, and uses the overlapping data to minimize the loss of information volume.

Description

Technical Field

[0001] The present invention relates to the technical field of audio coding and transmission, and particularly to a packet loss compensation method and apparatus for adapting window switching in advanced audio coding. Background Art

[0002] In an audio transmission system, especially for the transmission of audio data streams via the Internet or wireless networks, packet loss problems often occur. These lost packets will cause discontinuity of the audio signal, thus affecting the overall quality of the audio. Advanced Audio Coding (AAC) is a widely used audio coding standard, which provides high-quality audio through an efficient compression algorithm. However, during the transmission process, the AAC-encoded audio data also faces the problem of packet loss.

[0003] The AAC encoder adopts a frame structure with a fixed frame length. Each frame requires overlapping data from the previous frame during decoding and updates the overlapping data for the next frame. There are mainly two types of window functions used in AAC coding: long window and short window. The long window is usually used for stable audio signals, while the short window is used to process transient signals. This results in four window types at the frame connection: long window - long window, long window - short window, short window - long window, and short window - short window.

[0004] Traditional packet loss compensation methods usually include techniques such as interpolation, waveform substitution, and time domain extension. However, when the audio signal switches between the long window and the short window, the effects of these methods are often not ideal. The switching between the long window and the short window is mainly used to adapt to different types of audio signals. At the window switching point, the frequency and time resolution of the audio signal will change significantly, posing a greater challenge to packet loss compensation. In particular, since each frame depends on the overlapping data of the previous frame during decoding and needs to update the overlapping data for the next frame, these factors need to be particularly considered when dealing with packet loss compensation, otherwise it will lead to poor compensation effects and even introduce more audio artifacts. Summary of the Invention

[0005] In view of this, the present invention provides a packet loss compensation method and apparatus for adapting window switching in advanced audio coding to solve the problem of audio artifacts existing in the prior art packet loss compensation methods.

[0006] In a first aspect, the present invention provides a packet loss compensation method for adapting window switching in advanced audio coding. The method includes: determining a packet loss frame; when the packet loss frame is a single data frame, selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame; determining the window sequence of the packet loss frame according to the window sequence of the previous frame determined by decoding and the window sequence of the next frame; determining the overlapping data of the packet loss frame transmitted to the next frame according to the window sequence of the packet loss frame and the predicted data of the packet loss frame, completing the packet loss compensation, and the overlapping data is used for determining the output data of the next frame.

[0007] The packet loss compensation method for adapting window switching in advanced audio coding provided by the embodiment of the present invention determines the frequency points for predicting the packet loss frame according to the window sequence of the previous frame, realizes the prediction of the data of the packet loss frame, and determines the window sequence of the packet loss frame according to the window sequence of the previous frame and the window sequence of the next frame, so as to realize the determination of the overlapping data of the packet loss frame transmitted to the next frame. Thus, this packet loss compensation method compensates the packet loss frame through the previous frame and the next frame, has a very smooth effect during window switching, and uses the overlapping data to minimize the loss of information.

[0008] In an optional implementation manner, selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame includes: obtaining the data to be compensated for packet loss including the packet loss frame; determining whether the data to be compensated for packet loss is regular data or irregular data according to whether the pitch of the data to be compensated for packet loss is calculated; if it is regular data, adopting a time-domain compensation algorithm, and selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame; if it is irregular data, adopting a frequency-domain compensation algorithm, and selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame.

[0009] In this embodiment, the data is determined to be regular or irregular by calculating the pitch value, and time-domain compensation and frequency-domain compensation are respectively used for packet loss compensation. Thus, the information amount can be maximally utilized, and the accuracy and smoothness can be improved.

[0010] In an optional implementation manner, the window sequence includes a long window sequence and a short window sequence. Adopting a time-domain compensation algorithm, selecting frequency points for prediction according to the window sequence of the previous frame of the packet loss frame to predict the data of the packet loss frame includes: when the window sequence of the previous frame of the packet loss frame is a long window sequence, using the time-domain compensation algorithm and all the frequency points in the frequency domain of the previous frame to predict the data of the packet loss frame; when the window sequence of the previous frame of the packet loss frame is a short window sequence, using the time-domain compensation algorithm and the frequency points in the last one-eighth of the frequency domain of the previous frame to predict the data of the packet loss frame.

[0011] In this embodiment, when time-domain compensation is adopted, different frequency points are selected for prediction according to the window sequence of the previous frame. Among them, when the window sequence of the previous frame of the packet-loss frame is a long window sequence, the time-domain compensation algorithm and the frequency points of all frequency domains of the previous frame are used to predict the packet-loss frame data; when the window sequence of the previous frame of the packet-loss frame is a short window sequence, the time-domain compensation algorithm and the frequency points of the last one-eighth frequency domain of the previous frame are used to predict the packet-loss frame data. Thus, accurate compensation of packet-loss data can be achieved through a smaller calculation method.

[0012] In an alternative embodiment, the window sequence includes a long window sequence and a short window sequence. When using the frequency-domain compensation algorithm, the frequency points for prediction are selected according to the window sequence of the previous frame of the packet-loss frame to predict the packet-loss frame data, including: when the window sequence of the previous frame of the packet-loss frame is a long window sequence, the frequency points of all frequency domains of the previous frame are subjected to sequential adjustment, attenuation, and frequency-domain compensation with long window coding to predict the packet-loss frame data; when the window sequence of the previous frame of the packet-loss frame is a short window sequence, the frequency points of the last one-eighth frequency domain of the previous frame are subjected to sequential adjustment, attenuation, and frequency-domain compensation with short window coding for a preset number of times to predict the packet-loss frame data.

[0013] In this embodiment, when frequency-domain compensation is adopted, different frequency points are selected for prediction according to the window sequence of the previous frame. When the window sequence of the previous frame of the packet-loss frame is a long window sequence, the frequency points of all frequency domains of the previous frame are subjected to sequential adjustment, attenuation, and long window coding to obtain the packet-loss frame data; when the window sequence of the previous frame of the packet-loss frame is a short window sequence, the frequency points of the last eight groups of frequency domains of the previous frame are respectively subjected to sequential adjustment, attenuation, and short window coding to obtain the packet-loss frame data. Thus, accurate compensation of packet-loss data can be achieved through a smaller calculation method.

[0014] In an alternative embodiment, if it is irregular data, after using the frequency-domain compensation algorithm to select the frequency points for prediction according to the window sequence of the previous frame of the packet-loss frame to predict the packet-loss frame data, the method further includes: if it is irregular data, smoothing the predicted packet-loss frame data and the overlapping data of the previous frame to obtain the smoothed packet-loss frame data.

[0015] In this embodiment, frequency-domain prediction is used for irregular data. The predicted data is usually discontinuous with the end of the previous frame. Therefore, it is smoothed with the overlapping data left by the previous frame to make the compensated data more coherent.

[0016] In an alternative embodiment, the window sequence includes a long window sequence and a short window sequence. To smooth the predicted lost packet frame data and the overlapping data of the previous frame to obtain the smoothed lost packet frame data, the following steps are included: when the window sequence of the previous frame of the lost packet frame is a long window sequence, perform a first windowing process on a preset number of data points at the front of the lost packet frame data and the overlapping data of the previous frame to obtain the smoothed lost packet frame data; when the window sequence of the previous frame of the lost packet frame is a short window sequence, use the time-domain data of a preset frequency point in the overlapping data of the previous frame as the first part of the data; perform a second windowing process on the first short window frequency domain in the lost packet frame data and the windowed coded data in the overlapping data of the previous frame to obtain the second part of the data; perform windowed superposition on the remaining short window frequency domain data in the lost packet frame data to obtain the third part of the data. The first part of the data, the second part of the data, and the third part of the data are combined in sequence to form the smoothed lost packet frame data.

[0017] In this embodiment, when smoothing the overlapping data of the lost packet frame obtained by frequency domain compensation and the previous frame, different smoothing methods are selected according to the window sequence of the previous frame, thus ensuring the effect of data smoothing processing.

[0018] In an alternative embodiment, the window sequence includes long window - long window, long window - short window, short window - long window, and short window - short window. Determine the window sequence of the lost packet frame according to the window sequence of the previous frame and the window sequence of the next frame determined by decoding, including: when the window sequence of the next frame of the lost packet frame is long window - long window or long window - short window and the window sequence of the previous frame of the lost packet frame is long window - long window or short window - long window, the determined window sequence of the lost packet frame is long window - long window; when the window sequence of the next frame of the lost packet frame is long window - long window or long window - short window and the window sequence of the previous frame of the lost packet frame is long window - short window or short window - short window, the determined window sequence of the lost packet frame is end window - long window; when the window sequence of the next frame of the lost packet frame is short window - short window or short window - long window and the window sequence of the previous frame of the lost packet frame is long window - long window or short window - long window, the determined window sequence of the lost packet frame is long window - short window; when the window sequence of the next frame of the lost packet frame is short window - short window or short window - long window and the window sequence of the previous frame of the lost packet frame is long window - short window or short window - short window, the determined window sequence of the lost packet frame is short window - short window.

[0019] In this embodiment, by means of advanced audio coding and decoding, determine the window sequence of the lost packet frame according to the window sequence of the previous frame and the window sequence of the next frame of the lost packet frame, thus providing a data basis for the subsequent determination of transmitting the lost packet frame to the overlapping data of the next frame.

[0020] In an alternative embodiment, determining the overlapping data for transmitting a lost packet frame to the next frame based on the window sequence of the lost packet frame and the predicted lost packet frame data includes: when the data to be compensated for lost packets is irregular data, windowing the lost packet frame data according to the window sequence of the lost packet frame to determine the overlapping data for transmitting the lost packet frame to the next frame; when the data to be compensated for lost packets is regular data, windowing and folding the lost packet frame data according to the window sequence of the lost packet frame and then windowing again to determine the overlapping data for transmitting the lost packet frame to the next frame.

[0021] In this embodiment, since the prediction methods for lost packet data of irregular data and regular data are different, therefore, according to the corresponding prediction method, combined with the window sequence of the lost packet frame, the determination of the overlapping data for transmitting the lost packet frame to the next frame is realized, thereby providing a data basis for determining the data output of the next frame of the lost packet frame.

[0022] In an alternative embodiment, the method further includes: when the lost packet frame includes multiple consecutive data frames, determining the window sequence of the first lost packet frame according to the window sequence of the frame preceding the first lost packet frame, and determining the overlapping data for transmitting the first lost packet frame to the second lost packet frame according to the window sequence of the first lost packet frame and the predicted data of the first lost packet frame; determining the window sequence of the second lost packet frame according to the window sequence of the first lost packet frame, and determining the overlapping data for transmitting the second lost packet frame to the third lost packet frame according to the window sequence of the second lost packet frame and the predicted data of the second lost packet frame; and so on, determining the overlapping data for transmitting each lost packet frame to the next frame until the last lost packet frame is compensated for lost packets according to the lost packet compensation method for a single data frame.

[0023] In this embodiment, when the lost packet frame includes multiple consecutive data frames, then the lost packet compensation for each lost packet frame can be sequentially performed starting from the first lost packet frame according to the window sequence of the previous frame until the next frame of the lost packet frame is a good packet, and then it is processed in the manner of the above-mentioned single lost packet frame, so as to realize the lost packet compensation for multiple consecutive data frames.

[0024] In a second aspect, the present invention provides a lost packet compensation device for adapting window switching in advanced audio coding. The device includes: a lost packet frame acquisition module for determining a lost packet frame; a prediction module for, when the lost packet frame is a single data frame, selecting frequency points for prediction according to the window sequence of the frame preceding the lost packet frame to predict the data of the lost packet frame; a window sequence determination module for determining the window sequence of the lost packet frame according to the window sequence of the previous frame determined by decoding and the window sequence of the next frame; and an update module for determining the overlapping data for transmitting the lost packet frame to the next frame according to the window sequence of the lost packet frame and the predicted data of the lost packet frame, completing the lost packet compensation, and the overlapping data is used for determining the output data of the next frame.

[0025] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the packet loss compensation method for adapting window switching in advanced audio coding according to the first aspect or any corresponding embodiment thereof.

[0026] In a fourth aspect, the present invention provides a computer-readable storage medium, on which computer instructions are stored. The computer instructions are used to cause a computer to perform the packet loss compensation method for adapting window switching in advanced audio coding according to the first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 is a schematic flowchart of the packet loss compensation method for adapting window switching in advanced audio coding according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of different window sequence types according to an embodiment of the present invention;

[0030] Figure 3 is a schematic flowchart of another packet loss compensation method for adapting window switching in advanced audio coding according to an embodiment of the present invention;

[0031] Figure 4 is a structural block diagram of the packet loss compensation device for adapting window switching in advanced audio coding according to an embodiment of the present invention;

[0032] Figure 5 is a schematic hardware structure diagram of the computer device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0034] According to an embodiment of the present invention, there is provided an embodiment of a packet loss compensation method for adapting window switching in advanced audio coding. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0035] In this embodiment, there is provided a packet loss compensation method for adapting window switching in advanced audio coding, which can be used in electronic devices such as computers, mobile phones, and tablet computers. Figure 1 It is a flowchart of a packet loss compensation method for adapting window switching in advanced audio coding according to an embodiment of the present invention, as Figure 1 shown, the process includes the following steps:

[0036] Step S101, determine the lost packet frame.

[0037] Specifically, advanced audio coding is used to encode and compress audio data. The encoded and compressed data consists of multiple data frames, and each data frame is a frame structure with a fixed frame length. In this embodiment, a data frame with a fixed frame length of 1024 sampling points is obtained using a sampling rate of 48 kHz. However, the duration corresponding to 1024 sampling points is more than twenty milliseconds, and in more than twenty milliseconds, there may be a steady signal or a mutated signal. For a steady signal, a window function of the long window type is used for processing, and for a mutated signal, a window function of the short window type is used for processing. Therefore, in a data frame (which can also be called a data packet), as Figure 2 shown, there may be four window sequence types: long window - long window, long window - short window, short window - long window, and short window - short window.

[0038] Among them, for the three window sequences of long window - long window, long window - short window, and short window - long window, they are all encoded data packets containing 1024 points obtained by performing a Modified Discrete Cosine Transform (MDCT) on 2048 points of the original data, or encoded data packets containing 1024 points obtained by windowing, folding, and performing a Discrete Cosine Transform (DCT) on 2048 points. For the short window - short window window sequence, it is composed of 448 points at the front and back and 8 short windows with overlapping heads and tails in the middle. The 448 points are points that do not contain audio information, and can also be called zero points. Each short window is obtained by windowing 256 points, folding, and then using 128 - point DCT processing. Thus, the data of the short window - short window window sequence after encoding is data that has undergone 8 times of 256 - point MDCT, and after folding, it is 8 groups of 128 - point data stored in sequence.

[0039] Therefore, based on the differences in the four window sequence encoding methods, there are also certain differences in their decoding methods. For the three window sequences of long window - long window, long window - short window, and short window - long window, when decoding a data packet or data frame, the data packet is first decoded to obtain frequency domain data. After performing an inverse modified discrete cosine transform (IMDCT) on 2048 points of the frequency domain data and then unfolding it, that is, performing an inverse discrete cosine transform (IDCT) in reverse, the data after decoding the data packet or the current frame is obtained. However, this current frame data still needs to be superimposed with the overlapping data left by the previous frame to obtain the output data of the current frame. At the same time, the data after decoding the current frame also needs to leave overlapping data for the next frame to use. Therefore, for the convenience of data decoding, the data after each data decoding can be saved or cached, so that the next frame of data decoding can directly obtain it from the cache.

[0040] For the window sequence of short window - short window, when decoding it, the data packet is also first decoded to obtain frequency domain data. However, since each short window in this data packet is processed by adding a 256 - point window, folding it, and then using a 128 - point DCT, during decoding, the 256 - point frequency domain data is unfolded after performing an inverse modified discrete cosine transform (IMDCT) to obtain the data after decoding the data packet or the current frame. The subsequent processing process is the same as that of the other three window sequences and will not be elaborated here.

[0041] In addition, according to the different window sequences, the way of superimposing the overlapping data of the current frame and the previous frame is also different. The specific superimposing method is long window with long window, and short window with short window. For example, if the window sequence of the previous frame is long window - long window, then the current frame can only be long window - long window or long window - short window. That is, the long window of the overlapping data left by the previous frame (the overlapping data is the second half of the entire data frame) is superimposed with the first - half long window of the current frame, and the second - half data of the current frame is used as the overlapping data and superimposed with the next frame. Another example is that if the window sequence of the previous frame is short window - short window, then the current frame can only be short window - short window or short window - long window.

[0042] Since in the process of audio data transmission, the AAC-encoded audio data also faces the problem of packet loss. Specifically, the lost packet frames can be determined by means of sequence number detection. Among them, in the process of audio data transmission, each data packet is assigned a unique sequence number for recombination and recovery after the audio data is received. In addition, when the sequence numbers are assigned to the data packets, the sequence numbers of the audio data increase continuously. If the situation of discontinuous or missing sequence numbers is detected, it indicates that packet loss has occurred. Therefore, the specific lost packet frames can be determined according to the sequence packets.

[0043] Step S102, when the lost packet frame is a single data frame, select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the data of the lost packet frame. Specifically, after the lost packet frame is determined, it is necessary to confirm whether a single data frame is lost or multiple consecutive data frames are lost. When a single data frame is lost, there is no packet loss in the previous frame and the next frame of the lost packet frame, that is, the previous frame and the next frame are good packets. Therefore, packet loss compensation can be performed based on the previous frame and the next frame.

[0044] Specifically, when performing packet loss compensation for the lost packet frame, it is first necessary to predict the data of the lost packet frame (hereinafter referred to as the current frame). In this embodiment, the data of the previous frame is selected to predict the data of the current frame. Since the encoding and decoding methods of different window sequences are different, the frequency points for prediction are selected according to the window sequence of the previous frame for the prediction of the current frame. According to the encoding and decoding methods adopted above, there are differences in the encoding and decoding between the three window sequences of long window - long window, long window - short window, and short window - long window (hereinafter simply referred to as long window sequences) and the window sequence of short window - short window (hereinafter simply referred to as short window sequence). Therefore, different frequency points can be selected for prediction for the long window sequence and the short window sequence.

[0045] Step S103, determine the window sequence of the lost packet frame according to the window sequences of the previous frame and the next frame determined by decoding. Specifically, since in advanced audio coding, the overlapping data left by the current frame needs to be transmitted to the next frame to facilitate the determination of the output data of the next frame, after the data of the lost packet frame is predicted, it is also necessary to determine the overlapping data of the lost packet frame transmitted to the next frame, and this overlapping data will be saved in the buffer for the next frame to obtain when determining the output data. Before the update, the window sequence of the lost packet frame can be determined first. Specifically, according to the above-mentioned superposition method of the superposed data during decoding, the window sequence of the current frame can be determined from the window sequences of the previous frame and the next frame.

[0046] Step S104: Determine the overlapping data for transmitting the lost packet frame to the next frame based on the window sequence of the lost packet frame and the predicted lost packet frame data, complete the lost packet compensation, and the overlapping data is used to determine the output data of the next frame. Specifically, after determining the window sequence of the lost packet frame, the overlapping data for transmitting the lost packet frame to the next frame can be determined according to the encoding and decoding method corresponding to the window sequence, and the compensation for the lost packet frame is completed.

[0047] The lost packet compensation method for adapting window switching in advanced audio coding provided by the embodiments of the present invention determines the frequency points for predicting the lost packet frame according to the window sequence of the previous frame, realizes the prediction of the lost packet frame data, and determines the window sequence of the lost packet frame according to the window sequence of the previous frame and the window sequence of the next frame, so as to realize the determination of the overlapping data for transmitting the lost packet frame to the next frame. Thus, this lost packet compensation method compensates for the lost packet frame through the previous frame and the next frame, has a very smooth effect during window switching, and uses the overlapping data to minimize the loss of information.

[0048] In this embodiment, a lost packet compensation method for adapting window switching in advanced audio coding is provided, and the process includes the following steps:

[0049] Step S201: Determine the lost packet frame. For details, please refer to Figure 1 Step S101 of the embodiment shown, which will not be elaborated here.

[0050] Step S202: When the lost packet frame is a single data frame, select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the lost packet frame data.

[0051] Specifically, the above step S202 includes:

[0052] Step S2021: Obtain the data to be compensated for lost packets including the lost packet frame; the data to be compensated for lost packets includes the determined lost packet frame, that is, the data to be compensated for lost packets includes the lost packet frame and multiple data frames before and after the lost packet frame.

[0053] Step S2022: Determine whether the data to be compensated for lost packets is regular data or irregular data according to whether the pitch of the data to be compensated for lost packets is calculated. Specifically, algorithms in related technologies can be used to calculate the pitch value, such as the autocorrelation method, cepstrum method, short-time amplitude difference method, etc. When using the autocorrelation method, the autocorrelation calculation can be performed on the data to be compensated for lost packets to obtain the autocorrelation value, and then the autocorrelation value is compared with a pre-set threshold. When the autocorrelation value is greater than the threshold, the autocorrelation value is used as the pitch value, that is, the pitch value is calculated; when the autocorrelation value is less than or equal to the threshold, it means that the pitch value is not calculated, and the pitch value is recorded as 0.

[0054] Specifically, when the pitch value is not 0, it indicates that the data to be packet loss compensated is regular data; when the pitch value is 0, it indicates that the data to be packet loss compensated is irregular data.

[0055] Step S2024, if it is regular data, adopt the time-domain compensation algorithm, and select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the lost packet frame data.

[0056] Step S2025, if it is irregular data, adopt the frequency-domain compensation algorithm, and select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the lost packet frame data.

[0057] Specifically, by detecting whether the data to be packet loss compensated is regular, time-domain prediction and frequency-domain prediction are respectively used for compensation processing. Among them, for relatively regular signals, the time-domain compensation technology can restore the lost data in the time domain through methods such as linear predictive coding (LPC) filtering; while for relatively irregular signals, the frequency-domain compensation technology can achieve more effective compensation in the frequency domain through spectrum analysis and reconstruction. Thus, different processing strategies are adopted for different data to ensure smooth transition of the compensated signal, reduce artifacts and discontinuities.

[0058] In an optional implementation manner, the above step S2024 specifically includes:

[0059] Step a1, when the window sequence of the previous frame of the lost packet frame is a long window sequence, adopt the time-domain compensation algorithm and the frequency points of the entire previous frame frequency domain to predict the lost packet frame data.

[0060] Step a2, when the window sequence of the previous frame of the lost packet frame is a short window sequence, adopt the time-domain compensation algorithm and the frequency points of the last eighth of the previous frame frequency domain to predict the lost packet frame data.

[0061] Specifically, when the window sequence of the previous frame is a long window sequence (including long window-long window, long window-short window, and short window-long window), all points are used for encoding and decoding. Therefore, when the previous frame is a long window sequence, the frequency points of the entire previous frame frequency domain are used for data prediction. When the window sequence of the previous frame is a short window sequence (including short window-short window), the encoded data is the data of 256-point MDCT done 8 times, and after folding, it is 8 groups of 128-point data stored in order. Therefore, for this type, only the frequency points of the last eighth of the previous frame frequency domain (the last eighth of the frequency domain is a complete short window frequency domain) can be selected for prediction. When predicting, linear predictive coding or Burg prediction algorithm can be used for prediction based on the selected frequency points. The specific prediction method can refer to the specific processing flow of relevant algorithms and will not be elaborated here. Finally, the time-domain data containing 2048 points is obtained through prediction.

[0062] In an alternative embodiment, step S2025 specifically includes:

[0063] Step b1, when the window sequence of the previous frame of the lost packet frame is a long window sequence, perform sequential adjustment, attenuation, and frequency-domain compensation of long window coding on the frequency points in the entire frequency domain of the previous frame, and predict the lost packet frame data.

[0064] Step b2, when the window sequence of the previous frame of the lost packet frame is a short window sequence, perform sequential adjustment, attenuation, and frequency-domain compensation of short window coding on the frequency points in the last one-eighth of the frequency domain of the previous frame for a preset number of times, and predict the lost packet frame data.

[0065] Specifically, when the data to be compensated for lost packets is irregular data, the frequency points selected during prediction are the same as those when the data to be compensated for lost packets is regular data. That is, when the window sequence of the previous frame is a long window sequence, all the frequency points in the frequency domain are selected; when the window sequence of the previous frame is a short window sequence, the frequency points in the last one-eighth of the frequency domain of the previous frame are selected. However, the specific prediction methods are different. Among them, when all the frequency points in the frequency domain are selected, the selected frequency points are sequentially adjusted (such as randomly shuffling the data of each frequency point), then the adjusted data is attenuated, and then long window IDCT (that is, the decoding process of the long window sequence mentioned in step S101 above) is performed to predict the lost packet frame data. When predicting using the last one-eighth of the frequency domain of the previous frame, the selected frequency points are sequentially adjusted and attenuated 8 times, and then the attenuated data is subjected to 8 times of short window IDCT (that is, the decoding process of the short window sequence mentioned in step S101 above) to predict the lost packet frame data.

[0066] It should be noted that since the time-domain compensation for regular data is carried out immediately after the end of the previous frame, there is no need to smooth it with the previous frame. However, after the frequency-domain prediction of irregular data, the data is usually discontinuous with the end of the previous frame. Therefore, it is necessary to smooth it with the overlapping data left by the previous frame.

[0067] Thus, after step S2025, it further includes:

[0068] Step S2026, if it is irregular data, smooth the predicted lost packet frame data and the overlapping data of the previous frame to obtain the smoothed lost packet frame data.

[0069] Specifically, step S2026 includes:

[0070] Step c1, when the window sequence of the previous frame of the packet loss frame is a long window sequence, perform first windowing processing on the data points of a preset number before the packet loss frame data and the overlapping data of the previous frame to obtain the smoothed packet loss frame data. Specifically, when the window sequence of the previous frame is a long window sequence, the overlapping data of the previous frame refers to the data placed in the cache according to a certain layout position after the frequency domain data obtained by decoding the previous frame data packet passes through IDCT and expansion. Therefore, when the window sequence of the previous frame is a long window sequence, the first 512 points of the predicted packet loss frame data can be superimposed with the overlapping data of the previous frame using the Kaiser-Bessel Derived Window (KBD window) as the output of the actual first 512 points, thereby achieving a good smoothing effect.

[0071] Step c2, when the window sequence of the previous frame of the packet loss frame is a short window sequence, use the time domain data of the preset frequency points in the overlapping data of the previous frame as the first part of the data; specifically, when the window sequence of the previous frame is a short window sequence, the remaining overlapping data of the previous frame includes 448 points of time domain data and 128 points of IDCT data. Use these 448 points of time domain data as the data of the first 448 points of the smoothed packet loss frame data, that is, the first part of the smoothed packet loss frame data.

[0072] Step c3, perform second windowing processing on the first short window frequency domain in the packet loss frame data and the windowed encoded data in the overlapping data of the previous frame to obtain the second part of the data. Specifically, for irregular data, when the window sequence of the previous frame of the packet loss frame is a short window sequence, it is obtained by performing 8 times of short window IDCT, that is, it contains 8 groups of 128-point data. At this time, the first 128-point short window frequency domain can be selected to be superimposed with the 128-point IDCT data in the overlapping data left by the previous frame using the Sine Window to obtain the second part of the smoothed packet loss frame data.

[0073] Step c4, perform windowing and superposition on the remaining short window frequency domain data in the packet loss frame data to obtain the third part of the data. The first part of the data, the second part of the data, and the third part of the data are combined in sequence to form the smoothed packet loss frame data. Specifically, since the predicted packet loss frame data contains 8 groups of 128-point data, and the first 128-point short window frequency domain is used during superposition, the remaining 7 short window frequency domains can be windowed and superimposed later to obtain 448 points of time domain data in the third part. Thus, the smoothed packet loss frame data is composed of 448 points of data in the first part, 128 points of data in the second part, and 448 points of data in the third part.

[0074] Step S203, determine the window sequence of the packet loss frame according to the window sequence of the previous frame and the window sequence of the next frame determined by decoding; for details, please refer to Figure 1Step S103 of the illustrated embodiment will not be elaborated here.

[0075] Step S204, determine the overlapping data of the lost packet frame transmitted to the next frame according to the window sequence of the lost packet frame and the predicted lost packet frame data, complete the lost packet compensation, and the overlapping data is used to determine the output data of the next frame. For details, please refer to Figure 1 Step S104 of the illustrated embodiment will not be elaborated here.

[0076] In this embodiment, a lost packet compensation method for adapting window switching in advanced audio coding is provided. The method includes the following steps:

[0077] Step S301, determine the lost packet frame; for details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be elaborated here.

[0078] Step S302, when the lost packet frame is a single data frame, select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the lost packet frame data; for details, please refer to Figure 1 Step S102 of the illustrated embodiment will not be elaborated here.

[0079] Step S303, determine the window sequence of the lost packet frame according to the window sequence of the previous frame determined by decoding and the window sequence of the next frame.

[0080] Specifically, the above step S303 includes:

[0081] Step S3031, when the window sequence of the next frame of the lost packet frame is long window - long window or long window - short window and the window sequence of the previous frame of the lost packet frame is long window - long window or short window - long window, the determined window sequence of the lost packet frame is long window - long window.

[0082] Step S3032, when the window sequence of the next frame of the lost packet frame is long window - long window or long window - short window and the window sequence of the previous frame of the lost packet frame is long window - short window or short window - short window, the determined window sequence of the lost packet frame is end window - long window.

[0083] Step S3033, when the window sequence of the next frame of the lost packet frame is short window - short window or short window - long window and the window sequence of the previous frame of the lost packet frame is long window - long window or short window - long window, the determined window sequence of the lost packet frame is long window - short window.

[0084] Step S3034, when the window sequence of the next frame of the lost packet frame is short window - short window or short window - long window and the window sequence of the previous frame of the lost packet frame is long window - short window or short window - short window, the determined window sequence of the lost packet frame is short window - short window.

[0085] Specifically, when updating the cache, it is necessary to determine the window sequence of the lost packet frame based on the window sequence of the previous frame and the window sequence of the next frame. Since the window sequence of the next frame needs to be determined after decoding the next frame, the window sequence of the lost packet frame can be determined after decoding the next frame and before obtaining the output data. The specific determination method refers to the superposition method, that is, the long window and the long window are superimposed, and the short window and the short window are superimposed.

[0086] In this embodiment, the four window sequences are defined as follows: 0 is long window - long window, 1 is long window - short window, 2 is short window - short window, and 3 is short window - long window. Thus, after decoding the next frame, if the window sequence is 0 or 1 and the window sequence of the previous frame of the lost packet frame is 0 or 3, the window sequence of the lost packet frame is long window - long window; if the window sequence of the next frame of the lost packet frame is 0 or 1 and the window sequence of the previous frame of the lost packet frame is 1 or 2, the window sequence of the lost packet frame is short window - long window; if the window sequence of the next frame of the lost packet frame is 2 or 3 and the window sequence of the previous frame of the lost packet frame is 0 or 3, the window sequence of the lost packet frame is long window - short window; if the window sequence of the next frame of the lost packet frame is 2 or 3 and the window sequence of the previous frame of the lost packet frame is 1 or 2, the window sequence of the lost packet frame is short window - short window.

[0087] Step S304, determine the overlapping data of the lost packet frame transmitted to the next frame according to the window sequence of the lost packet frame and the predicted lost packet frame data, complete the lost packet compensation, and the overlapping data is used to determine the output data of the next frame.

[0088] Specifically, the above step S304 includes:

[0089] S3041, when the data to be lost packet compensated is irregular data, window the lost packet frame data according to the window sequence of the lost packet frame to determine the overlapping data of the lost packet frame transmitted to the next frame; specifically, when the data to be lost packet compensated is irregular data, since the predicted lost packet frame data is the data after IDCT, the predicted lost packet frame data can be windowed and updated according to the determined window sequence of the lost packet frame to determine the overlapping data of the lost packet frame transmitted to the next frame. It should be noted that the windowing and updating method can refer to the processing method of good packets in advanced audio coding.

[0090] S3042, when the data to be lost packet compensated is regular data, window the lost packet frame data, fold it, and then window it again to determine the overlapping data of the lost packet frame transmitted to the next frame. Specifically, when the data to be lost packet compensated is regular data, the predicted data is the time-domain data obtained by time-domain compensation and there is no data after IDCT. Therefore, the lost packet frame data can be windowed, folded, and then windowed again to determine the overlapping data of the lost packet frame transmitted to the next frame.

[0091] In this embodiment, a lost packet compensation method adapted to window switching in advanced audio coding is provided. The method includes the following steps:

[0092] Step S401, determine the lost packet frame; for details, please refer to Figure 1 Step S101 of the embodiment shown, which will not be elaborated here.

[0093] Step S402, when the lost packet frame is a single data frame, select the frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the data of the lost packet frame; for details, please refer to Figure 1 Step S102 of the embodiment shown, which will not be elaborated here.

[0094] Step S403, determine the window sequence of the lost packet frame according to the window sequences of the previous frame and the next frame determined by decoding; for details, please refer to Figure 1 Step S103 of the embodiment shown, which will not be elaborated here.

[0095] Step S404, determine the overlapping data for the lost packet frame to be transmitted to the next frame according to the window sequence of the lost packet frame and the predicted data of the lost packet frame, complete the lost packet compensation, and the overlapping data is used to determine the output data of the next frame. For details, please refer to Figure 1 Step S104 of the embodiment shown, which will not be elaborated here.

[0096] Step S405, when the lost packet frame includes multiple consecutive data frames, determine the window sequence of the first lost packet frame according to the window sequence of the previous frame of the first lost packet frame, and determine the overlapping data for the first lost packet frame to be transmitted to the second lost packet frame according to the window sequence of the first lost packet frame and the predicted data of the first lost packet frame; specifically, when the lost packet frame includes multiple consecutive data frames, only the first lost packet frame has a previous frame (i.e., the previous frame is a good packet) and the last lost packet frame has a next frame (i.e., the next frame is a good packet), and the middle lost packet frames have neither a previous frame nor a next frame (i.e., both the previous frame and the next frame are lost packet frames). At this time, the lost packet compensation cannot be performed in the same way as when the lost packet frame is a single data frame, so a new lost packet compensation method needs to be determined.

[0097] Specifically, when the lost packet frame includes multiple consecutive data frames, the lost packet compensation can start from the first lost packet frame. The previous frame of the first lost packet frame is a good packet and the next frame is a lost packet, so the lost packet compensation is based on the previous frame of the lost packet frame. First, the frequency points for prediction can be selected according to the window sequence of the previous frame of the first lost packet frame in the same way as in Step S102 to predict the data of the first lost packet frame; then determine the window sequence of the first lost packet frame according to the window sequence of the previous frame. If the window sequence of the previous frame is a long window sequence, then determine the window sequence of the lost packet frame as long window - long window. If the window sequence of the previous frame is a short window sequence, then determine the window sequence of the lost packet frame as short window - short window. Then determine the overlapping data for the first lost packet frame to be transmitted to the second lost packet frame according to the window sequence of the first lost packet frame and the predicted data of the first lost packet frame in the same way as in Step S104.

[0098] Step S406: Determine the window sequence of the second lost packet frame according to the window sequence of the first lost packet frame, and determine the overlapping data from the second lost packet frame to the third lost packet frame according to the window sequence of the second lost packet frame and the predicted data of the second lost packet frame. Specifically, when the lost packet compensation for the first lost packet frame is completed, the first lost packet frame serves as the previous frame of the second lost packet frame, and the lost packet compensation for the second lost packet frame can be performed in a manner similar to that in step S405.

[0099] Step S407: And so on, determine the overlapping data from each lost packet frame to the next frame until the lost packet compensation for the last lost packet frame is performed according to the lost packet compensation method for a single data frame. Specifically, when the lost packet compensation for the second lost packet frame is completed, it serves as the previous frame of the third lost packet frame for the lost packet compensation of the third lost packet frame, and so on. Until the lost packet compensation for the last lost packet frame, at this time, its previous frame (the penultimate lost packet frame) and the next frame are both good packets, so the lost packet compensation for the lost packet frame as a single data frame can be performed by referring to steps S102 to S104 above. Thus, the lost packet compensation for multiple consecutive lost packet frames is completed.

[0100] As a specific application embodiment of the embodiment of the present invention, as Figure 3 shown, the lost packet compensation method for adapting window switching in advanced audio coding can be implemented according to the following process:

[0101] Step 1: Obtain audio data for lost packet detection. Specifically, lost packet detection can be performed according to the sequence numbers of the audio data. Normally, the sequence numbers of the audio data should be continuously increasing. If it is detected that the sequence numbers are discontinuous or missing, it is determined that a lost packet has occurred.

[0102] Step 2: When it is detected that a lost packet of a single data frame has occurred, determine the window sequence of the previous frame of the lost packet frame. Among them, the four window sequences are defined as follows: 0 is long window - long window, 1 is long window - short window, 2 is short window - short window, and 3 is short window - long window.

[0103] Step 3: Calculate the gene value (gene frequency) of the audio data. When the gene value is 0, that is, when the gene frequency cannot be detected, the audio data is regarded as an irregular signal; when the gene value is not 0, that is, when the gene frequency can be detected, the audio data is regarded as a regular signal.

[0104] Step 4: For regular signals, initialize the PLC model according to long and short windows, and then perform time-domain compensation using the time-domain compensation algorithm. Specifically, when the window sequence of the previous frame is 0, 1, 3, use all the frequency domains of the previous frame for prediction; if the window sequence is 2, use the last one-eighth of the frequency domain of the previous frame for prediction (since the short window - short window consists of 8 short windows, and the last one-eighth of the frequency domain is the frequency domain of a complete short window), and predict 2048 points (assuming a frame length of 1024 points at a sampling rate of 48 kHz + overlapping data for the next frame). Since the time-domain compensation is immediately after the end of the previous frame, there is no need to smooth it with the previous frame. Take the first 1024 points of the prediction as the output, and store the time-domain data of these 2048 points in the buffer.

[0105] Step 5: For irregular signals, use the frequency-domain compensation method. Specifically, when the window sequence of the previous frame is 0, 1, 3, use all the frequency domains of the previous frame for prediction, randomly shuffle and attenuate the frequency points, and then perform long-window IDCT; if the window sequence is 2, use the last one-eighth of the frequency domain of the previous frame for prediction, perform 8 random shuffles and attenuations, and perform 8 short-window IDCTs to predict the lost packet frame data.

[0106] Step 6: For irregular signals, the data after audio prediction is usually discontinuous with the end of the previous frame, so it is necessary to smooth it with the overlapping data left by the previous frame. Specifically, if the window sequence of the previous frame is 0, 1, 3, superimpose the first 512 points of the prediction data with the overlapping data left by the previous frame using the KBD window (Kaiser - Bessel Derived Window) as the output of the first 512 points of the actual lost packet frame, which can achieve a good smoothing effect. If the window sequence of the previous frame is 2, then there will be 448 points of time-domain data and 128 points of IDCT data in the overlapping data left by the previous frame. At this time, the first 448 points of the lost packet frame output data are the 448 points of this overlapping data. The next 128 points are the superposition of the 128 points of the frequency domain of the first short window and the 128 points of the overlapping data using the Sine Window. The subsequent 448 points of the output are the time-domain data obtained by successively windowing and superimposing the next short-window IDCT data, thereby obtaining the smoothed lost packet frame data.

[0107] Step 7: For the lost packet frame data, it is also necessary to determine the overlapping data transmitted to the next frame for use by the next frame. Since the window sequence of the next frame cannot be known in advance, when determining the overlapping data, it cannot be done in the current frame, but needs to be done after the next frame is unpacked and before output. Specifically, after the next frame of good packet is unpacked, if the window sequence is 0 or 1, and the window sequence of the previous frame of the lost packet frame is 0 or 3, then the window sequence of the lost packet frame is long window - long window; if the window sequence of the previous frame of the lost packet frame is 1 or 2, it means that the window sequence of the lost packet frame is short window - long window. If the window sequence after the next frame of good packet is unpacked is 2 or 3, and the window sequence of the previous frame of the lost packet frame is 0 or 3, it means that the window sequence of the lost packet frame is long window - short window; if the window sequence of the previous frame of the lost packet frame is 1 or 2, it means that the window sequence of the lost packet frame is short window - short window.

[0108] Step 8: For the unregular signal, when determining the overlapping data transmitted from the lost packet frame to the next frame, according to the window sequence of the lost packet frame judged in Step 7 and the data after IDCT obtained in Step 5, update the cache according to the window adding and placement method of the good packet, that is, determine the overlapping data transmitted from the lost packet frame to the next frame, and update this overlapping data to the buffer.

[0109] Step 9: For the regular signal, since the regular audio uses time domain compensation and there is no data after IDCT, it is necessary to fold according to the window sequence at the lost packet position to imitate the folding process before DCT during encoding, fold to obtain the IDCT data and then do window adding and folding to update the cache, that is, determine the overlapping data transmitted from the lost packet frame to the next frame, and update this overlapping data to the buffer.

[0110] Step 10: When the lost packet frame is multiple consecutive data frames, the lost packet compensation for each lost packet frame can be sequentially performed starting from the first lost packet frame according to the window sequence of the previous frame until the next frame of the lost packet frame is a good packet, and then process it according to the method of a single lost packet frame described above, so as to realize the lost packet compensation for multiple consecutive data frames.

[0111] In this embodiment, a lost packet compensation device adapted to window switching in advanced audio coding is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0112] This embodiment provides a lost packet compensation device adapted to window switching in advanced audio coding, as Figure 4 shown, including:

[0113] A lost packet frame acquisition module 31, configured to determine a lost packet frame;

[0114] A prediction module 32, configured to, when the lost packet frame is a single data frame, select frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the data of the lost packet frame;

[0115] A window sequence determination module 33, configured to determine the window sequence of the lost packet frame according to the window sequence of the previous frame and the window sequence of the next frame determined by decoding;

[0116] An update module 34, configured to determine the overlapping data for transmitting the lost packet frame to the next frame according to the window sequence of the lost packet frame and the predicted data of the lost packet frame, complete lost packet compensation, and the overlapping data is used to determine the output data of the next frame.

[0117] In an alternative embodiment, the prediction module includes: a data acquisition module, configured to acquire the data to be compensated for lost packets including the lost packet frame; a judgment module, configured to determine whether the data to be compensated for lost packets is regular data or irregular data according to whether the pitch of the data to be compensated for lost packets is calculated; a first prediction unit, configured to, if it is regular data, adopt a time-domain compensation algorithm, and select frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the data of the lost packet frame; a second prediction unit, configured to, if it is irregular data, adopt a frequency-domain compensation algorithm, and select frequency points for prediction according to the window sequence of the previous frame of the lost packet frame to predict the data of the lost packet frame.

[0118] In an alternative embodiment, the window sequence includes a long window sequence and a short window sequence. The first prediction unit is specifically configured to: when the window sequence of the previous frame of the lost packet frame is a long window sequence, adopt a time-domain compensation algorithm and the frequency points of the entire previous frame frequency domain to predict the data of the lost packet frame; when the window sequence of the previous frame of the lost packet frame is a short window sequence, adopt a time-domain compensation algorithm and the frequency points of the last one-eighth of the previous frame frequency domain to predict the data of the lost packet frame.

[0119] In an alternative embodiment, the window sequence includes a long window sequence and a short window sequence. The second prediction unit is specifically configured to: when the window sequence of the previous frame of the lost packet frame is a long window sequence, perform sequential adjustment, attenuation, and frequency-domain compensation with long window coding on the frequency points of the entire previous frame frequency domain to predict the data of the lost packet frame; when the window sequence of the previous frame of the lost packet frame is a short window sequence, perform sequential adjustment, attenuation, and frequency-domain compensation with short window coding on the frequency points of the last one-eighth of the previous frame frequency domain for a preset number of times to predict the data of the lost packet frame.

[0120] In an alternative embodiment, the apparatus further includes: a smoothing module, configured to, if it is irregular data, smooth the predicted data of the lost packet frame and the overlapping data of the previous frame to obtain the smoothed data of the lost packet frame.

[0121] In an alternative embodiment, the window sequence includes a long window sequence and a short window sequence. The smoothing module is specifically configured to: when the window sequence of the previous frame of the packet loss frame is a long window sequence, perform a first windowing process on a preset number of data points before the packet loss frame data and the overlapping data of the previous frame to obtain the smoothed packet loss frame data; when the window sequence of the previous frame of the packet loss frame is a short window sequence, use the time-domain data of the preset frequency points in the overlapping data of the previous frame as the first part of the data; perform a second windowing process on the first short window frequency domain in the packet loss frame data and the windowed encoded data in the overlapping data of the previous frame to obtain the second part of the data; perform windowed superposition on the remaining short window frequency domain data in the packet loss frame data to obtain the third part of the data. The first part of the data, the second part of the data, and the third part of the data are sequentially combined to form the smoothed packet loss frame data.

[0122] In an alternative embodiment, the window sequence includes long window - long window, long window - short window, short window - long window, and short window - short window. The window sequence determination module is specifically configured to: when the window sequence of the next frame of the packet loss frame is long window - long window or long window - short window and the window sequence of the previous frame of the packet loss frame is long window - long window or short window - long window, determine that the window sequence of the packet loss frame is long window - long window; when the window sequence of the next frame of the packet loss frame is long window - long window or long window - short window and the window sequence of the previous frame of the packet loss frame is long window - short window or short window - short window, determine that the window sequence of the packet loss frame is end window - long window; when the window sequence of the next frame of the packet loss frame is short window - short window or short window - long window and the window sequence of the previous frame of the packet loss frame is long window - long window or short window - long window, determine that the window sequence of the packet loss frame is long window - short window; when the window sequence of the next frame of the packet loss frame is short window - short window or short window - long window and the window sequence of the previous frame of the packet loss frame is long window - short window or short window - short window, determine that the window sequence of the packet loss frame is short window - short window.

[0123] In an alternative embodiment, the update module is specifically configured to: when the data to be compensated for packet loss is irregular data, window the packet loss frame data according to the window sequence of the packet loss frame to determine the overlapping data of the packet loss frame transmitted to the next frame; when the data to be compensated for packet loss is regular data, window the packet loss frame data according to the window sequence of the packet loss frame, fold it, and then window it to determine the overlapping data of the packet loss frame transmitted to the next frame.

[0124] In an alternative embodiment, the apparatus further includes: a consecutive packet loss compensation module, specifically configured to, when the lost packet frames include multiple consecutive data frames, determine the window sequence of the first lost packet frame according to the window sequence of the previous frame of the first lost packet frame, and determine the overlapping data from the first lost packet frame to the second lost packet frame according to the window sequence of the first lost packet frame and the predicted data of the first lost packet frame; determine the window sequence of the second lost packet frame according to the window sequence of the first lost packet frame, and determine the overlapping data from the second lost packet frame to the third lost packet frame according to the window sequence of the second lost packet frame and the predicted data of the second lost packet frame; and so on, determine the overlapping data from each lost packet frame to the next frame until the last lost packet frame is compensated for packet loss according to the packet loss compensation method for a single data frame.

[0125] The further function descriptions of the above-mentioned respective modules are the same as those in the corresponding embodiments above, and will not be elaborated here.

[0126] The embodiment of the present invention further provides a computer device having the above-mentioned Figure 4 packet loss compensation device for adapting window switching in advanced audio coding as shown.

[0127] Please refer to Figure 5 , Figure 5 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present invention. As shown in Figure 5 , the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (such as an array of servers, a set of blade servers, or a multi-processor system). Figure 5 In

[0128] Processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, processor 10 can further include a hardware chip. The above-mentioned hardware chip can be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof.

[0129] Among them, the memory 20 stores instructions that can be executed by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiments.

[0130] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of a computer device presented by a kind of mini-program landing page, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories can be connected to the computer device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0131] The memory 20 may include volatile memory, for example, random access memory; the memory may also include non-volatile memory, for example, flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memory.

[0132] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0133] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-temporary machine-readable storage medium and to be downloaded through a network and stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may further include a combination of the above types of memory. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by the computer, the processor, or the hardware, the method shown in the above embodiments is implemented.

[0134] A part of the present invention can be applied as a computer program product, for example, computer program instructions, which, when executed by a computer, can call or provide the methods and / or technical solutions according to the present invention through the operations of the computer. Those skilled in the art should understand that the forms of existence of computer program instructions in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways for computer program instructions to be executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.

[0135] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A packet loss compensation method for adaptive window switching in advanced audio coding, characterized in that: The method comprises: Determine the lost frames; When the packet loss frame is a single data frame, the frequency point used for prediction is selected according to the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data; Determine the window sequence of the packet loss frame according to the window sequence of the previous frame and the window sequence of the next frame determined by decoding, and the next frame is the next frame of the packet loss frame; The overlapping data of the packet loss frame transmitted to the next frame is determined according to the window sequence of the packet loss frame and the predicted packet loss frame data to complete the packet loss compensation. The overlapping data is used to determine the output data of the next frame.

2. The method according to claim 1, characterized in that The frequency point for prediction is selected according to the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data, including: Obtaining the packet loss compensation data including the packet loss frame; Determining whether the data to be compensated for packet loss is regular data or irregular data according to whether the fundamental pitch of the data to be compensated for packet loss is calculated; If the data is regular, a time domain compensation algorithm is used to select the frequency point for prediction based on the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data; If the data is irregular, a frequency domain compensation algorithm is used to select a frequency point for prediction based on the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data.

3. The method according to claim 2, characterized in that The window sequence includes a long window sequence and a short window sequence. A time domain compensation algorithm is used to select a frequency point for prediction according to the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data, including: When the window sequence of the previous frame of the packet loss frame is a long window sequence, the time domain compensation algorithm and the frequency points of all frequency domains of the previous frame are used to predict the packet loss frame data; When the window sequence of the previous frame of the packet loss frame is a short window sequence, the packet loss frame data is predicted using a time domain compensation algorithm and the frequency point of the frequency domain of one eighth of the previous frame.

4. The method according to claim 2, characterized in that: The window sequence includes a long window sequence and a short window sequence, and a frequency domain compensation algorithm is used to select a frequency point for prediction according to the window sequence of the previous frame of the packet loss frame to predict the packet loss frame data, including: When the window sequence of the previous frame of the packet loss frame is a long window sequence, the frequency points of all frequency domains of the previous frame are sequentially adjusted, attenuated, and frequency domain compensated by long window coding to predict the packet loss frame data; When the window sequence of the previous frame of the packet loss frame is a short window sequence, the frequency points of the one-eighth frequency domain after the previous frame are sequentially adjusted, attenuated, and frequency domain compensated by short window coding for a preset number of times to predict the packet loss frame data.

5. The method according to claim 2, characterized in that: If the data is irregular, a frequency domain compensation algorithm is used to select a frequency point for prediction according to a window sequence of a previous frame of the packet loss frame to predict the packet loss frame data, and the method further includes: If it is irregular data, the predicted packet loss frame data and the overlapping data of the previous frame are smoothed to obtain the smoothed packet loss frame data.

6. The method according to claim 5, characterized in that The window sequence includes a long window sequence and a short window sequence, and the predicted packet loss frame data and the overlapping data of the previous frame are smoothed to obtain the smoothed packet loss frame data, including: When the window sequence of the previous frame of the packet loss frame is a long window sequence, a preset number of data points before the packet loss frame data and the overlapping data of the previous frame are subjected to a first windowing process to obtain smoothed packet loss frame data; When the window sequence of the previous frame of the packet loss frame is a short window sequence, the time domain data of the preset frequency point in the overlapping data of the previous frame is used as the first part of the data; Performing a second windowing process on the first short window frequency domain in the packet loss frame data and the windowed coded data in the previous frame of overlapping data to obtain a second part of data; The remaining short window frequency domain data in the packet loss frame data is windowed and superimposed to obtain the third part of data, and the first part of data, the second part of data and the third part of data are sequentially combined to form the smoothed packet loss frame data.

7. The method according to claim 1, characterized in that The window sequence includes long window-long window, long window-short window, short window-long window and short window-short window. The window sequence of the packet loss frame is determined according to the previous frame window sequence and the next frame window sequence determined by decoding, including: When the window sequence of the next frame of the packet loss frame is long window-long window or long window-short window and the window sequence of the previous frame of the packet loss frame is long window-long window or short window-long window, the window sequence of the packet loss frame is determined to be long window-long window; When the window sequence of the next frame of the packet loss frame is long window-long window or long window-short window and the window sequence of the previous frame of the packet loss frame is long window-short window or short window-short window, the window sequence of the packet loss frame is determined to be end window-long window; When the window sequence of the next frame of the packet loss frame is short window-short window or short window-long window and the window sequence of the previous frame of the packet loss frame is long window-long window or short window-long window, the window sequence of the packet loss frame is determined to be long window-short window; When the window sequence of the next frame of the packet loss frame is short window-short window or short window-long window and the window sequence of the previous frame of the packet loss frame is long window-short window or short window-short window, the window sequence of the packet loss frame is determined to be short window-short window.

8. The method according to claim 2, characterized in that: The overlapping data of the packet loss frame to be transmitted to the next frame is determined according to the window sequence of the packet loss frame and the predicted packet loss frame data, including: When the data to be compensated for packet loss is irregular data, the packet loss frame data is windowed according to the window sequence of the packet loss frame to determine the overlapping data of the packet loss frame transmitted to the next frame; When the data to be compensated for packet loss is regular data, the packet loss frame data is windowed and folded according to the window sequence of the packet loss frame, and then windowed again to determine the overlapping data of the packet loss frame transmitted to the next frame.

9. The method according to claim 1, characterized in that: The method further comprises: When the packet loss frame includes multiple continuous data frames, determining the window sequence of the first packet loss frame according to the window sequence of the previous frame of the first packet loss frame, and determining the overlapping data transmitted from the first packet loss frame to the second packet loss frame according to the window sequence of the first packet loss frame and the predicted data of the first packet loss frame; Determine the window sequence of the second packet loss frame according to the window sequence of the first packet loss frame, and determine the overlapping data of the second packet loss frame transmitted to the third packet loss frame according to the window sequence of the second packet loss frame and the predicted second packet loss frame data; In this way, the overlapping data transmitted from each packet loss frame to the next frame is determined until the last packet loss frame is compensated for packet loss according to the packet loss compensation method of a single data frame.

10. A packet loss compensation device for adaptive window switching in advanced audio coding, characterized in that: The device comprises: A packet loss frame acquisition module, used to determine packet loss frames; A prediction module, used for selecting a frequency point for prediction according to a window sequence of a previous frame of the packet loss frame to predict the packet loss frame data when the packet loss frame is a single data frame; A window sequence determination module, used to determine the window sequence of the packet loss frame according to the window sequence of the previous frame and the window sequence of the next frame determined by decoding; The updating module is used to determine the overlapping data of the packet loss frame transmitted to the next frame according to the window sequence of the packet loss frame and the predicted packet loss frame data to complete the packet loss compensation, and the overlapping data is used to determine the output data of the next frame.

11. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the packet loss compensation method for adaptive window switching in advanced audio coding as described in any one of claims 1 to 9 by executing the computer instructions.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the packet loss compensation method for adaptive window switching in advanced audio coding according to any one of claims 1 to 9.

Images