Audio pulse noise processing method, related device and storage medium

Abstract

The invention discloses an audio pulse noise processing method, a related device and a storage medium. The method comprises the following steps: separating audio stream data from a video source of an input player and decoding the audio stream data; when a frame of to-be-played audio data of the audio stream data is read, according to the type of the to-be-played audio data, analyzing parameters of the autoregression model and a corresponding threshold value; wherein the type of the audio data to be played comprises a first frame, a middle frame and a tail frame; comparing the audio data to be played with the analyzed threshold value, and determining each segment of noise data; according to the type of the to-be-played audio data and the interval of each segment of noise data in the to-be-played audio data, predicting each segment of noise data through an autoregression model and corresponding logic, and repairing the noise data by using predicted data; and providing the repaired audio data to be played to a player for playing.

Description

Technical Field

[0001] The present application relates to the technical field of audio noise processing, and in particular to an audio impulse noise processing method and related devices and storage media. Background Art

[0002] Impulse noise is a common problem in audio. It is different from general noise in that it has the characteristics of short duration and sharp sound. It often appears in the AAC buffer mechanism of source recording, post-sound enhancement and long source playback. Therefore, if the audio impulse noise cannot be detected and repaired, the user's viewing experience will be greatly affected during the audio and video playback process.

[0003] Currently, the impulse noise of audio data is checked and restored in offline mode. Specifically, the entire audio data is uploaded to the editing software, such as traditional audition and DaVinci, and then the audio data is analyzed, played and displayed by the editing software, so that the impulse noise can be determined from the audio data and the impulse noise can be repaired through corresponding editing operations. Finally, the audio data repaired in the editing software is provided to the player for playback.

[0004] However, if this method is not strictly reviewed, there will be some pulse noise caused by current, microphone, etc. In addition, for longer videos, they need to be fragmented when playing, and the video source encoded in AAC will add zeros to the AAC data when aligning packets. This will cause a new obvious pulse noise to be generated when switching between fragments. And if the sound effect algorithm is also enabled, the noise impact of this fragment will be superimposed on all subsequent frames. Therefore, it can be seen that the existing method cannot guarantee that there is no pulse noise during audio playback, and it will still affect the user's viewing experience to a certain extent. Summary of the invention

[0005] Based on the above-mentioned deficiencies of the prior art, the present application provides an audio impulse noise processing method and related devices and storage media to solve the problem that the prior art cannot effectively ensure that there is no impulse noise during audio playback.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] The first aspect of the present application provides an audio impulse noise processing method, comprising:

[0008] Separate the audio stream from the video source input to the player and decode the separated audio stream data;

[0009] Whenever a frame of the decoded audio stream data to be played is read, the error of the autoregressive model corresponding to the audio data to be played is determined according to the type of the audio data to be played; wherein the type of the audio data to be played includes the first frame audio data, the middle frame audio data and the last frame audio data;

[0010] Determining a threshold value corresponding to the audio data to be played according to an error of the autoregressive model corresponding to the audio data to be played;

[0011] By comparing the audio data to be played with a threshold value corresponding to the audio data to be played, each segment of noise data in the audio data to be played is determined;

[0012] According to the type of the audio data to be played and the interval where each section of noise data in the audio data to be played is located in the audio data to be played, the autoregressive model is used to predict each sampling point in each section of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data; wherein the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval; p is the order of the autoregressive model;

[0013] The repaired audio data to be played is provided to the player for playing.

[0014] Optionally, in the above-mentioned audio impulse noise processing method, determining the error of the autoregressive model corresponding to the audio data to be played according to the type of the audio data to be played includes:

[0015] If the audio data to be played belongs to the first frame of audio data, a leading zero padding operation is performed on the audio data to be played to obtain the completed data of the audio data to be played;

[0016] If the audio data to be played belongs to the middle frame audio data or the last frame audio data of the slice, the signals of the last p sampling points of the repaired audio data of the previous frame are combined with the audio data to be played to obtain the complementary data of the audio data to be played;

[0017] Based on the supplementary data of the audio data to be played, the error of the autoregressive model corresponding to the audio data to be played is calculated using the Yule Walker equation;

[0018] If the audio data to be played belongs to the last frame of audio data of the film source, the error of the autoregressive model corresponding to the previous frame of audio data is determined as the error of the autoregressive model corresponding to the audio data to be played.

[0019] Optionally, in the above-mentioned audio impulse noise processing method, determining each segment of noise data in the audio data to be played by comparing the audio data to be played with a threshold value corresponding to the audio data to be played includes:

[0020] The audio data to be played is filtered by using a quasi-differential FIR filter to obtain filtered data of the audio data to be played; wherein the quasi-differential FIR filter is a filter after the original signal is introduced into the FIR filter as feedback through quasi-differential;

[0021] Determine the position of the sampling point in the filtered data of the audio data to be played that is not less than the threshold value corresponding to the audio data to be played, and obtain each filtered data noise point;

[0022] Determine, according to each of the noise points of the filtered data, each of the corresponding noise points in the audio data to be played;

[0023] The data between two adjacent noise points in the audio data to be played and whose interval is greater than the preset number of sampling points is determined as a section of noise data in the audio data to be played.

[0024] Optionally, in the above-mentioned audio impulse noise processing method, the audio data to be played belongs to the first frame of audio data, and the noise data is within the first p sampling points of the audio data to be played, and the use of the autoregressive model to predict each sampling point in each segment of the noise data, and interpolating and fusing the signals of each sampling point in the noise data through the predicted data to repair the noise data includes:

[0025] Reversing the current audio data to be played;

[0026] The inverted audio data to be played is calculated using the Yule Walker equation to obtain model parameters corresponding to the inverted audio data to be played;

[0027] Performing reverse linear prediction on each sampling point in the noise data by using the inverted model parameters corresponding to the audio data to be played through the autoregressive model to obtain a prediction signal for each sampling point in the noise data;

[0028] The original signal of each sampling point in the noise data in the inverted audio data to be played is replaced with its predicted signal, and the inverted audio data to be played is inverted.

[0029] Optionally, in the above-mentioned audio impulse noise processing method, the audio data to be played belongs to the first frame of audio data, and the noise data is in the middle sampling point interval of the audio data to be played, and the use of the autoregressive model to predict each sampling point in each segment of the noise data, and interpolating and fusing the signals of each sampling point in the noise data through the predicted data to repair the noise data includes:

[0030] Using the Yule Walker equation to calculate the current audio data to be played, to obtain the model parameters corresponding to the current audio data to be played;

[0031] Using the autoregressive model and the model parameters corresponding to the current audio data to be played, forward prediction and backward prediction are performed on each sampling point in the noise data to obtain a forward prediction signal and a backward prediction signal of each sampling point in the noise data;

[0032] The original signal of each sampling point in the noise data is replaced by a weighted result of the forward prediction signal and the backward prediction signal of each sampling point according to the corresponding first weight; wherein the first weight corresponding to the sampling point is determined according to the ratio of the point position of the sampling point to the length of the noise data.

[0033] Optionally, in the above-mentioned audio impulse noise processing method, the audio data to be played belongs to the first frame of audio data, and the noise data is within the interval of the last p sampling points of the audio data to be played, and the use of the autoregressive model to predict each sampling point in each segment of the noise data, and interpolating and fusing the signals of each sampling point in the noise data through the predicted data to repair the noise data includes:

[0034] Using the Yule Walker equation to calculate the current audio data to be played, to obtain the model parameters corresponding to the current audio data to be played;

[0035] Using the autoregressive model and the model parameters corresponding to the current audio data to be played, the last p sampling points of the audio data to be played are predicted, and the first p sampling points of the next frame of audio data are predicted, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffer prediction signal of the next frame;

[0036] Replacing the original signal of each sampling point belonging to the noise data among the last p sampling points of the audio data to be played with a weighted result of the original signal of the sampling point and the predicted signal of the sampling point according to the corresponding second weight;

[0037] Replacing the original signal of each of the sampling points that do not belong to the noise data among the last p sampling points of the audio data to be played with the product of the original signal of the sampling point and the corresponding second weight;

[0038] The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

[0039] Optionally, in the above-mentioned audio impulse noise processing method, the audio data to be played belongs to intermediate frame audio data, and the use of the autoregressive model to predict each sampling point in each segment of the noise data, and interpolating and fusing the signals of each sampling point in the noise data through the predicted data to repair the noise data includes:

[0040] For each segment of the noise data in the audio data to be played, if the noise data is within the first p sampling points of the audio data to be played, the prediction signal of each sampling point in the noise data is predicted by using the autoregressive model and the model parameters corresponding to the current audio data to be played; wherein the model parameters corresponding to the audio data to be played are calculated in advance by using the Yule Walker equation on the signals of the last p sampling points of the transmitted previous frame of audio data and the spliced ​​data of the current audio data to be played;

[0041] Replacing the original signal of each sampling point in the noise data with its prediction signal, and smoothing the signals of the first p sampling points of the current audio data to be played using the buffered prediction signal transmitted to the audio data to be played;

[0042] If the noise data is within the middle sampling point interval of the audio data to be played, forward prediction and backward prediction are performed on each sampling point in the noise data by using the autoregressive model using the model parameters corresponding to the current audio data to be played, so as to obtain a forward prediction signal and a backward prediction signal of each sampling point in the noise data;

[0043] Repairing each sampling point in the noise data by using a forward prediction signal and a backward prediction signal of each sampling point in the noise data;

[0044] If the noise data is within the interval of the last p sampling points of the audio data to be played, the autoregressive model is used to predict the last p sampling points of the audio data to be played and the first p sampling points of the next frame of audio data by using the model parameters corresponding to the current audio data to be played, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffered prediction signal of the next frame;

[0045] Repairing the last p sampling points of the audio data to be played using the prediction signals of the last p sampling points of the audio data to be played;

[0046] The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

[0047] Optionally, in the above audio impulse noise processing method, it also includes:

[0048] If the audio data to be played does not contain noise, and the previous frame of audio data of the audio data to be played contains noise, then calculating the average signal loudness of the first p sampling points of the audio data to be played, and calculating the average signal loudness of the last p sampling points of the previous frame of audio data of the audio data to be played;

[0049] If the difference between the average signal loudness of the first p sampling points of the audio data to be played and the average signal loudness of the last p sampling points of the previous frame of audio data is greater than a preset decibel, the audio data to be played is weakened;

[0050] If the difference between the average signal loudness of the last p sampling points of the previous frame of audio data of the to-be-played audio data and the average signal loudness of the first p sampling points of the to-be-played audio data is greater than a preset decibel, the to-be-played audio data is enhanced.

[0051] A second aspect of the present application provides an audio impulse noise processing device, comprising:

[0052] A pre-processing unit, used to separate the audio stream from the video source input to the player and decode the separated audio stream data;

[0053] The error calculation unit is used to determine the error of the autoregressive model corresponding to the audio data to be played every time a frame of the decoded audio stream data to be played is read according to the type of the audio data to be played; wherein the type of the audio data to be played includes the first frame audio data, the middle frame audio data and the last frame audio data;

[0054] A threshold calculation unit, used to determine a threshold corresponding to the audio data to be played according to an error of the autoregressive model corresponding to the audio data to be played;

[0055] a detection unit, configured to determine each segment of noise data in the audio data to be played by comparing the audio data to be played with a threshold value corresponding to the audio data to be played;

[0056] A repair unit, used to predict each sampling point in each segment of the noise data using the autoregressive model according to the type of the audio data to be played and the interval where each segment of noise data in the audio data to be played is located in the audio data to be played, and interpolate and fuse the signals of each sampling point in the noise data through the predicted data to repair the noise data; wherein the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval; p is the order of the autoregressive model;

[0057] A feedback unit is used to provide the repaired audio data to be played to the player for playing.

[0058] A third aspect of the present application provides an electronic device, including:

[0059] Memory and processor;

[0060] Wherein, the memory is used to store programs;

[0061] The processor is used to execute the program, and when the program is executed, it is specifically used to implement the audio impulse noise processing method as described in any one of the above items.

[0062] In a fourth aspect, the present application provides a computer storage medium for storing a computer program, wherein when the computer program is executed by a processor, the computer program is used to implement the audio impulse noise processing method as described in any one of the above.

[0063] The present application provides an audio impulse noise processing method, which separates the audio stream of the video source input to the player and decodes the separated audio stream data. Then, whenever a frame of the decoded audio stream data to be played is read, the error of the autoregressive model corresponding to the audio data to be played is determined according to the type of the audio data to be played, so as to detect it during audio playback, so that all impulse noises can be detected, whether they are original or generated by subsequent fragments. Among them, the types of audio data to be played include first frame audio data, middle frame audio data and tail frame audio data, so that the corresponding model error can be accurately determined according to the position of a frame of audio data. Then, according to the error of the autoregressive model corresponding to the audio data to be played, the threshold corresponding to the audio data to be played is determined, and the audio data to be played is filtered using a quasi-differential FIR filter to obtain the filtered data of the audio data to be played. Among them, the quasi-differential FIR filter is a filter that introduces the original signal as feedback into the FIR filter through quasi-differential to ensure the accuracy of impulse noise detection. Then, by comparing the filtered data of the audio data to be played with the threshold corresponding to the audio data to be played, the noise data of each segment in the audio data to be played is determined. Then, according to the type of the audio data to be played and the interval of each noise data in the audio data to be played, the autoregressive model is used to predict each sampling point in each noise data segment, and the signal of each sampling point in the noise data is interpolated and fused through the predicted data to repair the noise data. Among them, the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval. p is the order of the autoregressive model. Therefore, according to the position of the noise data, the corresponding logic is used to call the autoregressive model to predict the signal of the noise point, and the noise data is repaired based on the prediction result. Finally, the repaired audio data to be played is provided to the player for playback. Thus, a method of automatically detecting and repairing the impulse noise appearing at each position during the playback process is realized, so as to ensure that all noises can effectively detect and repair the impulse noise, that is, completely process all the impulse noises that appear, thereby effectively ensuring the user's viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0065] Figure 1 A flowchart of an audio impulse noise processing method provided in an embodiment of the present application;

[0066] Figure 2 A flowchart of a method for determining noise data provided in an embodiment of the present application;

[0067] Figure 3 A flowchart of a method for repairing noise data in the front section of a first frame of audio data provided by an embodiment of the present application;

[0068] Figure 4 A flowchart of a method for repairing noise data in a middle section of a first frame of audio data provided by an embodiment of the present application;

[0069] Figure 5 A flowchart of a method for repairing noise data in a tail section of a first frame of audio data provided by an embodiment of the present application;

[0070] Figure 6 A schematic diagram of the architecture of an audio impulse noise processing device provided in an embodiment of the present application;

[0071] Figure 7 A schematic diagram of the architecture of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] In this application, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0074] This embodiment provides a method for processing audio impulse noise. Figure 1 As shown, the following steps are included:

[0075] S101, separating the audio stream from the video source input into the player, and decoding the separated audio stream data.

[0076] It should be noted that, considering that when a video is played through a player, due to reasons such as fragmentation, the audio of the video may generate impulse noise, so in the embodiment of the present application, in order to completely eliminate all impulse noise, the audio of the video source input to the player for playback is directly processed in real time, and the offline processing method is no longer adopted. Therefore, for the video source input to the player, the audio stream is first separated to obtain its audio stream data, and then the separated audio stream data is decoded to obtain audio standard 16-bit PCM data.

[0077] S102 , whenever a frame of to-be-played audio data of the decoded audio stream data is read, according to the type of the to-be-played audio data, an error of an autoregressive model corresponding to the to-be-played audio data is determined.

[0078] The types of the audio data to be played include first frame audio data, middle frame audio data and last frame audio data.

[0079] The first frame audio data refers to the audio data of the first frame of the entire sound source that is not entirely noise. Therefore, if the first frame is entirely noise, the next frame will be further determined as the first frame audio data. Of course, if multiple frames are entirely noise, for example, if all 5 frames are noise, it means that the noise range is large and does not belong to impulse noise. At this time, the player can be informed of the input stream error or other noise reduction methods can be used.

[0080] The tail frame audio data is the last frame of each slice, not just the last frame of the last slice, that is, not just the last frame of the entire sound source, and the remaining frame data between the first frame audio data and the tail frame audio data are intermediate frame audio data.

[0081] Optionally, in order to facilitate the type of audio data to be played that is currently read, the first frame identifier and the last frame identifier can be set at the initialization, and initialized to 1 and 0 respectively. After the first frame of audio data is processed, the first frame identifier is set to 0. If the first frame audio data is all noise, the next frame still needs to be processed with the first frame logic, so the first frame identifier will be restored to 1. Therefore, when the next frame is processed, if the first frame is represented as 0, it is determined that it belongs to the middle frame audio data, and enters the middle frame logic for processing. If the first frame identifier is 1, it is determined that it belongs to the first frame audio data, and the first frame logic is used for processing.

[0082] Specifically, when the audio stream data is played, it will be read frame by frame for playback, so each frame of audio data is read, and it is used as the audio data to be played for impulse noise processing, and then played after processing. The current audio is mainly encoded in ACC, so in the embodiment of the present application, the audio stream data is mainly read with ACC encoding 1024 as the standard sampling point for explanation, that is, each frame of audio data read includes a signal of 1024 sampling points.

[0083] The autoregressive model (AR model) belongs to the time series model. The model reflects the relationship between an observation and several lagged observations. The core idea is that the current value of the time series can be expressed as a linear combination of past values. Therefore, it is necessary to define the parameters of the autoregressive model first. For a p-order AR model, it is expressed as:

[0084]

[0085] Among them, X t is the current sampling point. It is the model parameter of AR, that is, the weight corresponding to each sampling.

[0086] Therefore, the theoretical value of the signal at a regular sampling point can be analyzed by the autoregressive model. Although the result of the analysis may have a certain error with the actual value, it will not be particularly large. If the error is large, it means that the signal is noise, that is, it does not follow the law of the change of audio data. Therefore, in the embodiment of the present application, the size of the normal error between the real value of the audio data to be played and the value of the audio data to be played analyzed by the autoregressive model is first determined, that is, the error of the autoregressive model corresponding to the audio data to be played, and then the noise data can be analyzed through the error.

[0087] Since the order of the AR model is usually 1 / 4 of the length of a frame of audio, the value of p for a frame of ACC audio data is 256. This is the frame shift of the standard windowing function. The length of the AR model is set to the sum of the length of a frame of audio plus the order of the AR model, so it can be specifically 1024+256=1280, to ensure that a complete frame of audio data is generated with the least number of previous frame sampling points when a frame is full of noise. Therefore, the length of the AR model is greater than the length of the read frame of audio. This means that when a frame of audio data is processed using the AR model, data needs to be added to a frame of audio data. There is no data to refer to in front of the first frame of audio, so zero padding is required to make its length meet 1280, while the data of the middle frame can directly refer to the previous frame, and the length of the audio data of the tail frame may be shorter, in which case the previous frame can be directly referred to. Therefore, the audio data of the first frame, the middle frame and the tail frame need to be processed using corresponding strategies respectively. Therefore, it is necessary to determine the error of the autoregressive model corresponding to the audio data to be played according to the type of the audio data to be played.

[0088] Specifically, by predicting the signal of the sampling point of the audio data to be played through the autoregressive model, and then calculating the error between the predicted signal of the sampling point and the original signal of the sampling point, the error of the autoregressive model corresponding to the audio data to be played can be determined.

[0089] Optionally, in another embodiment of the present application, a specific implementation of step S102 includes:

[0090] If the audio data to be played belongs to the first frame of audio data, the audio data to be played is pre-padded with zeros to obtain the completed data of the audio data to be played. If the audio data to be played belongs to the middle frame of audio data or the last frame of the slice, the signal of the last p sampling points of the repaired audio data of the previous frame is combined with the audio data to be played to obtain the completed data of the audio data to be played. Then, based on the completed data of the audio data to be played, the error of the autoregressive model corresponding to the audio data to be played is calculated using the Yule Walker equation.

[0091] It should be noted that, for the first frame audio data, since there is no reference data in front of it, it can only be padded with zeros, that is, multiple zeros are added before the first frame data to make its length reach the length of the autoregressive model. For the middle frame audio data, the signal of the last p sampling points of the repaired audio data of the previous frame can be directly combined with the audio data to be played. For the tail frame audio data, if it is the fragmented tail frame audio data, its length is normal, so it is the same as the middle frame.

[0092] After the data is padded to a length of 1280, that is, after the length of the AR model is padded, in the embodiment of the present application, the parameters of the autoregressive model are calculated using the Yule Walker equation, and then these parameters are substituted into the autoregressive model to predict the audio data to be played using the autoregressive model. Finally, the error is determined based on the difference between the predicted value and the true value.

[0093] Among them, for a p-order autoregressive model, the Yule Walker equation can be expressed as:

[0094]

[0095] It can be further written as:

[0096]

[0097]

[0098]

[0099] Where R is the autocorrelation matrix, which can be calculated using the Toeplitz matrix; is the parameter of the autoregressive model; r is the autocorrelation vector; n is the length of the autoregressive model; p is the order of the autoregressive model; x is the audio data.

[0100] If the audio data to be played belongs to the last frame of audio data of the film source, the error of the autoregressive model corresponding to the previous frame of audio data is determined as the error of the autoregressive model corresponding to the audio data to be played.

[0101] Since the length of the audio data of the last frame of the source is usually smaller than 1024, even if the 256 sampling points of the previous frame are added, the length of the model cannot be reached, so we can only refer to the parameters of the previous frame of audio data.

[0102] S103: Determine a threshold value corresponding to the audio data to be played according to an error of an autoregressive model corresponding to the audio data to be played.

[0103] It should be noted that the error of the autoregressive model corresponding to the audio data to be played can reflect whether the sampling point is a noise point. Therefore, the error of the autoregressive model corresponding to the audio data to be played can be used to determine the threshold corresponding to the audio data to be played, which is used to detect impulse noise.

[0104] Optionally, since the error is usually expressed as a variance, and the standard deviation can provide a more intuitive measure, the square root of the error is used to calculate the threshold in the embodiment of the present application. In order to filter out impulse noise and set different noise screening levels according to requirements, a coefficient k is introduced in the embodiment of the application, which is usually 4 and can also be set according to requirements. Therefore, the threshold is specifically the product of the square root of the error and the coefficient k.

[0105] S104: Determine each section of noise data in the audio data to be played by comparing the filtered data of the audio data to be played with the threshold value corresponding to the audio data to be played.

[0106] It should be noted that the threshold corresponding to the audio data to be played reflects the normal error of the autoregressive model corresponding to the audio data to be played. Therefore, by comparing the audio data to be played with the threshold corresponding to the audio data to be played, the noise data in the audio data to be played that is not less than the threshold can be determined, that is, the data with excessive error can be determined, which is the noise data.

[0107] Optionally, in another embodiment of the present application, a specific implementation of step S104 is as follows: Figure 2 As shown, the following steps are included:

[0108] S201 , using a quasi-differential FIR filter to filter the audio data to be played, to obtain filtered data of the audio data to be played.

[0109] The quasi-differential FIR filter is a filter obtained by introducing the original signal as feedback into the FIR filter through quasi-differential.

[0110] It should be noted that the output of the FIR filter is the convolution of the original continuous signal difference and the coefficient of the AR model, that is, the first-order difference convolution of the original signal and the filtered signal. The first-order difference has a relatively strong numerical anti-interference ability, which can effectively represent the difference between the original signal and its AR model prediction value, and usually produces a larger value in the difference signal. Therefore, this difference signal is usually used to detect abnormal values ​​or damaged samples, so that more accurate detection can be performed. Therefore, in the embodiment of the present application, the original signal of the audio data to be played is first filtered by the FIR filter.

[0111] Specifically, the traditional FIR filter is expressed as:

[0112]

[0113] Since the FIR filter only contains the feedforward term of the input signal, which will cause the output of the filter to become zero in a finite time, in the embodiment of the present application, the original signal is introduced for feedback to obtain an improved quasi-differential FIR filter. It is specifically expressed as:

[0114]

[0115] Among them, d t is the filtered signal; x t is the original signal; p is the order of the autoregressive model; are the parameters of the autoregressive model.

[0116] S202: Determine the positions of sampling points in the filtered data of the audio data to be played that are not less than the threshold corresponding to the audio data to be played, and obtain noise points of each filtered data.

[0117] S203: Determine each noise point in the corresponding audio data to be played according to each noise point of the filtered data.

[0118] Specifically, according to the positions of the noise points of each filtering data, the samples in the to-be-played audio data at the corresponding positions are determined as the noise points of the to-be-played audio data.

[0119] S204: Determine data between two adjacent noise points in the audio data to be played whose interval is greater than a preset number of sampling points as a section of noise data in the audio data to be played.

[0120] Specifically, for adjacent noise points, when the number of the two is greater than a preset number of sampling points, such as greater than 2, it is determined that the data between the two needs to be processed by difference, so it is determined as noise data.

[0121] Optionally, the signal of each sampling point in the noise data may be set to Nan, so that the signal can be used later and the noise data can be accurately located.

[0122] S105. According to the type of the audio data to be played and the interval of each noise data segment in the audio data to be played, use the autoregressive model to predict each sampling point in each noise data segment, and interpolate and fuse the signals of each sampling point in the noise data through the predicted data to repair the noise data.

[0123] The interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval.

[0124] It should be noted that, similar to the cases of the first frame, the middle frame and the last frame, since they need to be processed by an autoregressive model, the reference data that can be used in the first interval of the audio data, that is, the first 256 sampling points, the last interval, that is, the last 256 sampling points, and the middle sampling points, that is, the [256,768] interval, are different. Therefore, not only the noise data needs to be repaired accordingly according to the different types of the audio data to be played, but also the position of the noise data in the audio data to be played.

[0125] Specifically, during the repair process, the autoregressive model can be used to predict each sampling point in each segment of noise data to obtain the predicted signal of each sampling point. Then, the predicted signal of the sampling point is subjected to difference processing to obtain the interpolation result, so that it is no longer a large pulse data. Finally, the interpolation result is used to replace the original signal, thereby realizing the repair of the noise data, that is, eliminating the noise data.

[0126] It should be noted that during the noise detection and repair process, the audio data on each channel is processed separately. Therefore, for the first frame of audio data, if the audio data on a certain channel is all noise, since there is no reference to the previous frame of audio data for repair, the repaired audio data on other channels can be assigned to this channel.

[0127] For the first frame of audio data, when there is noise in multiple intervals, it is necessary to process the noise data in the middle sampling point interval first, and then process the noise data in the relatively special first p sampling point intervals and last p sampling point intervals.

[0128] Optionally, in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data, and the noise data is within the first p sampling points of the audio data to be played, a specific implementation of step S105 is as follows: Figure 3 As shown, including:

[0129] S301, reverse the current audio data to be played.

[0130] It should be noted that since there is no sampling point before the first frame of audio data, and the data of a certain sampling point is predicted by the autoregressive model, at least the data of the p sampling points before it is required, that is, the data of 256 sampling points are required. Therefore, the noise data in the first 256 sampling points of the first frame of audio data cannot be directly predicted by the autoregressive model. Therefore, in the embodiment of the present application, the current audio data to be played is reversed so that the noise data is no longer located in the first 256 points, and then at least 256 sampling points are used to predict the sampling points therein.

[0131] S302: Calculate the inverted audio data to be played using the Ulvak equation to obtain model parameters corresponding to the inverted audio data to be played.

[0132] Since the audio data to be played is reversed at this time, the change pattern of the data is also different, so it is necessary to calculate the reversed audio data to be played again through the Yule Walker equation to obtain the value of the model parameter corresponding to the reversed audio data to be played.

[0133] S303 , using the autoregressive model and the model parameters corresponding to the inverted audio data to be played, perform reverse linear prediction on each sampling point in the noise data to obtain a prediction signal for each sampling point in the noise data.

[0134] Specifically, the model parameters corresponding to the inverted audio data to be played are substituted into the autoregressive model. Then, for each sampling point of the noise data, the autoregressive model is used to predict the predicted signal of the sampling point by using the signal of the sampling point before the sampling point.

[0135] S304: Replace the original signal of each sampling point in the noise data in the inverted audio data to be played with its predicted signal, and invert the inverted audio data to be played.

[0136] Since the original signal of the sampling point is noise, and the predicted signal is relatively accurate and does not belong to noise, the original signal is replaced with the predicted signal to repair the noise data. Since the audio data to be played was reversed before, it needs to be reversed again after the repair to restore it to its original order.

[0137] It should also be noted that if there are multiple noise data segments within the first 256 sampling points of the first frame of audio data, each segment is processed according to the above logic, and needs to be processed in the order of the later the processing is, the earlier the processing is. Because the audio data needs to be reversed during processing, and the previous data needs to be used during prediction, it is necessary to repair the noise data at the front after the reversal first to ensure the accuracy of the repair of the noise data at the back. The noise data at the front after the reversal is the later before the reversal, so the noise data at the back needs to be processed first.

[0138] Optionally, in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data, and the noise data is in the middle sampling point interval of the audio data to be played, a specific implementation of step S105 is as follows: Figure 4 As shown, the following steps are included:

[0139] S401, using the Yule Walker equation to calculate the current audio data to be played, to obtain model parameters corresponding to the current audio data to be played.

[0140] It should be noted that when calculating the autoregressive model corresponding to the audio data to be played, the model parameters corresponding to the current audio data to be played are usually calculated, so the previously calculated model parameters can be directly obtained at this time.

[0141] S402, using the autoregressive model and the model parameters corresponding to the current audio data to be played, forward predicting and backward predicting each sampling point in the noise data, to obtain a forward prediction signal and a backward prediction signal of each sampling point in the noise data.

[0142] It should be noted that for the sampling points in the middle sampling point interval, there are sufficient reference data both forward and backward to use the autoregressive model for prediction. Therefore, in order to achieve a better repair effect, for the sampling points in the middle sampling point interval,

[0143] Specifically, the model parameters corresponding to the audio data to be played are substituted into the autoregressive model, and then the autoregressive model is used to predict each sampling point in the noise data of the audio data to be played, that is, to predict from the non-missing value before the noise until the noise result is reached. Then, each sampling point of the noise data is predicted backward, that is, to predict from the non-missing value after the end of the noise until the beginning of the noise is reached.

[0144] S403: Replace the original signal of each sampling point in the noise data with a weighted result of the forward prediction signal and the backward prediction signal of each sampling point according to the corresponding first weight.

[0145] The first weight corresponding to the sampling point is determined according to the ratio of the position of the sampling point to the length of the noise data.

[0146] It should be noted that in order to reduce the discontinuity of the padded data, the noise data needs to be smoothed. The conventional smoothing method is to uniformly take half of each data for smoothing, that is, to take the sum of half of the forward prediction and backward prediction results as the predicted value. However, in practice, the AR model forward prediction data is significantly more consistent for data close to the starting point of the noise than the predicted data close to the tail of the noise, and vice versa. Therefore, in an embodiment of the present application, a first weight corresponding to each sampling point is introduced to control the proportion of previous predictions and backward predictions.

[0147] So the specific smoothing formula is:

[0148]

[0149]

[0150] in, is the data to be finally inserted, i.e. the data to replace the original signal; is a first weight between 0 and 1; (S n+1 -S n ) is the length of the noise data; i is the current point, i.e. the position of a sampling point; X t To predict the signal forward; is the backward prediction signal.

[0151] Optionally, in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data, and the noise data is within the interval of the last p sampling points of the audio data to be played, a specific implementation of step S105 is as follows: Figure 5 As shown, including:

[0152] S501, using the Ulver's equation to calculate the current audio data to be played, to obtain model parameters corresponding to the current audio data to be played.

[0153] S502. Predict the last p sampling points of the audio data to be played and the first p sampling points of the next frame of audio data by using the model parameters corresponding to the current audio data to be played through the autoregressive model, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffer prediction signal of the next frame.

[0154] It should be noted that the last p sampling points of the audio data to be played will be connected to the next frame of audio data, and in order to avoid the noise rule in the first p sampling points of the next frame of audio data, which will cause a large difference in loudness and value between the two frames, it is necessary to smooth it. Therefore, when processing the last 256 sampling points of the audio data to be played, not only the predicted values ​​of the last 256 sampling points of the audio data to be played are predicted through the autoregressive model, but also the first 256 sampling points of the next frame of audio data are predicted, so as to facilitate the subsequent smoothing of the first 256 sampling points of the next frame of audio data after repair.

[0155] S503, replacing the original signal of each sampling point belonging to noise data among the last p sampling points of the audio data to be played with a weighted result of the original signal of the sampling point and the predicted signal of the sampling point according to the corresponding second weight, and replacing the original signal of each sampling point that does not belong to noise data among the last p sampling points of the audio data to be played with the product of the original signal of the sampling point and the corresponding second weight.

[0156] It should be noted that after the prediction signal is obtained, the conventional data filling method is to replace the noise with the prediction signal, while the non-noise remains unchanged. However, in the actual player frame processing, it is found that when the difference between continuous audio values ​​is too large, new noise will be introduced at the junction of real data and predicted data. Therefore, in this conventional smoothing method, half of each data is uniformly smoothed. The second weight is introduced in the application embodiment to improve the default AR model filling method to smooth the data.

[0157] The improved filling method can be expressed as:

[0158]

[0159]

[0160]

[0161]

[0162] in, is the smoothed signal; x t is the original signal; X t is the interpolation data, i.e. the prediction signal; is the second weight; (S n+1 -S n ) is the length of the noise data; i is the sampling point. Since the noise data is set to Nan, x t =Nan means the sampling point belongs to noise data; x t ≠Nan means that the sampling point does not belong to noise data.

[0163] S504: Combine the signals of the last p sampling points of the audio data to be played with the buffered prediction signal of the next frame, and transmit them to the next frame of audio data.

[0164] Since there is a fault in the first 256 sampling points of the next frame of audio data, the signals of the last p sampling points of the current audio data to be played need to be repaired, and the buffered prediction signal needs to be smoothed. Therefore, it is necessary to pass the signals of the last p sampling points of the audio data to be played, that is, the first 256 sampling points of the next frame of audio data predicted by the repaired 256 sampling points, to the next frame.

[0165] Therefore, it is also necessary to explain that if there is no noise in the first frame audio data, it is also necessary to predict the buffer prediction signal of the next frame and combine it with the last p sampling points of the first frame audio data to pass it to the next frame.

[0166] Optionally, in another embodiment of the present application, when the audio data to be played belongs to intermediate frame audio data, a specific implementation of step S105 includes:

[0167] For each noise data segment in the audio data to be played, if the noise data is within the first p sampling points of the audio data to be played, the prediction signal of each sampling point in the noise data is predicted by using the autoregressive model using the model parameters corresponding to the current audio data to be played. Then, the original signal of each sampling point in the noise data is replaced with its prediction signal, and the signal of the first p sampling points of the current audio data to be played is smoothed using the buffered prediction signal transmitted to the audio data to be played.

[0168] The model parameters corresponding to the audio data to be played are calculated in advance using the Yule Walker equation on the concatenated data of the signals of the last p sampling points of the last frame of audio data and the current audio data to be played.

[0169] Different from the first frame audio data, the intermediate frame has audio data in front of it, so it will buffer the 256 sampling points of the previous frame. Specifically, the last 256 sampling points of the previous frame are combined with the audio data of the intermediate frame, and then the model parameters and errors corresponding to the audio data to be played can be calculated through the Yule Walker equation.

[0170] Since there is a previous frame as a reference at this time, when there is operation data within the first 256 sampling points of the audio data to be played, the autoregressive model can be used directly to predict the prediction signal of each sampling point in the noise data. In addition, in order to ensure that the data between the previous frame and the current frame is as continuous as possible and no new noise points are introduced, the prediction signal and the 256 sampling points predicted by the previous frame need to be smoothed, and a weighted calculation is performed based on the buffered prediction signal transmitted from the previous frame. Among them, the specific weighting method can be performed in a weighted manner with the forward prediction signal and the backward prediction signal of the middle sampling point of the first frame, that is, according to the smoothing formula provided in step S403, except that the two signals calculated are replaced from the forward prediction signal and the backward prediction signal by the current prediction signal of the audio data to be played and the buffered prediction signal transmitted from the previous frame.

[0171] If the noise data is in the middle sampling point interval of the audio data to be played, the autoregressive model is used to forward predict and backward predict each sampling point in the noise data using the model parameters corresponding to the current audio data to be played, so as to obtain the forward prediction signal and backward prediction signal of each sampling point in the noise data, and use the forward prediction signal and backward prediction signal of each sampling point in the noise data to repair each sampling point in the noise data.

[0172] It should be noted that the repair method for the noise data in the middle sampling point interval in the middle frame audio data is the same as the repair method for the noise data in the middle sampling point interval in the first frame audio data, so it will not be repeated here.

[0173] If the noise data is within the interval of the last p sampling points of the audio data to be played, the autoregressive model is used to predict the last p sampling points of the audio data to be played and the first p sampling points of the next frame of audio data using the model parameters corresponding to the current audio data to be played, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffered prediction signal of the next frame, and the prediction signal of the last p sampling points of the audio data to be played is used to repair the last p sampling points of the audio data to be played, and finally the signal of the last p sampling points of the audio data to be played is combined with the buffered prediction signal of the next frame and passed to the next frame of audio data.

[0174] It should be noted that the repair method for the noise data in the interval of the last p sampling points in the intermediate frame audio data is the same as the repair method for the noise data in the interval of the last p sampling points in the first frame audio data, so it will not be repeated here.

[0175] As for the last frame audio data, there are mainly two cases: one is the last frame of the segment switched by the long source, and the other is the last one at the end of the source. Specifically, it can be determined according to the signal given by the player which type of audio data to be played currently belongs to.

[0176] If it is the last frame of the segment switched by the long film source, its length is normal, and the next allocated video frame will follow, so its processing method is consistent with that of the intermediate frame. Not only does it need to be repaired, but it also needs to pass 512 sampling points of data to the first frame of audio data of the next segment. If it is the last frame at the end of the film source, since there is no audio data afterwards, it is necessary to generate a buffer prediction signal and pass it. In addition, its length can be various possibilities within 1024, and when its length is less than 256 sampling points, the amount of data is too small. Therefore, when analyzing noise data and repairing, the parameters of the AR model used can directly use the parameters of the AR model of the previous frame. When it is greater than 256, it is necessary to refer to the AR parameters of the entire segment to generate data. In addition to the above two differences, the processing logic for the audio data of the tail frame is consistent with the processing logic for the audio data of the intermediate frame, so the number of times will not be repeated.

[0177] Optionally, in another embodiment of the present application, it may further include:

[0178] If the audio data to be played does not contain noise and the previous frame of audio data to be played contains noise, then the average signal loudness of the first p sampling points of the audio data to be played is calculated, and the average signal loudness of the last p sampling points of the previous frame of audio data to be played is calculated.

[0179] If the difference between the average signal loudness of the first p sampling points of the audio data to be played and the average signal loudness of the last p sampling points of the previous frame of audio data is greater than the preset decibel, the audio data to be played is weakened.

[0180] If the difference between the average signal loudness of the last p sampling points of the previous frame of audio data to be played and the average signal loudness of the first p sampling points of the audio data to be played is greater than the preset decibel, the audio data to be played is enhanced.

[0181] It should be noted that, in the embodiment of the present application, a loudness detection method is introduced to improve the conventional fade-in and fade-out method, so as to improve the problem of excessive loudness change between two frames caused by excessive fade-in and fade-out, thereby avoiding noise data when two frames of audio data are played.

[0182] Therefore, if there is no noise in the current audio data to be played, and there is noise in the previous frame of audio data to be played, the average loudness of the audio signal of the last 256 sampling points of the previous frame and the average loudness of the first 256 sampling points of the current frame will be calculated respectively. Then the difference between the two means is calculated. If the difference between the two exceeds the preset decibel, usually 6db, the relative data is enhanced. That is, when the loudness of the current frame is large, its loudness is weakened, which is equivalent to enhancing the previous frame. When the loudness of the current frame is small, its loudness is enhanced. The specific calculation logic can be expressed as:

[0183]

[0184]

[0185]

[0186]

[0187] Among them, x t-1 The original audio data of the previous frame; x t The original audio data of the current frame; and is the audio data after processing of the previous frame and the current frame; t is the number of cycles; i represents the position of the sampling point.

[0188] S106: Provide the repaired audio data to be played to the player for playing.

[0189] Specifically, the maximum and minimum out-of-bounds judgment is performed on the repaired audio data to be played, that is, the range of 16-bit PCM audio data is between [-32767, 32767], and the final clean signal is obtained and output to the player for playback, thereby completing the frame-level noise detection and repair logic during playback.

[0190] The present application provides an audio impulse noise processing method, which separates the audio stream of the video source input to the player and decodes the separated audio stream data. Then, every time a frame of audio data to be played of the decoded audio stream data is read, the error of the autoregressive model corresponding to the audio data to be played is determined according to the type of the audio data to be played. Among them, the type of the audio data to be played includes the first frame audio data, the middle frame audio data and the tail frame audio data, so that the corresponding model error can be accurately determined according to the position of a frame of audio data. Then, according to the error of the autoregressive model corresponding to the audio data to be played, the threshold corresponding to the audio data to be played is determined, and the audio data to be played is filtered by using a quasi-differential FIR filter to obtain the filtered data of the audio data to be played. Among them, the quasi-differential FIR filter is a filter that introduces the original signal as feedback into the FIR filter through quasi-differential to ensure the accuracy of impulse noise detection. Then, by comparing the filtered data of the audio data to be played with the threshold corresponding to the audio data to be played, the noise data of each segment in the audio data to be played is determined. Then, according to the type of the audio data to be played and the interval of each noise data in the audio data to be played, the autoregressive model is used to predict each sampling point in each noise data segment, and the signal of each sampling point in the noise data is interpolated and fused through the predicted data to repair the noise data. Among them, the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval. p is the order of the autoregressive model. Therefore, according to the position of the noise data, the corresponding logic is used to call the autoregressive model to predict the signal of the noise point, and the noise data is repaired based on the prediction result. Finally, the repaired audio data to be played is provided to the player for playback. Thus, a method of automatically detecting and repairing the impulse noise appearing at each position during the playback process is realized, so as to ensure that all noises can effectively detect and repair the impulse noise, that is, completely process all the impulse noises that appear, thereby effectively ensuring the user's viewing experience.

[0191] Another embodiment of the present application provides an audio impulse noise processing device, such as Figure 6 As shown, including:

[0192] The pre-processing unit 601 is used to separate the audio stream from the video source input to the player and decode the separated audio stream data.

[0193] The error calculation unit 602 is used to determine the error of the autoregressive model corresponding to the audio data to be played according to the type of the audio data to be played, each time a frame of the decoded audio stream data to be played is read. The type of the audio data to be played includes the first frame audio data, the middle frame audio data and the last frame audio data.

[0194] The threshold calculation unit 603 is used to determine the threshold corresponding to the audio data to be played according to the error of the autoregressive model corresponding to the audio data to be played.

[0195] The detection unit 604 is used to determine each segment of noise data in the audio data to be played by comparing the filtered data of the audio data to be played with the threshold value corresponding to the audio data to be played.

[0196] The repair unit 605 is used to predict each sampling point in each noise data segment using an autoregressive model according to the type of the audio data to be played and the interval where each noise data segment in the audio data to be played is located in the audio data to be played, and interpolate and fuse the signals of each sampling point in the noise data through the predicted data to repair the noise data. The interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval. p is the order of the autoregressive model.

[0197] The feedback unit 606 is used to provide the repaired audio data to be played to the player for playing.

[0198] Optionally, in the audio impulse noise processing device provided in another embodiment of the present application, the error calculation unit includes:

[0199] The first complement unit is used for performing a leading zero-filling operation on the audio data to be played when the audio data to be played belongs to the first frame of audio data, so as to obtain complement data of the audio data to be played.

[0200] The second completion unit is used to combine the signals of the last p sampling points of the repaired audio data of the previous frame with the audio data to be played when the audio data to be played belongs to the middle frame audio data or the last frame audio data of the fragment, so as to obtain the completion data of the audio data to be played.

[0201] The first error calculation unit is used to calculate the error of the autoregressive model corresponding to the audio data to be played by using the Yule Walker equation based on the supplementary data of the audio data to be played.

[0202] The error determination unit is used to determine the error of the autoregressive model corresponding to the previous frame of audio data as the error of the autoregressive model corresponding to the audio data to be played when the audio data to be played belongs to the last frame of audio data of the film source.

[0203] Optionally, in the audio impulse noise processing device provided in another embodiment of the present application, the detection unit includes:

[0204] A filtering unit, configured to filter the audio data to be played using a quasi-differential FIR filter to obtain filtered data of the audio data to be played; wherein the quasi-differential FIR filter is a filter that introduces the original signal as feedback into the FIR filter through quasi-differential;

[0205] The noise point detection unit is used to determine the position of the sampling point in the filtered data of the audio data to be played that is not less than the threshold value corresponding to the audio data to be played, and obtain each filtered data noise point.

[0206] The matching unit is used to determine each noise point in the corresponding audio data to be played according to each noise point of the filtered data.

[0207] The noise data determination unit is used to determine the data between two adjacent noise points in the audio data to be played and whose interval is greater than the preset number of sampling points as a section of noise data in the audio data to be played.

[0208] Optionally, in an audio impulse noise processing device provided in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data and the noise data is within the first p sampling points of the audio data to be played, the repair unit predicts each sampling point in each segment of noise data using an autoregressive model, and interpolates and fuses the signals of each sampling point in the noise data using the predicted data, and when repairing the noise data, is used to:

[0209] Reverse the current audio data to be played.

[0210] The inverted audio data to be played is calculated using the Yule Walker equation to obtain model parameters corresponding to the inverted audio data to be played.

[0211] The autoregressive model uses the model parameters corresponding to the inverted audio data to be played to perform reverse linear prediction on each sampling point in the noise data to obtain a prediction signal for each sampling point in the noise data.

[0212] The original signal of each sampling point in the noise data in the inverted audio data to be played is replaced with its predicted signal, and the inverted audio data to be played is inverted.

[0213] Optionally, in the audio impulse noise processing device provided in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data and the noise data is in the middle sampling point interval of the audio data to be played, the repair unit predicts each sampling point in each segment of noise data using an autoregressive model, and interpolates and fuses the signals of each sampling point in the noise data through the predicted data, and when repairing the noise data, is used to:

[0214] The current audio data to be played is calculated using the Yule Walker equation to obtain model parameters corresponding to the current audio data to be played.

[0215] The autoregressive model uses the model parameters corresponding to the current audio data to be played to perform forward prediction and backward prediction on each sampling point in the noise data, thereby obtaining a forward prediction signal and a backward prediction signal for each sampling point in the noise data.

[0216] The original signal of each sampling point in the noise data is replaced by the weighted result of the forward prediction signal and the backward prediction signal of each sampling point according to the corresponding first weight, wherein the first weight corresponding to the sampling point is determined according to the ratio of the point position of the sampling point to the length of the noise data.

[0217] Optionally, in an audio impulse noise processing device provided in another embodiment of the present application, when the audio data to be played belongs to the first frame of audio data and the noise data is within the interval of the last p sampling points of the audio data to be played, the repair unit predicts each sampling point in each segment of noise data using an autoregressive model, and interpolates and fuses the signals of each sampling point in the noise data using the predicted data, and when repairing the noise data, is used to:

[0218] The current audio data to be played is calculated using the Yule Walker equation to obtain model parameters corresponding to the current audio data to be played.

[0219] The autoregressive model uses the model parameters corresponding to the current audio data to be played to predict the last p sampling points of the audio data to be played, and the first p sampling points of the next frame of audio data are predicted to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffered prediction signal of the next frame.

[0220] The original signal of each sampling point belonging to the noise data among the last p sampling points of the audio data to be played is replaced by a weighted result of the original signal of the sampling point and the predicted signal of the sampling point according to the corresponding second weight.

[0221] The original signal of each sampling point that does not belong to noise data among the last p sampling points of the audio data to be played is replaced by the product of the original signal of the sampling point and the corresponding second weight.

[0222] The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

[0223] Optionally, in an audio impulse noise processing device provided in another embodiment of the present application, when the audio data to be played belongs to intermediate frame audio data, the repair unit predicts each sampling point in each segment of noise data using an autoregressive model, and interpolates and fuses the signals of each sampling point in the noise data using the predicted data, and when repairing the noise data, is used to:

[0224] For each noise data segment in the audio data to be played, if the noise data is within the first p sampling points of the audio data to be played, the prediction signal of each sampling point in the noise data is predicted by using the autoregressive model and the model parameters corresponding to the current audio data to be played. The model parameters corresponding to the audio data to be played are calculated in advance by using the Yule Walker equation to calculate the signals of the last p sampling points of the transmitted previous frame of audio data and the concatenated data of the current audio data to be played.

[0225] The original signal of each sampling point in the noise data is replaced with its prediction signal, and the signals of the first p sampling points of the current audio data to be played are smoothed using the buffered prediction signal transmitted to the audio data to be played.

[0226] If the noise data is in the middle sampling point interval of the audio data to be played, the autoregressive model is used to use the model parameters corresponding to the current audio data to be played to perform forward and backward predictions on each sampling point in the noise data to obtain forward prediction signals and backward prediction signals for each sampling point in the noise data.

[0227] Each sampling point in the noise data is repaired by using the forward prediction signal and the backward prediction signal of each sampling point in the noise data.

[0228] If the noise data is within the interval of the last p sampling points of the audio data to be played, the autoregressive model is used to predict the last p sampling points of the audio data to be played using the model parameters corresponding to the current audio data to be played, and the first p sampling points of the next frame of audio data are predicted to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffered prediction signal of the next frame.

[0229] The predicted signals of the last p sampling points of the audio data to be played are used to repair the last p sampling points of the audio data to be played.

[0230] The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

[0231] Optionally, in the audio impulse noise processing device provided in another embodiment of the present application, the device further includes:

[0232] The mean value calculation unit is used to calculate the mean value of the signal loudness of the first p sampling points of the audio data to be played when there is no noise in the audio data to be played and the mean value of the signal loudness of the last p sampling points of the audio data of the previous frame of audio data to be played, and calculate the mean value of the signal loudness of the last p sampling points of the audio data of the previous frame of audio data to be played.

[0233] The weakening unit is used to weaken the audio data to be played when the difference between the average signal loudness of the first p sampling points of the audio data to be played and the average signal loudness of the last p sampling points of the previous frame of audio data is greater than a preset decibel.

[0234] The adding unit is used to enhance the audio data to be played when the difference between the average signal loudness of the last p sampling points of the previous frame of audio data to be played and the average signal loudness of the first p sampling points of the audio data to be played is greater than a preset decibel.

[0235] It should be noted that the specific working process of each unit in the above embodiments of the present application can refer to the implementation process of the corresponding steps in the above method embodiments, and will not be repeated here.

[0236] A third aspect of the present application provides an electronic device, such as Figure 7 As shown, including:

[0237] Memory 701 and processor 702 .

[0238] The memory 701 is used to store programs.

[0239] The processor 702 is used to execute the program stored in the memory 701. When the program is executed, it is specifically used to implement the audio impulse noise processing method provided in any one of the above embodiments.

[0240] Another embodiment of the present application provides a computer storage medium for storing a computer program. When the computer program is executed by a processor, it is used to implement the audio impulse noise processing method provided in any one of the above embodiments.

[0241] Computer storage media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0242] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0243] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing audio impulse noise, characterized in that: include: Separate the audio stream from the video source input to the player and decode the separated audio stream data; Whenever a frame of the decoded audio stream data to be played is read, the error of the autoregressive model corresponding to the audio data to be played is determined according to the type of the audio data to be played; wherein the type of the audio data to be played includes the first frame audio data, the middle frame audio data and the last frame audio data; Determining a threshold value corresponding to the audio data to be played according to an error of the autoregressive model corresponding to the audio data to be played; By comparing the audio data to be played with a threshold value corresponding to the audio data to be played, each segment of noise data in the audio data to be played is determined; According to the type of the audio data to be played and the interval where each section of noise data in the audio data to be played is located in the audio data to be played, the autoregressive model is used to predict each sampling point in each section of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data; wherein the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval; p is the order of the autoregressive model; The repaired audio data to be played is provided to the player for playing.

2. The method according to claim 1, characterized in that The step of determining the error of the autoregressive model corresponding to the audio data to be played according to the type of the audio data to be played includes: If the audio data to be played belongs to the first frame of audio data, a leading zero padding operation is performed on the audio data to be played to obtain the completed data of the audio data to be played; If the audio data to be played belongs to the middle frame audio data or the last frame audio data of the slice, the signals of the last p sampling points of the repaired audio data of the previous frame are combined with the audio data to be played to obtain the complementary data of the audio data to be played; Based on the supplementary data of the audio data to be played, the error of the autoregressive model corresponding to the audio data to be played is calculated using the Yule Walker equation; If the audio data to be played belongs to the last frame of audio data of the film source, the error of the autoregressive model corresponding to the previous frame of audio data is determined as the error of the autoregressive model corresponding to the audio data to be played.

3. The method according to claim 1, characterized in that The step of comparing the audio data to be played with a threshold value corresponding to the audio data to be played to determine each segment of noise data in the audio data to be played includes: The audio data to be played is filtered by using a quasi-differential FIR filter to obtain filtered data of the audio data to be played; wherein the quasi-differential FIR filter is a filter after the original signal is introduced into the FIR filter as feedback through quasi-differential; Determine the position of the sampling point in the filtered data of the audio data to be played that is not less than the threshold value corresponding to the audio data to be played, and obtain each filtered data noise point; Determine, according to each of the noise points of the filtered data, each of the corresponding noise points in the audio data to be played; The data between two adjacent noise points in the audio data to be played and whose interval is greater than the preset number of sampling points is determined as a section of noise data in the audio data to be played.

4. The method according to claim 1, characterized in that The audio data to be played belongs to the first frame of audio data, and the noise data is within the first p sampling points of the audio data to be played, and the autoregressive model is used to predict each sampling point in each section of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data, including: Reversing the current audio data to be played; The inverted audio data to be played is calculated using the Yule Walker equation to obtain model parameters corresponding to the inverted audio data to be played; Performing reverse linear prediction on each sampling point in the noise data by using the inverted model parameters corresponding to the audio data to be played through the autoregressive model to obtain a prediction signal for each sampling point in the noise data; The original signal of each sampling point in the noise data in the inverted audio data to be played is replaced with its predicted signal, and the inverted audio data to be played is inverted.

5. The method according to claim 1, characterized in that The audio data to be played belongs to the first frame of audio data, and the noise data is in the middle sampling point interval of the audio data to be played, and the autoregressive model is used to predict each sampling point in each segment of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data, including: Using the Yule Walker equation to calculate the current audio data to be played, to obtain the model parameters corresponding to the current audio data to be played; Using the autoregressive model and the model parameters corresponding to the current audio data to be played, forward prediction and backward prediction are performed on each sampling point in the noise data to obtain a forward prediction signal and a backward prediction signal of each sampling point in the noise data; The original signal of each sampling point in the noise data is replaced by a weighted result of the forward prediction signal and the backward prediction signal of each sampling point according to the corresponding first weight; wherein the first weight corresponding to the sampling point is determined according to the ratio of the point position of the sampling point to the length of the noise data.

6. The method according to claim 1, characterized in that The audio data to be played belongs to the first frame of audio data, and the noise data is within the interval of the last p sampling points of the audio data to be played, and the autoregressive model is used to predict each sampling point in each section of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data, including: Using the Yule Walker equation to calculate the current audio data to be played, to obtain the model parameters corresponding to the current audio data to be played; Using the autoregressive model and the model parameters corresponding to the current audio data to be played, the last p sampling points of the audio data to be played are predicted, and the first p sampling points of the next frame of audio data are predicted, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffer prediction signal of the next frame; Replacing the original signal of each sampling point belonging to the noise data among the last p sampling points of the audio data to be played with a weighted result of the original signal of the sampling point and the predicted signal of the sampling point according to the corresponding second weight; Replacing the original signal of each of the sampling points that do not belong to the noise data among the last p sampling points of the audio data to be played with the product of the original signal of the sampling point and the corresponding second weight; The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

7. The method according to claim 6, characterized in that The audio data to be played belongs to the intermediate frame audio data, and the autoregressive model is used to predict each sampling point in each segment of the noise data, and the signal of each sampling point in the noise data is interpolated and fused by the predicted data to repair the noise data, including: For each segment of the noise data in the audio data to be played, if the noise data is within the first p sampling points of the audio data to be played, the prediction signal of each sampling point in the noise data is predicted by using the autoregressive model and the model parameters corresponding to the current audio data to be played; wherein the model parameters corresponding to the audio data to be played are calculated in advance by using the Yule Walker equation on the signals of the last p sampling points of the transmitted previous frame of audio data and the spliced ​​data of the current audio data to be played; Replacing the original signal of each sampling point in the noise data with its prediction signal, and smoothing the signals of the first p sampling points of the current audio data to be played using the buffered prediction signal transmitted to the audio data to be played; If the noise data is within the middle sampling point interval of the audio data to be played, forward prediction and backward prediction are performed on each sampling point in the noise data by using the autoregressive model using the model parameters corresponding to the current audio data to be played, so as to obtain a forward prediction signal and a backward prediction signal of each sampling point in the noise data; Repairing each sampling point in the noise data by using a forward prediction signal and a backward prediction signal of each sampling point in the noise data; If the noise data is within the interval of the last p sampling points of the audio data to be played, the autoregressive model is used to predict the last p sampling points of the audio data to be played and the first p sampling points of the next frame of audio data by using the model parameters corresponding to the current audio data to be played, so as to obtain the prediction signal of the last p sampling points of the audio data to be played and the buffered prediction signal of the next frame; Repairing the last p sampling points of the audio data to be played using the prediction signals of the last p sampling points of the audio data to be played; The signals of the last p sampling points of the audio data to be played are combined with the buffered prediction signal of the next frame and transmitted to the next frame of audio data.

8. The method according to claim 1, characterized in that Also includes: If the audio data to be played does not contain noise, and the previous frame of audio data of the audio data to be played contains noise, then calculating the average signal loudness of the first p sampling points of the audio data to be played, and calculating the average signal loudness of the last p sampling points of the previous frame of audio data of the audio data to be played; If the difference between the average signal loudness of the first p sampling points of the audio data to be played and the average signal loudness of the last p sampling points of the previous frame of audio data is greater than a preset decibel, the audio data to be played is weakened; If the difference between the average signal loudness of the last p sampling points of the previous frame of audio data of the to-be-played audio data and the average signal loudness of the first p sampling points of the to-be-played audio data is greater than a preset decibel, the to-be-played audio data is enhanced.

9. An audio impulse noise processing device, characterized in that: include: A pre-processing unit, used to separate the audio stream from the video source input to the player and decode the separated audio stream data; The error calculation unit is used to determine the error of the autoregressive model corresponding to the audio data to be played every time a frame of the decoded audio stream data to be played is read according to the type of the audio data to be played; wherein the type of the audio data to be played includes the first frame audio data, the middle frame audio data and the last frame audio data; A threshold calculation unit, used to determine a threshold corresponding to the audio data to be played according to an error of the autoregressive model corresponding to the audio data to be played; a detection unit, configured to determine each segment of noise data in the audio data to be played by comparing the audio data to be played with a threshold value corresponding to the audio data to be played; A repair unit, used to predict each sampling point in each segment of the noise data according to the type of the audio data to be played and the interval where each segment of noise data in the audio data to be played is located in the audio data to be played, and interpolate and fuse the signals of each sampling point in the noise data through the predicted data to repair the noise data; wherein the interval of the audio data to be played includes the first p sampling point interval, the last p sampling point interval and the middle sampling point interval; p is the order of the autoregressive model; A feedback unit is used to provide the repaired audio data to be played to the player for playing.

10. A computer storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, is used to implement the audio impulse noise processing method according to any one of claims 1 to 8.

Images