An audio encoding and decoding method based on compression sensing
By adopting adaptive compression perception and sparse decomposition technology in audio signal processing and combining psychoacoustic models, the problem of wasted sampling and storage resources in audio signal processing in the prior art is solved, and efficient audio encoding and codec and quality improvement are achieved.
Patent Information
- Application Number
- CN202111260920.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-28
- Publication Date
- 2024-12-31
- Estimated Expiration
- 2041-10-28
AI Technical Summary
The existing audio signal processing technology has shortcomings in the waste of sampling and storage resources, and has not fully utilized the sparse characteristics of audio signals in different transformation domains, resulting in low encoding efficiency.
Using the audio encoding and decoding method based on compression perception, the sparse description and observation matrix are constructed by adaptive compression sensing energy weighting and sparse decomposition, combined with the characteristics of the audio signal, an adaptive redundant dictionary is established, and the sparse decomposition and reconstruction of the audio signal is carried out. Combining psychoacoustic models improves encoding and decoding efficiency.
It realizes high compression ratio audio encoding and decoding, saves sampling and storage resources, and improves the encoding efficiency and quality of audio signals.
Smart Images
Figure CN113990335B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to an audio coding and decoding method based on compression sensing, and belongs to the technical field of sound processing. Background Art
[0002] Data compression is an important topic in information theory research, and is called source coding in information theory. In recent years, data compression has not only been limited to the research and discussion of coding methods, but has gradually formed a relatively unique system. It mainly studies the representation, transmission and conversion methods of data, with the aim of reducing the storage space occupied by data and the time required for transmission.
[0003] Recent studies have shown that audio signals also have certain sparsity in a certain transform domain, so compressed sensing can be combined with audio signals to explore various new methods in the field of audio signal processing. Traditional audio signal processing is based on the Nyquist sampling theorem, and the sampling frequency is at least 8kHz, or even more. In the specific audio signal processing process, further compression is required. For example, in audio recognition, high-speed sampling is first performed, and then a small number of feature parameters are extracted, and finally recognition is performed, which wastes a lot of sampling and storage resources. How to remodel the audio signal to obtain fewer samples without affecting the quality of the audio is a research hotspot in the current field of audio signal processing. At present, there are relatively few studies on the use of compressed sensing in the field of audio signal processing at home and abroad, and it is in its infancy. It mainly involves the reconstruction performance research of approximately sparse sound signals under different bases, the application of compressed sensing based on the sine coding framework, the improvement of speech recognition performance of compressed sensing in a noisy environment, and information hiding. Theoretical research, they apply perceptual compression to different audio application fields, and rarely comprehensively analyze the sparse characteristics of audio signals in different orthogonal transform domains or atomic dictionaries and the performance impact of their compressed reconstruction characteristics on the particularity of audio signals. At the same time, these previous studies have shown that audio signals (including speech signals) have sparse characteristics in many transformation domains such as frequency domain, KLT, wavelet and DCT, which has been applied to various audio coding or speech recognition related audio signal processing algorithms. The present invention combines compressed sensing with audio signals, takes advantage of the characteristics of compressed sensing measurement with few sampling values, saves sampling, storage and transmission resources, and studies new research methods and new application fields in the field of audio compression in audio signal processing. Summary of the invention
[0004] Purpose of the invention: In order to overcome the deficiencies in the prior art, the present invention provides an audio coding and decoding method based on compressed sensing, which solves the problem of low-complexity and low-storage audio coding in actual audio communication systems.
[0005] Technical solution: To achieve the above purpose, the technical solution adopted by the present invention is:
[0006] An audio coding and decoding method based on compressed sensing, the coding method comprises the following steps:
[0007] Step 1: Get the input audio signal.
[0008] Step 2: Adaptive compressed sensing energy weighting based on sub-band energy, using sub-band energy statistics to perform adaptive compressed sensing energy weighting on the audio signal.
[0009] Step 3, adaptive compressed sensing sparse decomposition, divide the audio signal into different energy intervals according to energy, allocate the corresponding number of observations to each interval according to the interval energy, and adaptively allocate the number of observations according to the number of observations in the energy interval and the energy. Then, when the number of observations is determined, a random complete observation is constructed according to the non-adaptive compressed sensing theory, and the corresponding number of observations and the corresponding observation matrix are adaptively selected according to the energy of each component of the complete observation to reconstruct the original signal.
[0010] Step 4, using the sparse transformation matrix and observation matrix based on the audio signal characteristics, using the data classification statistical training method to obtain the sparse description and observation matrix suitable for the audio signal characteristics, and using two methods to decompose the sparse description of the audio signal. First, according to different signal categories, the sparse description degree of the low-frequency coefficients and high-frequency coefficients of the MDCT transform for different audio signals is obtained, and a local sparse description model of the audio signal is established. Then, an over-complete sparse decomposition redundant dictionary is constructed by data training, and the K singular decomposition learning algorithm is introduced to construct an over-complete redundant dictionary. Starting from the MDCT transform expansion of the audio signal, an adaptive redundant dictionary of the audio signal is constructed by solving the integral equation of the exponential decay autocorrelation function, and a signal sparse representation algorithm based on nonlinear approximation is designed by the algebraic structure of the dictionary, and a sparse decomposition model of compressed sensing of the audio signal is established, and then the coding quantization parameter is obtained.
[0011] Step 5: Input the audio signal into the adaptive compressed sensing psychoacoustic model to obtain psychoacoustic parameters.
[0012] Step 6: quantize the psychoacoustic parameters and the coding quantization parameters.
[0013] Step 7: entropy coding the psychoacoustic parameters and coding quantization parameters after the coefficients are quantized.
[0014] Step 8: Pack the entropy-coded psychoacoustic parameters and coding quantization parameters into a bitstream to form a coded data stream.
[0015] Preferably: the decoding method includes the inverse process of the encoding method, firstly obtaining the encoded data stream, performing data dequantization, and combining and restoring multiple frame signals to form a decoded audio data stream.
[0016] Preferably, for the reconstruction of the audio signal at the decoding end, in combination with the characteristics of the audio signal, only the l1 reconstruction, codebook prediction reconstruction, and l1 combined codebook reconstruction methods are used for the compressive sensing reconstruction characteristics of the audio signal, and the signal reconstruction signal-to-noise ratio is examined in the case of no coding quantization error.
[0017] Preferably, the discrete signal x0 ∈ R n is k-sparse in the orthogonal basis Ψ = {ψ i |ψ i ∈ R n , i = 1, 2,..., n}, that is:
[0018] x0 = Ψ -1 θ(1)
[0019] where the sparse coefficient vector θ = Ψx0 = (θ1, θ2,..., θ n ) T has only k non-zero coefficients, that is, ||θ||0 = k.
[0020] The k-sparse signal x0 is projected to generate m observations y = (y1, y2,..., y m ) T That is:
[0021] y = Φx0 = ΦΨ -1 θ = Tθ(2)
[0022] At this time, the solution of reconstructing x0 or θ from y is obtained by solving the l1 optimization problem:
[0023]
[0024] where m < n, Φ is the observation matrix, and T is the recovery matrix.
[0025] Preferably, the solution of formula (3) is obtained by using the orthogonal matching method or the matching pursuit method.
[0026] Compared with the prior art, the present invention has the following beneficial effects:
[0027] 1. Through the sparse redundant dictionary description of the audio signal, an adaptive dictionary training algorithm is obtained and combined with the compressive sensing reconstruction algorithm to propose a high-compression-rate audio codec technology framework based on compressive sensing.
[0028] 2. Application research of the adaptive compressive sensing method in the audio codec, and the compression rate and sound quality of the audio codec are improved by combining with the psychoacoustic model. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 is a schematic structural diagram of the present invention. DETAILED DESCRIPTION OF THE INVENTION
[0030] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.
[0031] An audio coding and decoding method based on compressed sensing, such as Figure 1 As shown, the encoding method includes the following steps:
[0032] Get the input audio signal.
[0033] The structure of the compressed sensing system of the present invention is similar to the traditional audio codec framework. The main difference is that the type of signal to be quantized in the audio coding is a reduced-dimensional signal after sparse decomposition of compressed sensing. The decoding output needs to perform a sparse decomposition audio signal reconstruction operation. The interval energy method is used for adaptive compressed sensing coding in the coding and decoding process. The adaptive compressed sensing audio coding is divided into two parts:
[0034] Adaptive compressed sensing energy weighting based on sub-band energy. Since the human ear has different sensitivities to different frequency components of the audio signal, and there is a mutual masking process between different frequency band components, optimizing the compressed sensing sampling process according to the perception characteristics of the human ear is of great significance for improving the efficiency of compressed sensing sampling. Compressed sensing sampling based on the auditory characteristics of the human ear requires weighted sampling of signals of different frequency bands in the Bark domain according to the psychoacoustic model, so that the compressed sensing sampling process better conforms to the masking characteristics of the psychoacoustic model. The present invention adopts sub-band energy statistics to perform adaptive compressed sensing energy weighting.
[0035] Adaptive compressed sensing sparse decomposition divides the audio signal into different energy intervals according to energy, and allocates the corresponding number of observations to each interval according to the interval energy. Then, when the number of observations is determined, a random complete observation is constructed according to the non-adaptive compressed sensing theory, and the corresponding number of observations and the corresponding observation matrix are adaptively selected according to the energy of each component of the complete observation to reconstruct the original signal.
[0036] Compressed sensing model of audio signals
[0037] Compressed sensing assumes that the discrete signal x0∈R n In the orthogonal basis Ψ={ψ i |ψ i ∈R n ,i=1,2,...,n} is k-sparse, that is:
[0038] x0=Ψ -1 θ (1)
[0039] where the sparse coefficient vector θ = Ψx0 = (θ1, θ2,..., θ n ) T has only k non-zero coefficients, i.e., ||θ0 = k. The compressive sensing theory projects the k-sparse signal x0 to generate m (m < n) observations y = (y1, y2,..., y m ) T That is:
[0040] y = Φx0 = ΦΨ -1 θ = Tθ (2)
[0041] where Φ is the observation matrix and T is the recovery matrix. At this time, the solution of x0 or θ can be reconstructed from y by solving the l1 optimization problem:
[0042]
[0043] The solution of formula (3) can be obtained by using orthogonal matching pursuit and matching pursuit, etc.
[0044] According to the compressive sensing theory above, the premise for a signal to adopt the compressive sensing theory is that the signal can be sparsely represented under a certain transformation. That is, when the signal is projected onto the transform basis, the absolute values of most transform coefficients are very small. Therefore, the obtained transform vector is sparse or approximately sparse, and it can be regarded as a concise expression of the original signal. Usually, the transform basis can be flexibly selected according to the characteristics of the signal itself. Commonly used ones include DCT transform basis, FFT transform basis, discrete wavelet transform basis, Curvelets basis, Gabor basis, and redundant dictionaries. However, for audio signals, the above transform bases cannot be used as the sparse decomposition transform bases for different audio categories. For example, for strongly periodic audio signals, the DCT transform basis is approximate coefficients, and for noise-type audio signals, the DCT transform basis cannot describe its coefficient characteristics. A single orthogonal transform basis cannot well describe the sparse characteristics of audio signals. The present invention adopts a sparse transform matrix and an observation matrix based on the characteristics of audio signals, obtains a sparse description and an observation matrix suitable for the characteristics of audio signals by means of data classification and statistical training, and adopts two methods for sparse description and decomposition of audio signals.
[0045] Firstly, a local audio signal sparse description method is used to perform sparse description decomposition of the audio signal. In audio coding, MDCT (modified MDCT transform) is widely used. It is an orthogonal transform basis that approximates DCT and KL transform, and does not have spectrum leakage and block effect similar to FFT transform. DCT transform and KL transform of audio signals are also approximate sparse transforms. Studies have shown that MDCT transform also has sparse transform characteristics for periodic audio signals, and its subjective test results of reconstructed signal-to-noise ratio are significantly better than DCT transform. However, MDCT transform has weak sparse characteristics for noise-type signals. The present invention obtains the sparse description degree of low-frequency coefficients and high-frequency coefficients of MDCT transform for different audio signals according to different signal categories, and establishes a local audio signal sparse description model.
[0046] Secondly, a data training method is used to construct an over-complete sparse decomposition redundant dictionary, and the K singular decomposition learning algorithm is introduced to construct an over-complete redundant dictionary. Starting from the MDCT transform of the audio signal, an adaptive redundant dictionary of the audio signal is constructed by solving the integral equation of the exponentially decaying autocorrelation function. The signal sparse representation algorithm based on nonlinear approximation is designed based on the algebraic structure of the dictionary, and a sparse decomposition model of compressed sensing of audio signals is established.
[0047] For the audio signal reconstruction at the decoding end, combined with the characteristics of the audio signal, only the l1 reconstruction, codebook prediction reconstruction and l1 joint codebook reconstruction methods are used to perform the compressed sensing reconstruction characteristics of the audio signal, and the signal reconstruction signal-to-noise ratio is examined in the absence of coding quantization error.
[0048] The audio signal is input into the adaptive compressed sensing psychoacoustic model to obtain psychoacoustic parameters. The psychoacoustic parameters and coding quantization parameters are coefficient quantized. The coefficient quantized psychoacoustic parameters and coding quantization parameters are entropy encoded. The entropy encoded psychoacoustic parameters and coding quantization parameters are packaged into code streams to form a coded data stream.
[0049] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. An audio coding and decoding method based on compressed sensing, characterized in that: The encoding method includes the following steps: Step 1: Obtain the input audio signal; Step 2: Based on the adaptive compressive sensing energy weighting of subband energy, perform adaptive compressive sensing energy weighting on the audio signal in a manner of subband energy statistics; Step 3: Adaptive compressive sensing sparse decomposition. Divide the audio signal into different energy intervals according to the energy, and allocate corresponding observation numbers to each interval according to the interval energy. Adaptively allocate the observation numbers according to the observation numbers and energy of the所属 energy interval. Then, after the observation numbers are determined, construct a random complete observation according to the non - adaptive compressive sensing theory. According to the energy of each component of the complete observation, adaptively select the corresponding number of observations and the corresponding observation matrix to reconstruct the original signal; For the reconstruction of the audio signal at the decoding end, combined with the characteristics of the audio signal, only use the l1 reconstruction, codebook prediction reconstruction, and l1 joint codebook reconstruction methods to perform the compressive sensing reconstruction characteristics of the audio signal, and examine the signal reconstruction signal - to - noise ratio in the case of no coding quantization error; Discrete signal x0∈R n In the orthogonal basis Ψ={ψ i |ψ i ∈R n ,i=1,2,...,n} is k-sparse, that is: x0=Ψ -1 i (1) where the sparse coefficient vector θ=Ψx0=(θ1,θ2,...,θ n ) T There are only k non-zero coefficients, i.e. ||θ||0=k; The k sparse signal x0 is projected to generate m observations y=(y1,y2,...,y m ) T Right now: y=Φx0=ΦΨ -1 θ=Tθ (2) At this time, solve the l1 optimization problem to reconstruct the solution of x0 or θ from y: Where m < n, Φ is the observation matrix, and T is the recovery matrix; Step 4: Use a sparse transformation matrix and an observation matrix based on the characteristics of the audio signal, and obtain a sparse description and an observation matrix suitable for the characteristics of the audio signal by means of data classification and statistical training, and perform sparse description decomposition of the audio signal in two ways. First, according to different signal categories, obtain the degree of sparse description of different audio signals by the low - frequency coefficients and high - frequency coefficients of the MDCT transform, and establish a local sparse description model of the audio signal. Then, use the data training method to construct an over - complete sparse decomposition redundant dictionary, introduce the K - singular value decomposition learning algorithm to construct the over - complete redundant dictionary. Starting from the MDCT transform expansion of the audio signal, construct an adaptive redundant dictionary of the audio signal by solving the integral equation of the exponentially decaying autocorrelation function, and design a signal sparse representation algorithm based on nonlinear approximation from the algebraic structure of the dictionary, establish a sparse decomposition model of compressive sensing of the audio signal, and then obtain the coding quantization parameters; Step 5: Input the audio signal into the adaptive compressive sensing psychoacoustic model to obtain psychoacoustic parameters; Step 6: Quantize the coefficients of the psychoacoustic parameters and the coding quantization parameters; Step 7: Perform entropy coding on the coefficient - quantized psychoacoustic parameters and coding quantization parameters; Step 8: Pack the entropy - coded psychoacoustic parameters and coding quantization parameters into a bitstream to form an encoded data stream.
2. The audio encoding and decoding method based on compressed sensing according to claim 1, characterized in that: The decoding method is the inverse process of the encoding method. First, obtain the encoded data stream, perform data de - quantization, and combine and restore multi - frame signals to form a decoded audio data stream.
3. The audio encoding and decoding method based on compressed sensing according to claim 2, characterized in that: The solution of formula (3) is obtained by using the orthogonal matching method or the matching pursuit method.