Parametric array loudspeaker nonlinear distortion modeling and compensating method based on deep learning

Abstract

The invention discloses a parametric array loudspeaker nonlinear distortion modeling and compensation method based on deep learning. The method comprises the following specific steps: (1) recording an audio sound signal output by a parametric array loudspeaker system and an input signal of the audio sound signal at a listening position where nonlinear distortion needs to be suppressed, and making a data set for modeling; (2) performing overall modeling on the parametric array loudspeaker system by using a neural network F, and training the neural network F by using the data set; modeling a linear part of the system by using an FIR (Finite Impulse Response) filter model Flin; (3) using the trained neural network F and the model Flin to train a neural network G as an inverse filter of the system; and (4) preprocessing an audio signal expected to be replayed through the neural network G, and inputting the preprocessed audio signal into a parametric array loudspeaker system to realize nonlinear compensation of the system. According to the method, accurate modeling and effective compensation can be carried out on the nonlinear distortion of the parametric array loudspeaker, and the nonlinear distortion of the parametric array loudspeaker is remarkably inhibited.

Description

Technical Field

[0001] The present invention belongs to the technical field of loudspeakers and proposes a method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning. Background Art

[0002] The working principle of the parametric array loudspeaker is to modulate the audio signal to be played onto the ultrasonic carrier. The modulated sound signal is amplified by the amplifier and then emitted by the ultrasonic transducer. The audio signal is self-demodulated in the air through the nonlinear effect of the air and heard by the human ear. Compared with traditional loudspeakers, parametric array loudspeakers can produce strong directivity at low frequencies, which is conducive to the transmission of directional sound and improves the confidentiality of speech. However, due to its special sound generation mechanism, the parametric array loudspeaker has serious nonlinear distortion, which greatly limits its application. The most advanced distortion suppression method based on Volterra filter (Mu Y, Ji P, Ji W, et al. Modeling and compensation for the distortion of parametric loudspeakers using a one-dimension Volterra filter [J]. IEEE / ACM Transactions on Audio, Speech, and Language Processing, 2014, 22 (12): 2169-2181.) still has difficulty in reducing the nonlinear distortion of parametric array loudspeakers to a low level. Summary of the invention

[0003] In order to solve the problem of serious nonlinear distortion of existing parametric array loudspeakers, the present invention proposes a nonlinear distortion modeling and compensation method for parametric array loudspeakers based on deep learning.

[0004] The technical solution adopted by the method of the present invention is:

[0005] The nonlinear distortion modeling and compensation method of the parametric array loudspeaker based on deep learning includes the following steps:

[0006] (1) Record the audio signal output by the parametric array speaker system and the input signal of the system at the listening position where nonlinear distortion needs to be suppressed, and create a data set for system modeling;

[0007] (2) Use a deep neural network F to model the parametric array speaker system as a whole, use the data set obtained in step (1) to train the neural network F; and use the FIR filter model F lin Model the linear portion of the system;

[0008] (3) Using the trained neural network F and FIR filter model F lin Train another deep neural network G as the inverse filter of the system;

[0009] (4) The desired audio signal is preprocessed by the deep neural network G and then input into the parametric array speaker system to compensate for the nonlinearity of the system.

[0010] Furthermore, in step (1), the electrical signal u[n] input to the parametric array speaker system and the acoustic signal y[n] collected at the listening position are recorded simultaneously, and the recorded input and output signals are aligned and segmented into segments for deep neural network training.

[0011] Furthermore, in step (2), the input of the deep neural network F is the recorded system input signal u[n], and the output is the estimated system output signal To reduce The difference between y[n] and y[n] is used to construct the loss function To train the network.

[0012] Furthermore, in the step (1), after training the neural network F, it is calculated whether the system total harmonic distortion and intermodulation distortion estimated by the neural network F and the actual measured value errors meet the threshold conditions. If so, it is considered that the system modeling is successful.

[0013] Furthermore, in step (1), the model F lin The input is the recorded system input signal u[n], and the output is the estimated system linear output signal By reducing The error between y[n] and update the model F lin until convergence.

[0014] Furthermore, in step (3), the input of the deep neural network G is the desired playback signal q[n], and the output is the preprocessed signal z[n]; q[n] is sequentially passed through the deep neural networks G and F to obtain the output signal of the parametric array speaker system estimated by the network F with the preprocessed signal z[n] as input. Add a certain delay to q[n] and then pass it through model F lin It can be obtained from the model F lin Estimation of the linear output signal of a parametric array loudspeaker system To reduce and The difference between them is used to construct the loss function To train the network G.

[0015] Furthermore, in step (4), the signal q[n] to be reproduced is pre-processed offline by the trained neural network G and then input into the parametric array speaker system to achieve compensation for the nonlinearity of the system.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention can significantly reduce the total harmonic distortion and intermodulation distortion of the audio sound generated by the parametric array loudspeaker, and the performance is significantly better than the most advanced Volterra filter-based method. The present invention provides a new method for improving the sound quality of the sound reproduction of the parametric array loudspeaker. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flowchart of the method of the present invention.

[0018] Figure 2 Schematic diagram of the structure of the WaveNet neural network used in the embodiment of the present invention.

[0019] Figure 3 1 is a comparison diagram of (a) total harmonic distortion (THD) and measured values ​​of the system estimated by the WaveNet neural network F obtained by system modeling in an embodiment of the present invention, and (b) intermodulation distortion (IMD) and measured values.

[0020] Figure 4 : This is a comparison diagram of the system (a) linear response sound pressure level (SPL), (b) THD and (c) IMD before and after compensation between the method of the present invention and the most advanced method based on Volterra filter. DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with the implementation modes and the accompanying drawings.

[0022] The process of the method of the present invention is as follows Figure 1 As shown, it includes the steps of data set recording, system overall modeling, system linear part modeling, inverse filter training and compensation. The specific implementation is as follows:

[0023] (1) First, determine the listening position where nonlinear distortion needs to be suppressed, and record the audio input and output signals of the parametric array speaker system at this position. In this embodiment, the input signal of the parametric array speaker system includes various types of signals such as speech, music, and ambient sound with a total duration of more than 2 hours for the overall modeling of the deep learning system (taken from the public data set NIGENS: Ivo Trowitzsch, Jalil Taghia, Youssef Kashef, and Klaus Obermayer (2019). The NIGENS general sound events database. Technische Berlin, Tech. Rep. arXiv: 1902.08314 [cs. SD]), with band-limited white noise for modeling the linear part of the system, and swept-frequency signals and single-frequency plus swept-frequency signals for calculating the total harmonic distortion (THD) and intermodulation distortion (IMD) of the system.

[0024] The electrical signal u[n] of the input parametric array speaker system and the acoustic signal y[n] collected by the microphone at the listening position are recorded simultaneously by the PULSE system in an anechoic chamber with a sampling rate of 44100 Hz. The time delay between u[n] and y[n] is obtained by calculating the cross-correlation between the two and aligning them. The signal used for overall modeling of the deep learning system is divided into sample segments with a length of 65536 sampling points, 70% of which are used as training sets, 20% as validation sets, and 10% as test sets. In addition, the input and output pairs of the system input, which are swept frequency signals and single frequency plus swept frequency signals, are also used as part of the test set to test the accuracy of the model's estimation of the system's THD and IMD.

[0025] (2) Next, the deep learning neural network F is trained based on the sorted data set to model the entire parametric array speaker system (including the linear part and the nonlinear part), such as Figure 1 The input of network F is the recorded system input signal u[n], and the output is its estimated system output signal The loss function used in training is

[0026] In this embodiment, the network F specifically adopts a feedforward variant of the WaveNet neural network, and its structure is as follows: Figure 2As shown. The input and output of the network are the system input signal sequence and its estimated system output signal sequence, respectively. The input signal first passes through a point-by-point one-dimensional convolution layer to adjust the number of channels from 1 to C; then the signal passes through a series of residual modules in turn, which consists of an expanded one-dimensional convolution layer, a gated Tanh activation, and a point-by-point one-dimensional convolution layer; each residual module has two outputs: one is the output signal v of the point-by-point one-dimensional convolution layer k [n], this signal is input to the "linear mixing layer"; another output signal u k+1 [n] The input u of this module is connected by residual connection k [n] is added to the output v of the point-by-point one-dimensional convolutional layer k [n], which is the input of the next residual module; the "linear mixing layer" is implemented by a point-by-point one-dimensional convolution layer, and the output of each residual module input into the "linear mixing layer" is the output signal of the WaveNet network. In this embodiment, the WaveNet structure used to implement the network F contains 9 residual modules, in which the expansion factors of the expansion convolution layer are 1, 2, 4, ..., 256, respectively, the number of channels C of all convolution layers is 16, and the convolution kernel size is 16.

[0027] In this embodiment, the loss function used to train the network F is

[0028]

[0029] where y and are the measured system output signal and model output signal, Y and These are the time spectra of the two respectively.

[0030] After training, if the error between the total harmonic distortion and intermodulation distortion of the system estimated by the network F and the measured value is less than the acceptable upper limit (such as 2%), the system modeling is considered successful. In this embodiment, the THD and IMD estimated by the trained WaveNet neural network F are compared with the measured values. Figure 2 As shown, the average errors of the two are 1.08% and 0.34% respectively, and it is considered that the overall system modeling of this embodiment is successful.

[0031] (3) Then model the linear part of the system, such as Figure 1 In this embodiment, based on the recorded system input, the input signal u[n] and the output signal y[n] are band-limited white noise, and the least mean square (LMS) algorithm is used to reduce the FIR filter model F lin Output The error between y[n] and update model F linIn this embodiment, the length of the FIR (finite impulse response) filter is set to 160.

[0032] (4) Then use the network F trained in step (2) and the FIR filter F obtained in step (3) lin Train the inverse filter G, such as Figure 1 As shown in (c). The input of network G is the desired playback signal q[n], and the output is the preprocessed signal z[n]; passing q[n] through the two neural networks G and F in turn, the output signal of the parametric array speaker system estimated by network F with the preprocessed signal z[n] as input can be obtained. Add a certain delay to q[n] and then pass it through the FIR filter model F lin It can be obtained from the model F lin Estimation of the linear output signal of a parametric array loudspeaker system To reduce and The difference between them is used to construct the loss function To train the network G; the loss function used in training is

[0033] In this embodiment, the network G is also implemented using a feedforward variant of the WaveNet neural network. In addition, the amplitude of the signal z[n] needs to be within the input dynamic range of the real system, so a Tanh activation function is added after its "linear mixing layer" to limit the network output amplitude. The network contains 24 residual modules, in which the expansion factors of the expansion convolution layer are 1, 2, 4, ..., 2048, 1, 2, 4, ..., 2048, respectively. The number of channels of all convolutional layers is 24, and the convolution kernel size is 4.

[0034] In this embodiment, the loss function used to train the network G is

[0035]

[0036] in is the linear output signal of the system, which is the expected signal plus a time delay and passes through the linear model F lin In this embodiment, the time delay is set to 100 sampling points (about 2.3 ms). The system output signal estimated by model F is obtained by passing the expected signal through models G and F in sequence. and They are and Time spectrum of .

[0037] (5) Finally, the inverse filter G obtained by the above training is used to perform offline preprocessing on the audio signal to be reproduced, and then input into the parametric array speaker system, thereby compensating for the nonlinearity of the system and reducing the nonlinear distortion of the system.

[0038] In this embodiment, the method of the present invention and the most advanced method based on Volterra filter (Mu Y, Ji P, Ji W, et al. Modeling and compensation for the distortion of parametric loudspeakers using a one-dimension Volterra filter [J]. IEEE / ACM Transactions on Audio, Speech, and Language Processing, 2014, 22 (12): 2169-2181.) are used to calculate the linear response sound pressure level (SPL), THD and IMD of the system before and after compensation. Figure 4 As shown. The method of the present invention can reduce the average THD and average IMD of the parametric array loudspeaker to less than 5% and 3% respectively, and the THD and IMD compensated by the method of the present invention are better than the method based on Volterra filter at all frequencies. It can be seen that the method of the present invention can significantly reduce the total harmonic distortion and intermodulation distortion of the parametric array loudspeaker system without affecting the linear response, and the performance is significantly better than the most advanced method based on Volterra filter.

Claims

1. A method for modeling and compensating nonlinear distortion of parametric array loudspeakers based on deep learning, characterized in that: The method comprises the following steps: (1) Record the audio signal output by the parametric array speaker system and the input signal of the system at the listening position where nonlinear distortion needs to be suppressed, and create a data set for system modeling; (2) Use a deep neural network F to model the parametric array speaker system as a whole, use the data set obtained in step (1) to train the neural network F; and use the FIR filter model F lin Model the linear portion of the system; (3) Using the trained neural network F and FIR filter model F lin Train another deep neural network G as the inverse filter of the system; (4) The desired audio signal is preprocessed by the deep neural network G and then input into the parametric array speaker system to compensate for the nonlinearity of the system.

2. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 1, characterized in that: In the step (1), the electrical signal u[n] input to the parametric array speaker system and the acoustic signal y[n] collected at the listening position are recorded simultaneously, and the recorded input and output signals are aligned and segmented into segments for deep neural network training.

3. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 1, characterized in that: In step (2), the input of the deep neural network F is the recorded system input signal u[n], and the output is the estimated system output signal To reduce The difference between y[n] and y[n] is used to construct the loss function To train the network.

4. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 1, characterized in that: In the step (1), after training the neural network F, it is calculated whether the errors between the total harmonic distortion and intermodulation distortion of the system estimated by the neural network F and the actual measured values ​​meet the threshold conditions. If so, it is considered that the system modeling is successful.

5. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 1, characterized in that: In the step (1), the model F lin The input is the recorded system input signal u[n], and the output is the estimated system linear output signal By reducing The error between y[n] and update the model F lin until convergence.

6. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 1, characterized in that: In step (3), the input of the deep neural network G is the desired playback signal q[n], and the output is the preprocessed signal z[n]; q[n] is sequentially passed through the deep neural networks G and F to obtain the output signal of the parametric array speaker system estimated by the network F with the preprocessed signal z[n] as input Add a certain delay to q[n] and then pass it through model F lin It can be obtained from the model F lin Estimation of the linear output signal of a parametric array loudspeaker system To reduce and The difference between them is used to construct the loss function To train the network G.

7. The method for modeling and compensating nonlinear distortion of a parametric array loudspeaker based on deep learning as claimed in claim 6, characterized in that: In step (4), the signal q[n] to be reproduced is pre-processed offline by the trained neural network G and then input into the parametric array speaker system to achieve compensation for the nonlinearity of the system.

Images