A high-quality audio generation method for DAS systems

Abstract

The present invention discloses a high-quality audio generation method for a DAS system, which pre-processes and extracts features of low-quality and high-quality audio data, and obtains data sets D1 and D2 respectively, which are a collection of pitch feature-Mel spectrum feature data pairs of multiple short time frames; constructs a high-quality audio generation model including a generative network and a discriminant network with multi-channel input; uses D2 to train the generative network to learn the prior distribution of high-quality audio; the discriminant network judges the generated signal of the generative network, and if the loss function is less than a set threshold, the training is terminated, otherwise the generative network and the discriminant network are optimized and the training is repeated; then the high-quality audio generation model is trained with D1 and D2; after pre-processing and feature extraction of the low-quality audio to be processed, the trained high-quality audio generation model is input to obtain high-quality audio. The present invention improves the spatial resolution and consistency of audio signals, and can generate high-quality audio more accurately.

Description

Technical Field

[0001] The invention relates to the fields of audio synthesis and analysis and distributed optical fiber acoustic sensing, and in particular to a high-quality audio generation method for a DAS system. Background Art

[0002] Distributed Acoustic Sensing (DAS) systems have been widely used in recent years. By distributing multiple acoustic sensors at different locations, DAS systems can comprehensively monitor environmental sounds. However, the quality of audio data collected by the DAS system varies, and is usually affected by factors such as environmental noise, sensor quality, and installation location, resulting in a lot of noise and distortion in the audio signal. This low-quality audio data faces many challenges in subsequent processing and application, especially in scenarios that require high-quality audio input, where its performance is particularly unsatisfactory. High-quality audio means that the signal strength in the audio is greater than the noise strength and the distortion is low, which gives a good auditory experience.

[0003] Traditional audio processing methods rely on a lot of manual intervention and complex preprocessing steps, including noise reduction, signal enhancement, and feature extraction. These methods are not only time-consuming and labor-intensive, but also difficult to ensure the consistency of processing effects. Especially when faced with low-quality audio data, the limitations of these traditional methods are more obvious, and they cannot effectively improve the overall quality of the audio. On the other hand, some existing machine learning and deep learning methods have made certain progress in the field of audio processing, but these methods usually require a large amount of high-quality audio data for training to learn clear and accurate audio features from the data. In the field of audio processing, collecting a large amount of high-quality audio data is a time-consuming and costly process, and the processing effects of these methods on low-quality audio data still need to be improved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention proposes a high-quality audio generation method for a DAS system.

[0005] The specific technical solutions are as follows:

[0006] A high-quality audio generation method for a DAS system comprises the following steps:

[0007] S1: Data is collected and processed at multiple optical fiber points of the DAS system to obtain low-quality audio data, which is preprocessed and feature extracted to obtain multiple short-time frame pitch feature-Mel spectrum feature data pairs, which are collected to obtain data set D1;

[0008] S2: Obtain high-quality audio data, preprocess and extract features using the method in S1 to obtain data set D2;

[0009] S3: Based on the sound features extracted by D2, a high-quality audio generation model is constructed, and the high-quality audio generation model includes a generation network and a discriminant network with multi-channel input; the generation network obtains a pitch feature that matches the Mel spectrum feature length according to a linear interpolation function and a pitch feature-Mel spectrum feature data pair, and after generating a multi-channel signal template based on this, downsampling and upsampling are performed, wherein the input of the first upsampling convolution layer is the concatenation result of the output of the downsampling module and the Mel spectrum feature after convolution, and the input of the subsequent upsampling convolution layer is the concatenation result of the output of the previous downsampling convolution layer and the output of the upsampling convolution layer; the discriminant network includes a multi-period discriminator for capturing the periodic features of the audio signal, and a multi-resolution discriminator for analyzing the details of the audio signal at different frequencies;

[0010] S4: Train the generative network with D2 as the training set to learn the prior distribution of high-quality audio; the discriminant network judges the generated signal of the generative network, and calculates the loss function based on the judgment result, and optimizes the generative network and the discriminant network; judge whether the loss function is less than the set threshold, if not, repeat the training process; otherwise, execute S5;

[0011] S5: Using datasets D1 and D2 as training sets, the method of S4 is used to jointly train the generative network and the discriminative network, and finally a trained high-quality audio generation model is obtained;

[0012] S6: After preprocessing and feature extraction of the low-quality audio to be processed, the audio is input into a trained high-quality audio generation model to obtain high-quality audio.

[0013] Furthermore, in S1, preprocessing and feature extraction are implemented through the following sub-steps:

[0014] S1.1: Normalize the amplitude of the audio signal to make it have a uniform volume level; normalize the audio signal to obtain the normalized audio signal x norm (n);

[0015] Use the sliding window method to convert x norm (n) is divided into multiple short time frames, where the mth short time frame signal is x n (m) indicates;

[0016] S1.2: An improved pitch extraction algorithm is used to extract the pitch feature f0 of the audio signal in each short time frame; the improved pitch extraction algorithm includes speech endpoint detection and high-frequency signal enhancement, as follows:

[0017] S1.2.1, speech endpoint detection: calculate the short-time energy and zero-crossing rate of the input short-time frame signal, and then determine whether the short-time frame signal is an audio signal according to the set threshold;

[0018] The expression of the short-time energy E(m) of the mth short-time frame signal is as follows:

[0019] ;

[0020] In the formula, R represents the overlap ratio, represents the sample point of the m-th short time frame, and N represents the frame length of the m-th short time frame;

[0021] The threshold expression of short-time energy is:

[0022] ;

[0023] In the formula, ρ is a proportional coefficient less than 1, which is set manually;

[0024] The expression of the zero crossing rate Z(m) of the mth short time frame signal is as follows:

[0025] ;

[0026] Where sgn() represents the sign function. If the input value is positive, the output sgn() is 1; if the input value is zero or negative, the output sgn() is 0.

[0027] The threshold Z of the zero crossing rate threshold It is set by humans;

[0028] The threshold of short-time energy and the threshold of zero-crossing rate are used for endpoint detection. If VAD(m) is 1, the m-th frame short-time frame signal is determined to be an audio signal. If VAD(m) is 0, the m-th frame short-time frame signal is determined to be a non-audio signal and the short-time frame signal is discarded. The judgment expression of endpoint detection is as follows:

[0029] ;

[0030] S1.2.2, High-frequency signal enhancement: For the audio signal detected by speech endpoints, use a pre-emphasis filter to emphasize the high-frequency part, and then perform autocorrelation calculation to detect the periodicity in the signal; calculate the pitch feature f0 based on the optimal period corresponding to the maximum value of the autocorrelation function, and the expression is as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] In the formula, x pe represents the pre-emphasized audio signal, α represents the pre-emphasis coefficient; R(τ) represents the autocorrelation function, τ represents the time delay, τ best Indicates the optimal period corresponding to the maximum autocorrelation function;

[0035] S1.3. Use Mel spectrum to represent pitch feature f0: Perform short-time Fourier transform on the audio signal processed by S1.2.2, and map the spectrum to the Mel scale to obtain the Mel spectrum feature, which is expressed as follows:

[0036] ;

[0037] ;

[0038] In the formula, X(f,t) represents the short-time Fourier transform result at frequency f and time t, w[ ] represents the window function, n0 represents the starting position of the window function, represents the complex exponential term of the short-time Fourier transform; f mel Represents Mel spectrum features;

[0039] Each short time frame gets a set of f0 and Mel spectrum features f mel , the set of all short time frame pitch feature-Mel spectrum feature data pairs is data set D1.

[0040] Furthermore, the generation network includes: a pitch interpolation module, a template generation module, a downsampling module, a Mel spectrum convolution module, an upsampling module, a LeakyReLU activation function, and a tanh activation function;

[0041] The input of the pitch interpolation module is the pitch feature-Mel spectrum feature data pair. Through the linear interpolation function, based on the time index of the pitch feature f0 and the f0 feature value, interpolation is performed on the time index of the Mel spectrum graph to obtain a pitch feature that matches the Mel spectrum feature length. The expression is as follows:

[0042] ;

[0043] In the formula, t i is the time index of the original pitch feature, f0(t i ) is the time index t i The eigenvalue of f0 at t mel is the time index of the Mel-spectrogram;

[0044] The template generation module is used to generate a multi-channel signal template according to the pitch feature that matches the Mel spectrum feature length. The expression is as follows:

[0045] ;

[0046] ;

[0047] Where, T a (t) represents the signal template of channel a, A(t) represents the amplitude, Indicates the phase of the ath channel; Represents a set of multi-channel signal templates, c is the number of channels, a=1,2,…,c;

[0048] The downsampling module includes three 7×7 downsampling convolutional layers, whose dilation rates are 1, 3, and 5 in order; a LeakyReLU activation function is used after each downsampling convolutional layer;

[0049] The Mel spectrum convolution module includes a one-dimensional 7×7 convolution layer with weight normalization and a padding of 3, and the length of the Mel spectrum feature of the input is consistent with the length of the Mel spectrum feature after the convolution of the output;

[0050] The upsampling module includes a 3×3 upsampling convolution layer with a dilation rate of 1, a 7×7 upsampling convolution layer with a dilation rate of 3, and an 11×11 upsampling convolution layer with a dilation rate of 5, and a LeakyReLU activation function is used after each upsampling convolution layer; the input of the first upsampling convolution layer is the concatenation result of the output of the downsampling module and the convolutional Mel spectrum feature;

[0051] The downsampling convolution layer and upsampling convolution layer of the same layer are skipped and connected through a splicing layer, and the splicing layer is used to splice the two inputs in the channel dimension; the two inputs of the splicing layer are the output of the downsampling convolution layer of the corresponding number of layers and the output of the upsampling convolution layer of the corresponding number of layers, respectively, and the output of the splicing layer of this layer is used as the input of the upsampling convolution layer of the next layer.

[0052] Furthermore, the multi-cycle discriminator includes: a spectrum conversion module, a convolution layer, and a LeakyReLU activation function; the spectrum conversion module is used to convert the time domain signal into a frequency domain signal through a short-time Fourier transform; the convolution layer includes 5 3×3 convolution layers, and a LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer is used to expand the input spectrum map from a single channel to 32 channels, maintaining the spatial dimension of the feature map; the middle three convolution layers are used to maintain the number of channels of the feature map at 32 layer by layer, so as to gradually reduce the height of the feature map and extract deeper features layer by layer; the last convolution layer reduces the number of output channels to 1 to integrate the features;

[0053] The multi-resolution discriminator includes: a period processing module, a convolution layer, and a LeakyReLU activation function; the period processing module is used to make the time length divisible by the set period according to the period in the period list to avoid information loss; this process includes reflection padding to adjust the length and convert the one-dimensional signal into a two-dimensional format; the convolution layer includes 5 5×5 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer expands the input signal from a single channel to 64 channels to maintain the spatial dimension of the feature map; the middle three convolution layers are used to increase the number of channels of the feature map to 128, 256 and 512 layer by layer to extract deep feature information; the last convolution layer reduces the number of output channels to 1 to integrate features.

[0054] Furthermore, in S4, the network is generated by training with D2 as the training set, which is specifically achieved through the following sub-steps:

[0055] S4.1.1: According to the pitch feature f0 in D2, determine the time index of f0 and obtain the pitch feature that matches the Mel spectrum feature length; use the template generation module to generate a multi-channel signal template according to the pitch feature that matches the Mel spectrum feature length ;

[0056] S4.1.2: Downsample the signal template of the multi-channel, and after each downsampling, the length of the signal is halved;

[0057] S4.1.3: Convolve the Mel spectrum feature, and concatenate the downsampled feature obtained in S4.1.2 with the convolved Mel spectrum feature to obtain the concatenated feature C;

[0058] S4.1.4: Upsample feature C. When upsampling in the first layer, concatenate feature C with the features of the first upsampled layer as the input for the next upsampled layer. In the subsequent upsampling stage, concatenate the features of the upsampled layer in this layer with the features of the downsampled layer in the corresponding number of layers as the input for the next upsampled layer. After the last upsampling and concatenation, use tanh as the nonlinear activation function to limit the output value to the range of (-1,1).

[0059] Furthermore, in S4, the discriminant network is used to judge the generated signal of the generating network, as follows:

[0060] The multi-channel discriminant network is used to process the audio data of multiple fiber points simultaneously, and its input original signal is:

[0061] ;

[0062] In the formula, Represents the output of the c-th channel after the generation network;

[0063] The generated audio signal The real high-quality audio signal is input into the multi-resolution discriminator for discrimination and feature extraction. First, the two multi-channel input data are converted into spectrograms, then passed through three convolutional layers and LeakyReLU activation function, and finally passed through a convolutional layer and flattened output; and repeated according to the set resolution array, repeated for different resolutions; the expression is as follows:

[0064] ;

[0065] ;

[0066] ;

[0067] Where D γ represents the γth convolutional layer of the multi-resolution discriminator, C in_3 represents the number of input channels of the γth convolutional layer of the multi-resolution discriminator, C out_3 represents the number of output channels of the γth convolutional layer of the multi-resolution discriminator, k γ_3 represents the size of the convolution kernel of the γth convolutional layer of the multi-resolution discriminator, s γ_3 represents the convolution step size of the γth convolutional layer of the multi-resolution discriminator; y mrd is the final output of the multi-resolution discriminator, according to y mrd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true.

[0068] At the same time, the generated audio signal and real high-quality audio signals are input into the multi-periodic discriminator for discrimination and feature extraction. The input data is first processed according to the periodicity of the discriminator, including padding the input data to ensure that its length conforms to the periodicity; if the input length If it is not an integer multiple of the period peroid, the required filling amount n is calculated by reflection filling pad , the expression is as follows:

[0069] ;

[0070] After padding, the input data is rearranged into a two-dimensional form, and the expression is as follows:

[0071] ;

[0072] In the formula, view is the function of adjusting the dimension, and x represents the input data after padding;

[0073] Next, the input data is processed through three convolutional layers and the LeakyReLU activation function, which is expressed as follows:

[0074] ;

[0075] ;

[0076] Where D p represents the pth convolutional layer of the multi-cycle discriminator, C in represents the number of input channels of the multi-cycle discriminator, C out represents the number of output channels of the multi-cycle discriminator, k p represents the size of the convolution kernel of the pth convolution layer of the multi-cycle discriminator, s p represents the convolution step size of the pth convolutional layer of the multi-cycle discriminator; according to y mpd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true.

[0077] This process is controlled by a set period array and is repeated m times, where m is the number of elements in the period array; the multi-period discriminator for each period is used to capture the time series features corresponding to the period, allowing the network to analyze the input signal at multiple time scales.

[0078] Furthermore, in S4, the loss function is used to optimize the generator network and the discriminant network, and it is determined whether the loss function is less than a set threshold. If so, the training is terminated and S5 is executed; otherwise, the training is repeated;

[0079] The loss function expression of the generated network is as follows:

[0080] ;

[0081] Where w is the weight of the Mel-spectrogram loss, w is greater than 1; L mel is the loss based on Mel-spectrogram, L env is the envelope-based loss;

[0082] Mel-spectrogram-based loss L mel for:

[0083] ;

[0084] ;

[0085] In the formula, The Mel-spectrogram representing high-quality audio signals Mel transform of scales, The Mel-spectrogram output by the generative network is represented by Mel transform of scales; For the Mel spectrum loss of scales, represents the smooth L1 loss function;

[0086] Envelope-based loss L env for:

[0087] ;

[0088] ;

[0089] ;

[0090] Where y is a high-quality audio signal, To generate the signal output by the network, E(y) is the envelope of y, for The envelope of y, E(-y) is the envelope of the negative signal of y, E(- )for The envelope of the negative signal; represents the maximum pooling function, represents the L1 loss function, which calculates the sum of the absolute differences between the two signal envelopes;

[0091] The loss function L of the discriminant network discriminator The loss L based on the multi-cycle discriminator mpd and multi-resolution discriminator L mrd The loss is combined with the following expression:

[0092] ;

[0093] ;

[0094] ;

[0095] In the formula, is the score of the multi-cycle discriminator for high-quality audio signals, is the score of the multi-cycle discriminator on the generated signal, is the score of the multi-resolution discriminator for high-quality audio signals, is the score of the multi-resolution discriminator on the generated signal; M is the number of samples of high-quality audio signal, y j is the jth sample of high-quality audio signal, Generate a signal for the jth.

[0096] A high-quality audio generation system for a DAS system, comprising: a data acquisition module, a preprocessing and feature extraction module, a generation network, a discriminant network, a training module, and a conversion module;

[0097] The data acquisition module is used to collect and process data from multiple optical fiber points of the DAS system to obtain low-quality audio data, and to obtain high-quality audio data;

[0098] The preprocessing and feature extraction module is used to perform amplitude standardization, normalization, and segmentation of low-quality audio data into multiple short time frames, and then obtain the pitch feature-Mel spectrum feature data pairs of each short time frame according to the pitch extraction algorithm, and collect them to obtain data set D1; and perform the same operation on high-quality audio data to obtain data set D2;

[0099] The generation network includes: a pitch interpolation module, a template generation module, a downsampling module, a Mel spectrum convolution module, an upsampling module, a LeakyReLU activation function, and a tanh activation function; the input of the pitch interpolation module is a pitch feature-Mel spectrum feature data pair, and interpolation is performed on the time index of the Mel spectrum graph based on the time index of the pitch feature f0 and the f0 feature value through a linear interpolation function to obtain a pitch feature matching the Mel spectrum feature length; the template generation module is used to generate a multi-channel signal template according to the pitch feature matching the Mel spectrum feature length; the Mel spectrum convolution module is used to convolve the Mel spectrum feature;

[0100] The downsampling module and the upsampling module follow the U-net network architecture. The downsampling module includes multiple layers of downsampling convolutional layers, and the upsampling module includes multiple layers of upsampling convolutional layers. The input of the first upsampling convolutional layer is the concatenation of the output of the downsampling module and the Mel spectrum feature after convolution, and the input of the subsequent upsampling convolutional layer is the concatenation of the output of the previous downsampling convolutional layer and the output of the upsampling convolutional layer.

[0101] The discriminant network includes a multi-period discriminator for capturing the periodic features of the audio signal, and a multi-resolution discriminator for analyzing the details of the audio signal at different frequencies; the multi-period discriminator includes: a spectrum conversion module, a convolution layer, and a LeakyReLU activation function; the spectrum conversion module is used to convert the time domain signal into a frequency domain signal by short-time Fourier transform; the convolution layer includes 5 3×3 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer is used to expand the input spectrum map from a single channel to 32 channels, maintaining the spatial dimension of the feature map; the middle three convolution layers are used to maintain the number of channels of the feature map at 32 layer by layer, so as to gradually reduce the height of the feature map and extract deeper features layer by layer; the last convolution layer reduces the number of output channels to 1 to integrate the features;

[0102] The multi-resolution discriminator includes: a period processing module, a convolution layer, and a LeakyReLU activation function; the period processing module is used to make the time length divisible by the set period according to the period in the period list to avoid information loss; this process includes reflection padding to adjust the length and convert the one-dimensional signal into a two-dimensional format; the convolution layer includes 5 5×5 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer expands the input signal from a single channel to 64 channels to maintain the spatial dimension of the feature map; the middle three convolution layers are used to increase the number of channels of the feature map to 128, 256 and 512 layer by layer to extract deep feature information; the last convolution layer reduces the number of output channels to 1 to integrate features;

[0103] The training module is used to train the generative network and the discriminative network in stages. In the first stage, D2 is used as the training set for training. In the second stage, D1 and D2 are used as the training sets for joint training. The trained high-quality audio generation model is output to the conversion module.

[0104] The input of the trained high-quality audio generation model in the conversion module is the low-quality audio data collected by the data acquisition module, preprocessed and processed by the feature extraction module, and the output is high-quality audio data.

[0105] Furthermore, the downsampling module includes three 7×7 downsampling convolutional layers, whose expansion rates are 1, 3, and 5 in order; a LeakyReLU activation function is used after each downsampling convolutional layer; the Mel spectrum convolution module includes a one-dimensional 7×7 convolutional layer with weight normalization, which is padded to 3, and the length of its input Mel spectrum feature map is consistent with the length of the output Mel spectrum feature map after convolution;

[0106] The upsampling module includes a 3×3 upsampling convolution layer with a dilation rate of 1, a 7×7 upsampling convolution layer with a dilation rate of 3, and an 11×11 upsampling convolution layer with a dilation rate of 5, and a LeakyReLU activation function is used after each upsampling convolution layer; the input of the first upsampling convolution layer is the concatenation result of the output of the downsampling module and the Mel spectrum feature after convolution; the downsampling convolution layer and the upsampling convolution layer of the same layer are skipped through a splicing layer, and the splicing layer is used to splice the two inputs in the channel dimension; the two inputs of the splicing layer are the output of the downsampling convolution layer of the corresponding number of layers and the output of the upsampling convolution layer of the corresponding number of layers, and the output of the splicing layer of this layer is used as the input of the upsampling convolution layer of the next layer.

[0107] The beneficial effects of the present invention are:

[0108] (1) The high-quality audio generation method for DAS proposed in the present invention can convert the low-quality audio output by DAS containing noise, muffled sound, and explosive sound into high-quality audio with higher sound quality. This process significantly improves the clarity and detail of the audio, making the processed audio suitable for high-demand application scenarios.

[0109] (2) The multi-channel input and multi-channel discrimination method proposed in the present invention can fully utilize spatial information by processing audio data of multiple fiber points at the same time, thereby improving the spatial resolution and consistency of the audio signal. The multi-channel discrimination network can better capture the audio features of different fiber points and enhance the accuracy and realism of audio generation.

[0110] (3) The pitch extraction method proposed in the present invention can more effectively obtain the key features of the audio signal. After these features are input into the network, high-quality audio can be generated more accurately, ensuring the realism and clarity of the audio signal.

[0111] (4) The present invention combines a multi-period discriminator with a multi-resolution discriminator to discriminate and optimize audio signals at different time scales and frequency resolutions. The multi-period discriminator focuses on capturing the periodic characteristics of the audio signal, while the multi-resolution discriminator is able to analyze the details of the audio signal at different frequencies. This multi-level discrimination method significantly enhances the quality and details of audio generation, so that the generated high-quality audio can perform well in various application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0112] Figure 1 It is a flow chart of a high-quality audio generation method for a DAS system in an embodiment of the present invention.

[0113] Figure 2 It is an architecture diagram of a generated network in an embodiment of the present invention.

[0114] Figure 3 It is a flowchart of the training of the generating network and the discriminating network in an embodiment of the present invention.

[0115] Figure 4 is a low-quality audio spectrum diagram to be processed in an embodiment of the present invention.

[0116] Figure 5 In the embodiment of the present invention, Figure 4 The corresponding output spectrogram of the trained high-quality audio generation model. DETAILED DESCRIPTION

[0117] The present invention will be described in detail below according to the accompanying drawings and preferred embodiments, and the purpose and effect of the present invention will become more clear. The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0118] The main advantage of the DAS system is that it has multiple sensors. Generally, there is one channel, that is, one sensor, for every meter of optical fiber. For the DAS system, multiple sensors can be analyzed jointly. Compared with a single channel, a DAS that integrates multiple channels can restore sound more effectively.

[0119] Based on the above architecture, this embodiment proposes a high-quality audio generation method for a DAS system, such as Figure 1 As shown, the specific steps include:

[0120] S1: Data is collected from multiple optical fiber points of the DAS system, and the optical fiber data collected from multiple points are processed to obtain low-quality audio data, which are then preprocessed and feature extracted to obtain data set D1.

[0121] Preprocessing and feature extraction specifically include the following steps:

[0122] S1.1: Preprocess the low-quality audio data. First, the audio signal is amplitude normalized to have a uniform volume level. Then, the audio signal is mean normalized. The normalized audio signal x norm The expression of (n) is as follows:

[0123]

[0124] Wherein, x(n) represents the original audio signal (i.e., the low-quality audio signal after amplitude normalization), μ represents the mean of the original audio signal, and σ represents the standard deviation of the original audio signal.

[0125] Secondly, the long normalized audio signal is divided into multiple short time frames according to the frame length N and the frame shift, so as to perform short time analysis in the subsequent steps; specifically, each short time frame is windowed using a Hamming window, the length of the sliding window is N, the sliding window starts from the starting position of the audio signal, takes the frame shift as the step length, and sequentially intercepts the audio frames until the audio signal ends. In this embodiment, the length of the sliding window N=25ms, and the frame shift is 10ms; the expression of each short time frame is as follows:

[0126]

[0127] In the formula, x n (m) represents the m-th short time frame signal (ie, sample value), and w(m) represents the window function of the m-th Hamming window.

[0128] S1.2: The pitch feature (f0) of the audio signal in each short time frame is extracted using an improved pitch extraction algorithm. The improved pitch extraction algorithm includes speech endpoint detection and high-frequency signal enhancement, which improves the ability to extract the pitch feature of high-frequency signals.

[0129] S1.2.1, speech endpoint detection: calculate the short-time energy and zero-crossing rate of the input short-time frame signal, and then determine whether the short-time frame signal is an audio signal based on the set threshold. This step combines the characteristics of the audio signal and can more accurately locate the beginning and end of the short-time frame audio signal, laying a good foundation for subsequent pitch feature extraction.

[0130] The calculation expression of the short-time energy E(m) of the mth short-time frame signal is as follows:

[0131]

[0132] In the formula, R represents the overlap ratio, represents the sample point of the m-th short time frame, and N represents the frame length of the m-th short time frame.

[0133] The threshold expression of short-time energy is:

[0134]

[0135] In the formula, ρ is a proportional coefficient less than 1 and is set artificially.

[0136] The calculation expression of the zero crossing rate Z(m) of the mth short time frame signal is as follows:

[0137]

[0138] Where sgn() represents the sign function. If the input value is positive, the output sgn() is 1; if the input value is zero or negative, the output sgn() is 0.

[0139] The threshold Z of the zero crossing rate threshold Set by humans.

[0140] Endpoint detection is performed based on the set threshold to determine whether the short time frame is an audio signal. The expression is as follows:

[0141]

[0142] Wherein, when VAD(m) is 1, it indicates that the m-th short-time frame signal is an audio signal; when it is 0, it indicates that the m-th short-time frame signal is a non-audio signal and the short-time frame signal is discarded.

[0143] S1.2.2, High-frequency signal enhancement: For the audio signal detected by speech endpoints, first use the pre-emphasis filter to emphasize the high-frequency part, and then perform autocorrelation calculation to detect the periodicity in the signal. Find the local maximum value in the autocorrelation function and use it as the optimal period to calculate the pitch feature f0. The expression is as follows:

[0144]

[0145]

[0146]

[0147] In the formula, x pe represents the pre-emphasized audio signal, α represents the pre-emphasis coefficient; R(τ) represents the autocorrelation function, τ represents the time delay, τ best Indicates the optimal period corresponding to the maximum autocorrelation function.

[0148] S1.3. Use Mel-spectrogram to represent the pitch feature f0: By performing a short-time Fourier transform (STFT) on the audio signal processed by S1.2.2 and mapping the spectrum to the Mel scale, a frequency distribution feature that is more consistent with human ear perception can be provided. The expression is as follows:

[0149]

[0150]

[0151] Where X(f,t) represents the STFT result at frequency f and time t, w[ ] represents the window function, n0 represents the starting position of the window function, represents the complex exponential term of the short-time Fourier transform; f mel Represents the Mel-spectrogram feature.

[0152] Each short time frame can obtain a set of f0 and Mel spectrum features f mel The collection of all short-time frame pitch feature-Mel spectrum feature data pairs is dataset D1, which is used as the input of the subsequent generative network to ensure that the generative network can accurately learn and characterize the key features of high-quality audio.

[0153] S2: Obtain high-quality audio, perform preprocessing and feature extraction on it (the specific method is the same as S1) to obtain data set D2.

[0154] S3: Based on the sound features extracted by D2, a high-quality audio generation model is constructed. The high-quality audio generation model includes a generation network and a discriminant network with multi-channel input.

[0155] like Figure 2 As shown, the generation network of multi-channel input includes: a pitch interpolation module, a template generation module, a downsampling module, a Mel spectrum convolution module, an upsampling module, a LeakyReLU activation function, and a tanh activation function.

[0156] The input of the pitch interpolation module is a pair of pitch feature-Mel spectrum feature data, and the output is a pitch feature that matches the length of the Mel spectrum feature. Since the pitch feature f0 is extracted at evenly distributed time points, these time points can be represented as equally spaced points on the feature length from 0 to f0; since the time length of the Mel spectrum graph is obtained through STFT, the time index of the Mel spectrum graph can also be represented as equally spaced points on the length of the Mel spectrum graph. The linear interpolation function constructed by the pitch interpolation module interpolates the time index of the Mel spectrum graph based on the time index of the pitch feature f0 and the f0 feature value to obtain a pitch feature that matches the length of the Mel spectrum feature. The expression is as follows:

[0157]

[0158] Where, t i is the time index of the original pitch feature f0, f0(t i ) is the time index t i The eigenvalue of f0 at t mel The time index of the mel-spectrogram.

[0159] The template generation module is used to generate a multi-channel signal template based on the pitch feature that matches the Mel spectrum feature length to match the subsequent multi-channel fiber data. The expression of the multi-channel signal template is as follows:

[0160]

[0161]

[0162] Where, T a (t) represents the signal template of channel a, A(t) represents the amplitude, Indicates the phase of the ath channel; Represents a set of multi-channel signal templates, c is the number of channels, a=1,2,…,c.

[0163] The downsampling module includes three 7×7 downsampling convolutional layers with different dilation rates of 1, 3, and 5, and a LeakyReLU activation function is used after each downsampling convolutional layer. The downsampling convolutional layer effectively captures features of different scales, uses the dilation rate to expand the receptive field, and applies the LeakyReLU activation function after each downsampling convolutional layer to enhance the nonlinear expression ability.

[0164] The Mel spectrum convolution module includes a one-dimensional 7×7 convolution layer with weight normalization and a padding of 3 to ensure that the length of the input and output feature maps are consistent; its input is the Mel spectrum feature and its output is the convolved Mel spectrum feature. This convolution layer is designed to effectively extract local patterns in the Mel spectrum feature and provide rich information for subsequent feature processing.

[0165] The upsampling module includes a 3×3 upsampling convolution layer with a dilation rate of 1, a 7×7 upsampling convolution layer with a dilation rate of 3, and an 11×11 upsampling convolution layer with a dilation rate of 5. The LeakyReLU activation function is used after each upsampling convolution layer to enhance the nonlinear expression ability. This design allows the model to capture multi-scale information when processing input features, integrate features of different scales, and provide rich information representation for subsequent processing. The input of the first upsampling convolution layer is the concatenation of the output of the downsampling module and the convolutional Mel spectrum features. The concatenation result is concatenated with the features after the first upsampling layer as the input of the next upsampling layer.

[0166] The downsampling convolution layer and upsampling convolution layer of the same layer are skipped through the splicing layer. The splicing layer includes a concat function to splice the two inputs in the channel dimension. The two inputs are the output of the downsampling convolution layer of the corresponding number of layers and the output of the upsampling convolution layer of the corresponding number of layers. The output of the splicing layer of this layer is used as the input of the next upsampling convolution layer.

[0167] The discriminant network includes a multi-period discriminator (MPD) and a multi-resolution discriminator (MRD).

[0168] MRD includes: spectrum conversion module, convolution layer, and LeakyReLU activation function. The spectrum conversion module uses short-time Fourier transform to convert time domain signals into frequency domain signals; the convolution layer includes 5 3×3 convolution layers, and the LeakyReLU activation function is used after each convolution layer. The initial convolution layer expands the input spectrum map from a single channel to 32 channels, maintaining the spatial dimension of the feature map. The middle three convolution layers are used to reduce the height of the feature map layer by layer, and keep the number of channels of the feature map at 32, extracting deeper features layer by layer. The last convolution layer reduces the number of output channels to 1 to further integrate the features.

[0169] MPD includes: cycle processing module, convolution layer, and LeakyReLU activation function. The cycle processing module ensures that the time length is divisible by the set cycle according to the cycle in the cycle list, thereby avoiding information loss. This process includes reflection padding to adjust the length and convert the one-dimensional signal into a two-dimensional format. The convolution layer includes 5 5×5 convolution layers, and the LeakyReLU activation function is used after each convolution layer. Among them, the initial convolution layer expands the input signal from a single channel to 64 channels, maintaining the spatial dimension of the feature map. The middle three convolution layers then increase the number of channels of the feature map to 128, 256, and 512 layer by layer to further extract deep feature information. The last convolution layer reduces the number of output channels to 1 to further integrate the features.

[0170] S4: Use D2 as the training set to train the generative network, learn the prior distribution of high-quality audio, and judge the output of the generative network to determine the error between the generated signal of the generative network and the real signal (i.e., high-quality audio). If the error exceeds the set threshold range, use the loss function to optimize the generative network and the discriminant network, and repeat the training process; otherwise, execute S5. Figure 3 As shown, it is implemented through the following sub-steps:

[0171] S4.1: Use D2 as the training set to train the generative network and learn the prior distribution of high-quality audio. The specific operations are as follows:

[0172] S4.1.1: Based on the pitch feature f0 extracted from the high-quality audio data, determine the time index of f0 and obtain the pitch feature that matches the Mel spectrum feature length. Use the template generation module to generate a multi-channel signal template based on the pitch feature that matches the Mel spectrum feature length.

[0173] S4.1.2: The multi-channel signal template is input into the downsampling module, and the multi-channel signal template is downsampled through a series of downsampling convolutional layers. After each downsampling, the length of the signal is halved to extract higher-dimensional features. The i-th downsampling convolutional layer H i The expression is as follows:

[0174]

[0175] In the formula, H i represents the i-th downsampling convolutional layer, C in_1 represents the number of input channels of the i-th downsampling convolutional layer, C out_1 represents the number of output channels of the i-th downsampling convolutional layer, k i_1 represents the size of the convolution kernel in the i-th downsampling convolution layer, s i_1 Represents the convolution stride of the i-th downsampling convolutional layer.

[0176] S4.1.3: Convolve the Mel spectrum feature through the Mel spectrum convolution module to obtain the convolved Mel spectrum feature, and concatenate the downsampled feature finally obtained in S4.1.2 with the convolved Mel spectrum feature to obtain the concatenated feature C. The concatenation expression is as follows:

[0177]

[0178] In the formula, H d is the feature after downsampling, and M is the Mel spectrum feature after convolution.

[0179] S4.1.4: Use a series of upsampling convolutional layers to gradually upsample the concatenated features C. When upsampling the first layer, concatenate the concatenated features with the features of the first upsampled layer as the input for the next upsampling layer. In the subsequent upsampling stage, concatenate the upsampled features of this layer with the downsampled features of the corresponding layer as the input for the next upsampling layer. After the last upsampling and concatenation, use tanh as the nonlinear activation function to limit the output value to the range of (-1,1).

[0180] The i-th upsampling convolutional layer U i The expression is as follows:

[0181]

[0182] When i=1, we have:

[0183]

[0184] in, Represents the concatenation of the upsampled features of this layer and the downsampled features of the corresponding layer; C in_2 represents the number of input channels of the i-th upsampling convolutional layer, C out_2 represents the number of output channels of the i-th upsampling convolutional layer, k i_2 represents the size of the convolution kernel in the i-th upsampling convolution layer, s i_2 Represents the convolution stride of the i-th upsampling convolutional layer.

[0185] The last level of restriction expression is as follows:

[0186]

[0187] In the formula, represents the output of the generative network, Indicates Upsampling layer.

[0188] S4.2: Use the multi-channel discriminant network to judge the output of the generated network. The specific operations are as follows:

[0189] The multi-channel discriminant network can process audio data from multiple fiber points simultaneously, and its input original signal is:

[0190]

[0191] In the formula, Represents the output of the c-th channel after the generation network.

[0192] The generated audio signal The real high-quality audio signal is input into MRD for discrimination and feature extraction. First, the two multi-channel input data are converted into spectrograms, then passed through a series of convolutional layers and LeakyReLU activation functions, and finally passed through a convolutional layer and flattened output. And according to the set resolution array (requirement: the values ​​in the array should contain multiple different cycles to capture the multi-scale features of the input signal, generally a small prime number such as 2, 3, 5, 7, etc.) is repeated for different resolutions.

[0193]

[0194]

[0195]

[0196] Where D γ represents the γth convolutional layer of MRD, C in_3 represents the number of input channels of the γth convolutional layer of MRD, C out_3 represents the number of output channels of the γth convolutional layer of MRD, k γ_3 represents the size of the convolution kernel of the γth convolution layer of MRD, s γ_3 represents the convolution step size of the γth convolutional layer of MRD; y mrd is the final output of MRD, according to y mrd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true.

[0197] At the same time, the generated audio signal and real high-quality audio signals are input into MPD for discrimination and feature extraction. The input data is first processed according to the periodicity of the discriminator, which includes padding the input data to ensure that its length conforms to the periodicity. Specifically, if the input length If it is not an integer multiple of the period peroid, the required filling amount n is calculated by reflection filling pad , the expression is as follows:

[0198]

[0199] After padding, the input data is rearranged into a two-dimensional form, simulating the form of two-dimensional convolution of the audio signal, and the formula is as follows:

[0200]

[0201] In the formula, view is the function of adjusting the dimension, and x represents the input data after padding.

[0202] Next, the input data is processed through a series of convolutional layers and the LeakyReLU activation function, which is formulated as follows:

[0203]

[0204]

[0205] Where D p represents the pth convolutional layer of MPD, C in is the number of input channels of MPD, C out is the number of output channels of MPD, k p is the size of the convolution kernel of the pth convolution layer of MPD, s p is the convolution step size of the pth convolutional layer of MPD; according to y mpd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true.

[0206] This process is controlled by a set period array and is repeated m times, where m is the number of elements in the period array. The multi-period discriminator for each period focuses on capturing the time series features corresponding to that period, allowing the network to analyze input signals at multiple time scales.

[0207] S4.3: Design a loss function, optimize the generator network and the discriminator network, and determine whether the loss function is less than the set threshold. If so, end the training and execute S5; otherwise, return to S4.1 and repeat the training.

[0208] For the generative network part, the loss function combines the loss based on the Mel spectrum (loss_mel) and the loss based on the envelope (loss_env), and assigns a larger weight to loss_mel. The formula is:

[0209]

[0210] Wherein, w is the weight assigned to loss_mel, and in this embodiment, w=1.2; L mel is the loss function based on the Mel-spectrogram, L env is the envelope-based loss function.

[0211] The loss function expression based on the Mel-spectrogram is as follows:

[0212]

[0213]

[0214] In the formula, The Mel-spectrogram representing the real signal (i.e., high-quality audio signal) Mel transform of scales, The Mel-spectrogram output by the generative network is represented by Mel transform of scales; For the Mel spectrum loss of scales, represents the smooth L1 loss function, the total Mel spectrum loss L mel is the average of all scale losses.

[0215] The envelope-based loss function expression is as follows:

[0216]

[0217]

[0218]

[0219] In the formula, y is the real signal, To generate the network output signal, E(y) and are y and The envelope of y, E(-y) is the envelope of the negative signal of y, E(- )for The envelope of the negative signal, represents the maximum pooling function, represents the L1 loss function, which calculates the sum of the absolute differences between the two signal envelopes.

[0220] For the discriminant network part, the loss function combines the loss based on the multi-cycle discriminator (loss_mpd) and the loss based on the multi-resolution discriminator (loss_mrd), and its formula is:

[0221]

[0222]

[0223]

[0224] In the formula, is the score of MPD for the real signal, is the score of the MPD for the generated signal, is the score of MRD for the true signal, is the score of MRD for generating signals. M is the number of real samples, y j is the jth true sample, Generate samples for the jth.

[0225] S5: Using datasets D1 and D2 as training sets, the method of S4 is used to jointly train the generative network and the discriminative network, and finally a trained high-quality audio generative model (i.e., a high-quality audio vocoder) is obtained. Unlike S4, in S4.2, the signals input into MRD and MPD are audio signals generated by the generative network, real high-quality audio signals, and real low-quality audio signals.

[0226] The reason why dataset D1 is added to dataset D2 is that the high-quality dataset D2 provides the vocoder with rich and clear features, which help the model establish a basic understanding of the audio signal, such as the pitch, rhythm, and pronunciation characteristics of speech. Low-quality audio usually contains noise, distortion, or other undesirable features. By adding dataset D1, the model can learn how to process and reduce these interferences and improve its ability to process audio in real environments. In addition, adding low-quality audio helps the model adapt to different input conditions, making it perform better in various practical applications. With the joint training of datasets D1 and D2, the model can achieve stronger robustness in environments with a variety of audio qualities.

[0227] The advantages of adding dataset D1 to dataset D2 are: improving the generalization ability of the model. During the training process, the model can learn how to perform audio synthesis on audio of different qualities, enhancing its adaptability and robustness in different application scenarios; improving the reconstruction effect of low-quality audio. Adding low-quality audio allows the model to better reconstruct damaged audio and improve its performance in the case of noise and distortion; reducing the risk of overfitting. By introducing diverse data, the training process of the model is more comprehensive, which can effectively reduce overfitting of high-quality data, thereby improving performance in real applications. Improving the naturalness of audio synthesis. Low-quality audio usually has different pronunciation and intonation characteristics. Adding these data can allow the vocoder to show more natural timbre changes when synthesizing.

[0228] S6: After preprocessing and feature extraction of the low-quality audio to be processed, the audio is input into a trained high-quality audio generation model to obtain high-quality audio.

[0229] like Figure 4The figure shows the spectrum diagram of the low-quality audio to be processed in this embodiment. After preprocessing and processing by the trained high-quality audio generation model, the spectrum diagram of the output high-quality audio is as follows: Figure 5 As shown in the figure, it can be seen that the processed audio has richer details and reduced plosive sounds and noise, etc., which shows that the method of the present invention has a good effect in converting high-quality audio.

[0230] In order to implement the high-quality audio generation method for DAS system proposed in the present invention, this embodiment proposes a high-quality audio generation system for DAS system, including: a data acquisition module, a preprocessing and feature extraction module, a generation network, a discriminant network, a training module, and a conversion module.

[0231] The data acquisition module is used to collect and process data from multiple optical fiber points of the DAS system to obtain low-quality audio data and high-quality audio data.

[0232] The preprocessing and feature extraction module is used to standardize the amplitude, normalize, and split the low-quality audio data into multiple short time frames, and then obtain the pitch feature-Mel spectrum feature data pairs of each short time frame according to the pitch extraction algorithm, and collect them to obtain data set D1; and perform the same operation on the high-quality audio data to obtain data set D2.

[0233] The architecture of the generating network and the discriminating network is described above.

[0234] The training module is used to train the generative network and the discriminative network in stages. The first stage uses D2 as the training set for training, and the second stage uses D1 and D2 as the training sets for joint training. The trained high-quality audio generation model is output to the conversion module.

[0235] The input of the high-quality audio generation model trained in the conversion module is the low-quality audio data collected by the data acquisition module, preprocessed and processed by the feature extraction module, and the output is high-quality audio data.

[0236] The present invention proposes a high-quality audio generation method for a DAS system. By constructing an efficient vocoder, high-quality audio data is first used for initial training, and then low-quality audio data is introduced for joint training, so that the vocoder can learn to convert between audio data of different qualities and generate high-quality audio signals. The audio feature extraction method is effectively improved, so that this method can not only significantly improve the overall quality of low-quality audio data collected by the DAS system, but also improve the adaptability and robustness of the system in practical applications, solve the shortcomings of traditional methods in audio processing, and promote the wide application of DAS systems in various application scenarios.

[0237] Those skilled in the art can understand that the above are only preferred examples of the invention and are not intended to limit the invention. Although the invention is described in detail with reference to the above examples, those skilled in the art can still modify the technical solutions recorded in the above examples or replace some of the technical features therein with equivalents. Any modification, equivalent replacement, etc. made within the spirit and principle of the invention shall be included in the protection scope of the invention.

Claims

1. A high-quality audio generation method for a DAS system, characterized in that: The following steps are involved: S1: Data is collected and processed at multiple optical fiber points of the DAS system to obtain low-quality audio data, which is preprocessed and feature extracted to obtain multiple short-time frame pitch feature-Mel spectrum feature data pairs, which are collected to obtain data set D1; S2: Obtain high-quality audio data, preprocess and extract features using the method in S1 to obtain data set D2; S3: Based on the sound features extracted by D2, a high-quality audio generation model is constructed, and the high-quality audio generation model includes a generation network and a discriminant network with multi-channel input; the generation network obtains a pitch feature that matches the Mel spectrum feature length according to a linear interpolation function and a pitch feature-Mel spectrum feature data pair, and after generating a multi-channel signal template based on this, downsampling and upsampling are performed, wherein the input of the first upsampling convolution layer is the concatenation result of the output of the downsampling module and the Mel spectrum feature after convolution, and the input of the subsequent upsampling convolution layer is the concatenation result of the output of the previous downsampling convolution layer and the output of the upsampling convolution layer; the discriminant network includes a multi-period discriminator for capturing the periodic features of the audio signal, and a multi-resolution discriminator for analyzing the details of the audio signal at different frequencies; S4: Use D2 as the training set to train the generative network and learn the prior distribution of high-quality audio; the discriminant network judges the generated signal of the generative network, and calculates the loss function based on the judgment result to optimize the generative network and the discriminant network; judge whether the loss function is less than the set threshold, if not, repeat the training process; Otherwise, execute S5; S5: Using datasets D1 and D2 as training sets, the method of S4 is used to jointly train the generative network and the discriminative network, and finally a trained high-quality audio generation model is obtained; S6: After preprocessing and feature extraction of the low-quality audio to be processed, the low-quality audio is input into the trained high-quality audio generation model to obtain high-quality audio; In S1, preprocessing and feature extraction are implemented through the following sub-steps: S1.1: Normalize the amplitude of the audio signal to make it have a uniform volume level; normalize the audio signal to obtain the normalized audio signal x norm (n); Use the sliding window method to convert x norm (n) is divided into multiple short time frames, where the mth short time frame signal is x n (m) indicates; S1.2: An improved pitch extraction algorithm is used to extract the pitch feature f0 of the audio signal in each short time frame; the improved pitch extraction algorithm includes speech endpoint detection and high-frequency signal enhancement, as follows: S1.2.1, speech endpoint detection: calculate the short-time energy and zero-crossing rate of the input short-time frame signal, and then determine whether the short-time frame signal is an audio signal according to the set threshold; The expression of the short-time energy E(m) of the mth short-time frame signal is as follows: Where R represents the overlap rate, ∈ represents the sample point of the m-th short time frame, and N represents the frame length of the m-th short time frame; The threshold expression of short-time energy is: E threshold =max(E(m))*ρ; In the formula, ρ is a proportional coefficient less than 1, which is set manually; The expression of the zero crossing rate Z(m) of the mth short time frame signal is as follows: Where sgn() represents the sign function. If the input value is positive, the output sgn() is 1; if the input value is zero or negative, the output sgn() is 0. The threshold Z of the zero crossing rate threshold It is set by humans; The threshold of short-time energy and the threshold of zero-crossing rate are used for endpoint detection. If VAD(m) is 1, the m-th frame short-time frame signal is determined to be an audio signal. If VAD(m) is 0, the m-th frame short-time frame signal is determined to be a non-audio signal and the short-time frame signal is discarded. The judgment expression of endpoint detection is as follows: S1.2.2, High-frequency signal enhancement: For the audio signal detected by speech endpoints, use a pre-emphasis filter to emphasize the high-frequency part, and then perform autocorrelation calculation to detect the periodicity in the signal; calculate the pitch feature f0 based on the optimal period corresponding to the maximum value of the autocorrelation function, and the expression is as follows: x pe (t)=x(t)-αx(t-1); R(τ)=∑ t x pe (t)·x pe (t-τ); In the formula, x pe represents the pre-emphasized audio signal, α represents the pre-emphasis coefficient; R(τ) represents the autocorrelation function, τ represents the time delay, τ best Indicates the optimal period corresponding to the maximum autocorrelation function; S1.

3. Use Mel spectrum to represent pitch feature f0: Perform short-time Fourier transform on the audio signal processed by S1.2.2, and map the spectrum to the Mel scale to obtain the Mel spectrum feature, which is expressed as follows: Where X(f,t) represents the short-time Fourier transform result at frequency f and time t, w[] represents the window function, n0 represents the starting position of the window function, and e -j2πfn represents the complex exponential term of the short-time Fourier transform; f mel represents the Mel spectrum feature; Each short time frame gets a set of f0 and Mel spectrum features f mel , the set of all short time frame pitch feature-Mel spectrum feature data pairs is the data set D1; The generation network includes: a pitch interpolation module, a template generation module, a downsampling module, a Mel spectrum convolution module, an upsampling module, a LeakyReLU activation function, and a tanh activation function; The input of the pitch interpolation module is the pitch feature-Mel spectrum feature data pair. Through the linear interpolation function, based on the time index of the pitch feature f0 and the f0 feature value, interpolation is performed on the time index of the Mel spectrum graph to obtain a pitch feature that matches the Mel spectrum feature length. The expression is as follows: Where, t i is the time index of the original pitch feature, f0(t i ) is the time index t i The eigenvalue of f0 at t mel is the time index of the Mel-spectrogram; The template generation module is used to generate a multi-channel signal template according to the pitch feature that matches the Mel spectrum feature length. The expression is as follows: S(t)=[T1(t),T2(t),…,T c (t)]; Where, T a (t) represents the signal template of channel a, A(t) represents the amplitude, represents the phase of the ath channel; S(t) represents the set of multi-channel signal templates, c is the number of channels, a=1,2,…,c; The downsampling module includes three 7×7 downsampling convolutional layers, whose dilation rates are 1, 3, and 5 in order; a LeakyReLU activation function is used after each downsampling convolutional layer; The Mel spectrum convolution module includes a one-dimensional 7×7 convolution layer with weight normalization and a padding of 3, and the length of the Mel spectrum feature of the input is consistent with the length of the Mel spectrum feature after the convolution of the output; The upsampling module includes a 3×3 upsampling convolution layer with a dilation rate of 1, a 7×7 upsampling convolution layer with a dilation rate of 3, and an 11×11 upsampling convolution layer with a dilation rate of 5, and a LeakyReLU activation function is used after each upsampling convolution layer; the input of the first upsampling convolution layer is the concatenation result of the output of the downsampling module and the convolutional Mel spectrum feature; The downsampling convolution layer and upsampling convolution layer of the same layer are skipped and connected through a splicing layer, and the splicing layer is used to splice the two inputs in the channel dimension; the two inputs of the splicing layer are the output of the downsampling convolution layer of the corresponding number of layers and the output of the upsampling convolution layer of the corresponding number of layers, respectively, and the output of the splicing layer of this layer is used as the input of the upsampling convolution layer of the next layer.

2. The high-quality audio generation method for DAS system according to claim 1, characterized in that: The multi-cycle discriminator includes: a spectrum conversion module, a convolution layer, and a LeakyReLU activation function; the spectrum conversion module is used to convert the time domain signal into the frequency domain signal through short-time Fourier transform; the convolution layer includes 5 3×3 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer is used to expand the input spectrum map from a single channel to 32 channels, maintaining the spatial dimension of the feature map; the middle three convolution layers are used to maintain the number of channels of the feature map at 32 layer by layer, so as to gradually reduce the height of the feature map and extract deeper features layer by layer; the last convolution layer reduces the number of output channels to 1 to integrate the features; The multi-resolution discriminator includes: a period processing module, a convolution layer, and a LeakyReLU activation function; the period processing module is used to make the time length divisible by the set period according to the period in the period list to avoid information loss; this process includes reflection padding to adjust the length and convert the one-dimensional signal into a two-dimensional format; the convolution layer includes 5 5×5 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer expands the input signal from a single channel to 64 channels to maintain the spatial dimension of the feature map; the middle three convolution layers are used to increase the number of channels of the feature map to 128, 256 and 512 layer by layer to extract deep feature information; the last convolution layer reduces the number of output channels to 1 to integrate features.

3. The high-quality audio generation method for a DAS system according to claim 1, characterized in that: In S4, the network is generated by training with D2 as the training set, which is specifically achieved through the following sub-steps: S4.1.1: According to the pitch feature f0 in D2, determine the time index of f0 and obtain the pitch feature matching the Mel spectrum feature length; use the template generation module to generate a multi-channel signal template S(t) according to the pitch feature matching the Mel spectrum feature length; S4.1.2: Downsample the signal template of the multi-channel, and after each downsampling, the length of the signal is halved; S4.1.3: Convolve the Mel spectrum feature, and concatenate the downsampled feature obtained in S4.1.2 with the convolved Mel spectrum feature to obtain the concatenated feature C; S4.1.4: Upsample feature C. When upsampling in the first layer, concatenate feature C with the features of the first upsampled layer as the input for the next upsampled layer. In the subsequent upsampling stage, concatenate the features of the upsampled layer in this layer with the features of the downsampled layer in the corresponding number of layers as the input for the next upsampled layer. After the last upsampling and concatenation, use tanh as the nonlinear activation function to limit the output value to the range of (-1,1).

4. The high-quality audio generation method for DAS system according to claim 2, characterized in that: In S4, the discriminant network is used to judge the generated signal of the generating network, as follows: The multi-channel discriminant network is used to process the audio data of multiple fiber points simultaneously, and its input original signal is: In the formula, Represents the output of the c-th channel after the generation network; The generated audio signal S gen (t) and the real high-quality audio signal are respectively input into the multi-resolution discriminator for discrimination and feature extraction. First, the two multi-channel input data are converted into spectrograms respectively, and then passed through three convolutional layers and LeakyReLU activation function, and finally passed through a convolutional layer and flattened output; and repeated according to the set resolution array, repeated execution for different resolutions; the expression is as follows: S xpec =STFT(S gen (t)); D r+1 =LeakyReLU(Conv2D(S spec ,C in_3 ,C out_3 ,k γ_3 ,s γ_3 )); y mrd =D r (S spec ); Where D γ represents the γth convolutional layer of the multi-resolution discriminator, C in_3 represents the number of input channels of the γth convolutional layer of the multi-resolution discriminator, C out_3 represents the number of output channels of the γth convolutional layer of the multi-resolution discriminator, k γ_3 represents the size of the convolution kernel of the γth convolutional layer of the multi-resolution discriminator, s γ_3 represents the convolution step size of the γth convolutional layer of the multi-resolution discriminator; y mrd is the final output of the multi-resolution discriminator, according to y mrd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true. At the same time, the generated audio signal S gen (t) and the real high-quality audio signal are respectively input into the multi-periodic discriminator for discrimination and feature extraction. The input data is first processed according to the periodicity of the discriminator, including padding the input data to ensure that its length conforms to the periodicity; if the input length l is not an integer multiple of the period peroid, the required padding amount n is calculated by reflection padding pad , the expression is as follows: n pad =period-(l mod period); After padding, the input data is rearranged into a two-dimensional form, and the expression is as follows: In the formula, view is the function of adjusting the dimension, and x represents the input data after padding; Next, the input data is processed through three convolutional layers and the LeakyReLU activation function, which is expressed as follows: D p+1 =LeakyReLU(Conv2D(x m ,C in ,C out ,j p ,s p )); y mpd =D p (x m ); Where D p represents the pth convolutional layer of the multi-cycle discriminator, C in represents the number of input channels of the multi-cycle discriminator, C out represents the number of output channels of the multi-cycle discriminator, k p represents the size of the convolution kernel of the pth convolution layer of the multi-cycle discriminator, s p represents the convolution step size of the pth convolutional layer of the multi-cycle discriminator; according to y mpd The second dimension in determines the authenticity of the signal. If the value in the second dimension is 0, the signal is false, and if it is 1, the signal is true. This process is controlled by a set period array and is repeated m times, where m is the number of elements in the period array; the multi-period discriminator for each period is used to capture the time series features corresponding to the period, allowing the network to analyze the input signal at multiple time scales.

5. The high-quality audio generation method for DAS system according to claim 1, characterized in that: In S4, the loss function is used to optimize the generator network and the discriminant network, and it is determined whether the loss function is less than a set threshold. If so, the training is terminated and S5 is executed; otherwise, the training is repeated; The loss function expression of the generated network is as follows: L generator =w*L mel +L env ; Where w is the weight of the Mel-spectrogram loss, w is greater than 1; L mel is the loss based on Mel-spectrogram, L env is the envelope-based loss; Mel-spectrogram-based loss L mel for: In the formula, The Mel transform of the ω-th scale of the Mel spectrogram representing the high-quality audio signal, represents the Mel transform of the ω-th scale of the Mel spectrogram output by the generative network; L mel,ω is the Mel spectrum loss of the ω-th scale, represents the smooth L1 loss function; Envelope-based loss L env for: E(y) = maxpool1d(y); Where y is a high-quality audio signal, To generate the signal output by the network, E(y) is the envelope of y, for The envelope of , E(-y) is the envelope of the negative signal of y, for The envelope of the negative signal; maxpoolld(·) represents the maximum pooling function, L1(·) represents the L1 loss function, and calculates the sum of the absolute differences between the two signal envelopes; The loss function L of the discriminant network discriminator The loss L based on the multi-cycle discriminator mpd and multi-resolution discriminator L mrd The loss is combined with the following expression: L discriminator =L mpd +L mrd ; Where D MPD (y) is the score of the multi-cycle discriminator for high-quality audio signals, is the score of the multi-cycle discriminator on the generated signal, D MRD (y) is the score of the multi-resolution discriminator for high-quality audio signals, is the score of the multi-resolution discriminator on the generated signal; M is the number of samples of high-quality audio signal, y j is the jth sample of high-quality audio signal, Generate a signal for the jth.

6. A high-quality audio generation system for a DAS system, used to implement the high-quality audio generation method for a DAS system according to any one of claims 1 to 5, characterized in that: include: Data acquisition module, preprocessing and feature extraction module, generation network, discriminant network, training module, conversion module; The data acquisition module is used to collect and process data from multiple optical fiber points of the DAS system to obtain low-quality audio data, and to obtain high-quality audio data; The preprocessing and feature extraction module is used to perform amplitude standardization, normalization, and segmentation of low-quality audio data into multiple short time frames, and then obtain the pitch feature-Mel spectrum feature data pairs of each short time frame according to the pitch extraction algorithm, and collect them to obtain data set D1; and perform the same operation on high-quality audio data to obtain data set D2; The generation network includes: a pitch interpolation module, a template generation module, a downsampling module, a Mel spectrum convolution module, an upsampling module, a LeakyReLU activation function, and a tanh activation function; the input of the pitch interpolation module is a pitch feature-Mel spectrum feature data pair, and interpolation is performed on the time index of the Mel spectrum graph based on the time index of the pitch feature f0 and the f0 feature value through a linear interpolation function to obtain a pitch feature matching the Mel spectrum feature length; the template generation module is used to generate a multi-channel signal template according to the pitch feature matching the Mel spectrum feature length; the Mel spectrum convolution module is used to convolve the Mel spectrum feature; The downsampling module and the upsampling module follow the U-net network architecture. The downsampling module includes multiple layers of downsampling convolutional layers, and the upsampling module includes multiple layers of upsampling convolutional layers. The input of the first upsampling convolutional layer is the concatenation of the output of the downsampling module and the Mel spectrum feature after convolution, and the input of the subsequent upsampling convolutional layer is the concatenation of the output of the previous downsampling convolutional layer and the output of the upsampling convolutional layer. The discriminant network includes a multi-period discriminator for capturing the periodic features of the audio signal, and a multi-resolution discriminator for analyzing the details of the audio signal at different frequencies; the multi-period discriminator includes: a spectrum conversion module, a convolution layer, and a LeakyReLU activation function; the spectrum conversion module is used to convert the time domain signal into a frequency domain signal by short-time Fourier transform; the convolution layer includes 5 3×3 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer is used to expand the input spectrum map from a single channel to 32 channels, maintaining the spatial dimension of the feature map; the middle three convolution layers are used to maintain the number of channels of the feature map at 32 layer by layer, so as to gradually reduce the height of the feature map and extract deeper features layer by layer; the last convolution layer reduces the number of output channels to 1 to integrate the features; The multi-resolution discriminator includes: a period processing module, a convolution layer, and a LeakyReLU activation function; the period processing module is used to make the time length divisible by the set period according to the period in the period list to avoid information loss; this process includes reflection padding to adjust the length and convert the one-dimensional signal into a two-dimensional format; the convolution layer includes 5 5×5 convolution layers, and the LeakyReLU activation function is used after each convolution layer, wherein the initial convolution layer expands the input signal from a single channel to 64 channels to maintain the spatial dimension of the feature map; the middle three convolution layers are used to increase the number of channels of the feature map to 128, 256 and 512 layer by layer to extract deep feature information; the last convolution layer reduces the number of output channels to 1 to integrate features; The training module is used to train the generative network and the discriminative network in stages. In the first stage, D2 is used as the training set for training. In the second stage, D1 and D2 are used as the training sets for joint training. The trained high-quality audio generation model is output to the conversion module. The input of the trained high-quality audio generation model in the conversion module is the low-quality audio data collected by the data acquisition module, preprocessed and processed by the feature extraction module, and the output is high-quality audio data.

Images