Tone quality enhancing system

Abstract

The invention relates to audio signal processing, in particular to a tone quality enhancement system which comprises a server, the server carries out framing windowing processing on original audio through a first audio processing module to obtain a plurality of audio clips, and a first feature extraction module is used for extracting fundamental frequency information and barker sub-band feature parameters of each audio clip; the server obtains enhancement factors of the audio clips through an enhancement factor obtaining module according to the fundamental frequency information and the Barker sub-band characteristic parameters of the audio clips, and performs enhancement processing on the audio clips according to the enhancement factors of the audio clips through an audio clip enhancement module; the server splices the enhanced audio clips through a first enhanced audio generation module to generate a first enhanced audio; according to the technical scheme provided by the invention, the defect that the audio signal is difficult to effectively process to enhance the tone quality in the prior art can be effectively overcome.

Description

Technical Field

[0001] The invention relates to audio signal processing, and in particular to a sound quality enhancement system. Background Art

[0002] Audio signal processing is an important technology with wide applications in audio recording, music production, and speech recognition. However, due to various factors, such as recording equipment, environmental noise, and interference during transmission, audio signals may be affected by noise, distortion, or other quality issues, resulting in reduced audio clarity and accuracy.

[0003] Existing audio processing methods based on signal enhancement mainly focus on processing techniques in the time domain or frequency domain, such as filtering, equalizer, noise reduction, etc. However, these methods are difficult to effectively solve complex problems in audio signal processing and cannot provide high-quality audio effects and clear sound. Summary of the invention

[0004] 1. Technical issues to be solved

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides a sound quality enhancement system, which can effectively overcome the defect of the prior art that it is difficult to effectively process audio signals to enhance the sound quality.

[0006] (II) Technical solution

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0008] A sound quality enhancement system includes a server, wherein the server performs frame-by-frame windowing processing on original audio through a first audio processing module to obtain multiple audio segments, and uses a first feature extraction module to extract fundamental frequency information and Bark subband feature parameters of each audio segment, wherein the server obtains enhancement factors of each audio segment according to the fundamental frequency information and Bark subband feature parameters of each audio segment through an enhancement factor acquisition module, and performs enhancement processing on each audio segment according to the enhancement factors of each audio segment through an audio segment enhancement module, and the server splices the enhanced audio segments through a first enhanced audio generation module to generate first enhanced audio;

[0009] The server performs wavelet decomposition on the original audio through the second audio processing module to obtain multiple frequency bands of the original audio and wavelet coefficients of each frequency band, and uses the second feature extraction module to extract the band characteristics of each frequency band. The server reconstructs the wavelet coefficients of each frequency band through the wavelet coefficient reconstruction module to obtain the reconstructed wavelet coefficients of each frequency band, and uses the initial enhanced audio generation module to perform wavelet reconstruction on the band characteristics and reconstructed wavelet coefficients of each frequency band to generate initial enhanced audio. The server extracts the audio features of the initial enhanced audio through the third feature extraction module, and uses the second enhanced audio generation module to perform audio reconstruction on the original audio and the audio features to generate second enhanced audio. The server superimposes the first enhanced audio and the second enhanced audio through the target audio generation module to generate the target audio.

[0010] Preferably, the first feature extraction module extracts the fundamental frequency information and Bark subband feature parameters of each audio clip, including:

[0011] Performing Fourier transform on each audio clip to extract fundamental frequency information of each audio clip, and at the same time extracting Bark subband characteristic parameters of each audio clip divided into a plurality of Bark subbands on the amplitude spectrum;

[0012] The Barker subband characteristic parameters include the energy and spectrum centroid of the Barker subband.

[0013] Preferably, the enhancement factor acquisition module acquires the enhancement factor of each audio segment according to the fundamental frequency information and the Bark subband characteristic parameter of each audio segment, including:

[0014] The fundamental frequency information and Bark subband characteristic parameters of each audio clip are respectively input into the pre-trained enhancement factor acquisition model to obtain the enhancement factor of each audio clip.

[0015] Preferably, the training process of the enhancement factor acquisition model includes:

[0016] S11, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0017] S12, setting the loss function and optimizer of the enhancement factor acquisition model;

[0018] S13, inputting the training set into the enhancement factor acquisition model to perform model training;

[0019] S14, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0020] S15. If the loss value is less than the preset threshold, the model training ends, and the current enhancement factor acquisition model is the trained enhancement factor acquisition model. Otherwise, the process returns to S13 and continues to use the training set for model training.

[0021] S16. Input the validation set into the trained enhancement factor to obtain the model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0022] S17, input the test set into the optimized enhancement factor acquisition model to evaluate the model performance;

[0023] Among them, the enhancement factor acquisition model is built based on the long short-term memory network LSTM.

[0024] Preferably, in S11, the historical data set is divided into a training set, a validation set and a test set according to a preset ratio, including:

[0025] S111, collecting historical audio data, performing frame division and windowing processing on the historical audio, and obtaining multiple historical audio clips;

[0026] S112, extracting fundamental frequency information and Bark subband characteristic parameters of each historical audio segment, and setting an enhancement factor label for each historical audio segment to form a historical data set;

[0027] S113. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

[0028] Preferably, the second feature extraction module extracts the frequency band features of each frequency band, including:

[0029] Perform feature extraction processing on each frequency band using a pre-trained frequency band feature extraction model to obtain frequency band features of each frequency band;

[0030] Among them, the frequency band feature extraction model is built based on the convolutional neural network CNN.

[0031] Preferably, the wavelet coefficient reconstruction module reconstructs the wavelet coefficients of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band, including:

[0032] Reconstructing the wavelet coefficients of each frequency band according to the wavelet coefficients of the adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band;

[0033] The reconstructed wavelet coefficient of the last frequency band is the wavelet coefficient of the last frequency band.

[0034] Preferably, the reconstructing the wavelet coefficients of each frequency band according to the wavelet coefficients of adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band includes:

[0035] Calculate the correlation between the wavelet coefficient of each frequency band and the wavelet coefficient of the adjacent frequency band of each frequency band as the weight corresponding to the wavelet coefficient of each frequency band;

[0036] The wavelet coefficients of each frequency band and their corresponding weights are weighted to obtain the reconstructed wavelet coefficients of each frequency band.

[0037] Preferably, the third feature extraction module extracts audio features of the initial enhanced audio, including:

[0038] Using a pre-trained audio feature extraction model to perform feature extraction processing on the initial enhanced audio to obtain audio features of the initial enhanced audio;

[0039] Among them, the audio feature extraction model is built based on the convolutional recurrent neural network ConvRNN.

[0040] Preferably, the second enhanced audio generation module performs audio reconstruction processing on the original audio and the audio features to generate the second enhanced audio, including:

[0041] After the original audio and the audio features are fused, they are input into a preset audio reconstruction layer for audio reconstruction processing to generate a second enhanced audio.

[0042] (III) Beneficial effects

[0043] Compared with the prior art, the sound quality enhancement system provided by the present invention has the following features:

[0044] Beneficial effects:

[0045] 1) The first audio processing module performs frame-by-frame windowing processing on the original audio to obtain multiple audio clips, the first feature extraction module extracts the fundamental frequency information and Barker subband feature parameters of each audio clip, the enhancement factor acquisition module obtains the enhancement factor of each audio clip according to the fundamental frequency information and Barker subband feature parameters of each audio clip, the audio clip enhancement module performs enhancement processing on each audio clip according to the enhancement factor of each audio clip, the first enhanced audio generation module splices the enhanced audio clips to generate the first enhanced audio, obtains the enhancement factor of each audio clip by extracting the fundamental frequency information and Barker subband feature parameters of each audio clip, and then enhances each audio clip according to the enhancement factor of each audio clip, and generates the first enhanced audio after splicing, so that the first enhanced audio with enhanced sound quality can be generated by using the enhancement factor;

[0046] 2) The second audio processing module performs wavelet decomposition on the original audio to obtain multiple frequency bands and wavelet coefficients of each frequency band of the original audio. The second feature extraction module extracts the band features of each frequency band. The wavelet coefficient reconstruction module reconstructs the wavelet coefficients of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band. The initial enhanced audio generation module performs wavelet reconstruction on the band features and reconstructed wavelet coefficients of each frequency band to generate initial enhanced audio. The third feature extraction module extracts the audio features of the initial enhanced audio. The second enhanced audio generation module performs audio reconstruction on the original audio and audio features to generate second enhanced audio, thereby generating second enhanced audio with enhanced sound quality through wavelet decomposition and reconstruction and audio reconstruction. At the same time, the target audio generation module superimposes the first enhanced audio and the second enhanced audio to generate target audio, so that the target audio has high-quality audio effects and can provide clear sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 It is a system schematic diagram of the present invention;

[0049] Figure 2 A schematic diagram of a process of generating a first enhanced audio using an enhancement factor in the present invention;

[0050] Figure 3 It is a flow chart of generating the second enhanced audio through wavelet decomposition and reconstruction and audio reconstruction in the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] A sound quality enhancement system, such as Figure 1 and Figure 2As shown, the server includes a server, which performs frame-by-frame windowing processing on the original audio through a first audio processing module to obtain multiple audio clips, and uses a first feature extraction module to extract fundamental frequency information and Barker subband feature parameters of each audio clip. The server obtains the enhancement factor of each audio clip according to the fundamental frequency information and Barker subband feature parameters of each audio clip through an enhancement factor acquisition module, and uses an audio clip enhancement module to enhance each audio clip according to the enhancement factor of each audio clip. The server splices the enhanced audio clips through a first enhanced audio generation module to generate a first enhanced audio.

[0053] ① The first feature extraction module extracts the fundamental frequency information and Barker subband feature parameters of each audio clip, including:

[0054] Performing Fourier transform on each audio clip to extract fundamental frequency information of each audio clip, and at the same time extracting Bark subband characteristic parameters of each audio clip divided into a plurality of Bark subbands on the amplitude spectrum;

[0055] The Barker subband characteristic parameters include the energy and spectrum centroid of the Barker subband.

[0056] ② The enhancement factor acquisition module obtains the enhancement factor of each audio clip according to the fundamental frequency information and Bark subband characteristic parameters of each audio clip, including:

[0057] The fundamental frequency information and Bark subband characteristic parameters of each audio clip are respectively input into the pre-trained enhancement factor acquisition model to obtain the enhancement factor of each audio clip.

[0058] In the technical solution of this application, the training process of the enhancement factor acquisition model includes:

[0059] S11, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio;

[0060] S12, setting the loss function and optimizer of the enhancement factor acquisition model;

[0061] S13, inputting the training set into the enhancement factor acquisition model to perform model training;

[0062] S14, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information;

[0063] S15. If the loss value is less than the preset threshold, the model training ends, and the current enhancement factor acquisition model is the trained enhancement factor acquisition model. Otherwise, the process returns to S13 and continues to use the training set for model training.

[0064] S16. Input the validation set into the trained enhancement factor to obtain the model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model;

[0065] S17, input the test set into the optimized enhancement factor acquisition model to evaluate the model performance;

[0066] Among them, the enhancement factor acquisition model is built based on the long short-term memory network LSTM.

[0067] Specifically, in S11, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including:

[0068] S111, collecting historical audio data, performing frame division and windowing processing on the historical audio, and obtaining multiple historical audio clips;

[0069] S112, extracting fundamental frequency information and Bark subband characteristic parameters of each historical audio segment, and setting an enhancement factor label for each historical audio segment to form a historical data set;

[0070] S113. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

[0071] In the above technical solution, the first audio processing module performs frame-by-frame windowing processing on the original audio to obtain multiple audio clips, the first feature extraction module extracts the fundamental frequency information and Barker subband feature parameters of each audio clip, the enhancement factor acquisition module obtains the enhancement factor of each audio clip according to the fundamental frequency information and Barker subband feature parameters of each audio clip, the audio clip enhancement module performs enhancement processing on each audio clip according to the enhancement factor of each audio clip, the first enhanced audio generation module splices the enhanced audio clips to generate the first enhanced audio, obtains the enhancement factor of each audio clip by extracting the fundamental frequency information and Barker subband feature parameters of each audio clip, and then enhances each audio clip according to the enhancement factor of each audio clip, and generates the first enhanced audio after splicing, so that the first enhanced audio with enhanced sound quality can be generated by using the enhancement factor.

[0072] like Figure 1 and Figure 3As shown, the server performs wavelet decomposition on the original audio through the second audio processing module to obtain multiple frequency bands of the original audio and wavelet coefficients of each frequency band, and uses the second feature extraction module to extract the band characteristics of each frequency band. The server reconstructs the wavelet coefficients of each frequency band through the wavelet coefficient reconstruction module to obtain the reconstructed wavelet coefficients of each frequency band, and uses the initial enhanced audio generation module to perform wavelet reconstruction on the band characteristics and reconstructed wavelet coefficients of each frequency band to generate initial enhanced audio. The server extracts the audio features of the initial enhanced audio through the third feature extraction module, and uses the second enhanced audio generation module to perform audio reconstruction on the original audio and audio features to generate second enhanced audio. The server superimposes the first enhanced audio and the second enhanced audio through the target audio generation module to generate the target audio.

[0073] ① The second feature extraction module extracts the frequency band features of each frequency band, including:

[0074] Perform feature extraction processing on each frequency band using a pre-trained frequency band feature extraction model to obtain frequency band features of each frequency band;

[0075] Among them, the frequency band feature extraction model is built based on the convolutional neural network CNN.

[0076] ② The wavelet coefficient reconstruction module reconstructs the wavelet coefficients of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band, including:

[0077] Reconstructing the wavelet coefficients of each frequency band according to the wavelet coefficients of the adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band;

[0078] The reconstructed wavelet coefficient of the last frequency band is the wavelet coefficient of the last frequency band.

[0079] Specifically, the wavelet coefficients of each frequency band are reconstructed according to the wavelet coefficients of the adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band, including:

[0080] Calculate the correlation between the wavelet coefficient of each frequency band and the wavelet coefficient of the adjacent frequency band of each frequency band as the weight corresponding to the wavelet coefficient of each frequency band;

[0081] The wavelet coefficients of each frequency band and their corresponding weights are weighted to obtain the reconstructed wavelet coefficients of each frequency band.

[0082] ③The third feature extraction module extracts the audio features of the initial enhanced audio, including:

[0083] Using a pre-trained audio feature extraction model to perform feature extraction processing on the initial enhanced audio to obtain audio features of the initial enhanced audio;

[0084] Among them, the audio feature extraction model is built based on the convolutional recurrent neural network ConvRNN.

[0085] ④ The second enhanced audio generation module performs audio reconstruction processing on the original audio and the audio features to generate a second enhanced audio, including:

[0086] After the original audio and the audio features are fused, they are input into a preset audio reconstruction layer for audio reconstruction processing to generate a second enhanced audio.

[0087] In the above technical scheme, the second audio processing module performs wavelet decomposition on the original audio to obtain multiple frequency bands and wavelet coefficients of each frequency band of the original audio, the second feature extraction module extracts the band characteristics of each frequency band, the wavelet coefficient reconstruction module reconstructs the wavelet coefficients of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band, the initial enhanced audio generation module performs wavelet reconstruction on the band characteristics and reconstructed wavelet coefficients of each frequency band to generate initial enhanced audio, the third feature extraction module extracts the audio characteristics of the initial enhanced audio, the second enhanced audio generation module performs audio reconstruction on the original audio and audio characteristics to generate second enhanced audio, so that the second enhanced audio with enhanced sound quality can be generated through wavelet decomposition and reconstruction and audio reconstruction, and at the same time, the target audio generation module superimposes the first enhanced audio and the second enhanced audio to generate the target audio, so that the target audio has high-quality audio effects and can provide clear sound.

[0088] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A sound quality enhancement system, characterized in that: The server includes a server, wherein the server performs frame-by-frame windowing processing on the original audio through a first audio processing module to obtain multiple audio segments, and uses a first feature extraction module to extract fundamental frequency information and Barker subband feature parameters of each audio segment, wherein the server obtains the enhancement factor of each audio segment according to the fundamental frequency information and Barker subband feature parameters of each audio segment through an enhancement factor acquisition module, and uses an audio segment enhancement module to enhance each audio segment according to the enhancement factor of each audio segment, and the server splices the enhanced audio segments through a first enhanced audio generation module to generate a first enhanced audio; The server performs wavelet decomposition on the original audio through the second audio processing module to obtain multiple frequency bands of the original audio and wavelet coefficients of each frequency band, and uses the second feature extraction module to extract the band characteristics of each frequency band. The server reconstructs the wavelet coefficients of each frequency band through the wavelet coefficient reconstruction module to obtain the reconstructed wavelet coefficients of each frequency band, and uses the initial enhanced audio generation module to perform wavelet reconstruction on the band characteristics and reconstructed wavelet coefficients of each frequency band to generate initial enhanced audio. The server extracts the audio features of the initial enhanced audio through the third feature extraction module, and uses the second enhanced audio generation module to perform audio reconstruction on the original audio and the audio features to generate second enhanced audio. The server superimposes the first enhanced audio and the second enhanced audio through the target audio generation module to generate the target audio.

2. The sound quality enhancement system according to claim 1, characterized in that: The first feature extraction module extracts the fundamental frequency information and Bark subband feature parameters of each audio clip, including: Performing Fourier transform on each audio clip to extract fundamental frequency information of each audio clip, and at the same time extracting Bark subband characteristic parameters of each audio clip divided into a plurality of Bark subbands on the amplitude spectrum; The Barker subband characteristic parameters include the energy and spectrum centroid of the Barker subband.

3. The sound quality enhancement system according to claim 2, characterized in that: The enhancement factor acquisition module acquires the enhancement factor of each audio segment according to the fundamental frequency information and the Bark subband characteristic parameters of each audio segment, including: The fundamental frequency information and Bark subband characteristic parameters of each audio clip are respectively input into the pre-trained enhancement factor acquisition model to obtain the enhancement factor of each audio clip.

4. The sound quality enhancement system according to claim 3, characterized in that: The training process of the enhancement factor acquisition model includes: S11, dividing the historical data set into a training set, a validation set, and a test set according to a preset ratio; S12, setting the loss function and optimizer of the enhancement factor acquisition model; S13, inputting the training set into the enhancement factor acquisition model to perform model training; S14, calculating the loss value based on the loss function, and the optimizer updates the model parameters according to the loss value and the network gradient information; S15. If the loss value is less than the preset threshold, the model training ends, and the current enhancement factor acquisition model is the trained enhancement factor acquisition model. Otherwise, the process returns to S13 and continues to use the training set for model training. S16. Input the validation set into the trained enhancement factor to obtain the model, evaluate the generalization ability of the model by observing the performance on the validation set, and tune the hyperparameters and structure of the model; S17, input the test set into the optimized enhancement factor acquisition model to evaluate the model performance; Among them, the enhancement factor acquisition model is built based on the long short-term memory network LSTM.

5. The sound quality enhancement system according to claim 4, characterized in that: In S11, the historical data set is divided into a training set, a validation set, and a test set according to a preset ratio, including: S111, collecting historical audio data, performing frame division and windowing processing on the historical audio, and obtaining multiple historical audio clips; S112, extracting fundamental frequency information and Bark subband characteristic parameters of each historical audio segment, and setting an enhancement factor label for each historical audio segment to form a historical data set; S113. Divide the historical data set into a training set, a validation set, and a test set according to a preset ratio.

6. The sound quality enhancement system according to claim 1, characterized in that: The second feature extraction module extracts the frequency band features of each frequency band, including: Perform feature extraction processing on each frequency band using a pre-trained frequency band feature extraction model to obtain frequency band features of each frequency band; Among them, the frequency band feature extraction model is built based on the convolutional neural network CNN.

7. The sound quality enhancement system according to claim 6, characterized in that: The wavelet coefficient reconstruction module reconstructs the wavelet coefficients of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band, including: Reconstructing the wavelet coefficients of each frequency band according to the wavelet coefficients of the adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band; Among them, the reconstructed wavelet coefficient of the last frequency band is the wavelet coefficient of the last frequency band.

8. The sound quality enhancement system according to claim 7, characterized in that: The reconstructing the wavelet coefficients of each frequency band according to the wavelet coefficients of the adjacent frequency bands of each frequency band to obtain the reconstructed wavelet coefficients of each frequency band includes: Calculate the correlation between the wavelet coefficient of each frequency band and the wavelet coefficient of the adjacent frequency band of each frequency band as the weight corresponding to the wavelet coefficient of each frequency band; The wavelet coefficients of each frequency band and their corresponding weights are weighted to obtain the reconstructed wavelet coefficients of each frequency band.

9. The sound quality enhancement system according to claim 7, characterized in that: The third feature extraction module extracts audio features of the initial enhanced audio, including: Using a pre-trained audio feature extraction model to perform feature extraction processing on the initial enhanced audio to obtain audio features of the initial enhanced audio; Among them, the audio feature extraction model is built based on the convolutional recurrent neural network ConvRNN.

10. The sound quality enhancement system according to claim 9, characterized in that: The second enhanced audio generation module performs audio reconstruction processing on the original audio and the audio features to generate a second enhanced audio, including: After the original audio and the audio features are fused, they are input into a preset audio reconstruction layer for audio reconstruction processing to generate a second enhanced audio.

Images