Audio signal enhancement method and system
Through the audio signal enhancement system, deep learning technology and feature extraction are used to solve complex environmental interference and personalized processing needs, intelligent enhancement of audio signals is achieved, audio quality and user experience are improved, and human resources are reduced.
Patent Information
- Application Number
- CN202411154970.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing audio signal processing technologies are difficult to effectively deal with noise, distortion and loss in complex environments, and cannot intelligently process personalized audio features, resulting in poor audio quality and user experience, requiring a large amount of human and material resources.
Design an audio signal enhancement system, including audio acquisition, quantization, verification, extraction and processing modules, establish an audio signal model through deep learning technology, extract short-time energy, zero-crossing rate and Mel frequency cepspectral characteristics, and calculate audio signals Evaluate indexes, adaptive adjustment strategies to improve audio signal quality.
It realizes intelligent and automatic processing of audio signals, reduces manual intervention, improves the processing efficiency and quality of audio signals, provides users with a clearer and more natural auditory experience, and reduces the use of human and material resources.
Smart Images

Figure CN118918918A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of audio signal processing, and in particular to an audio signal enhancement method and system. Background Art
[0002] In today's digital society, with the booming development of various application scenarios such as communications, entertainment and voice recognition, the demand for audio signal processing systems is growing. In these application scenarios, the original audio signals are often interfered by various environmental factors, including but not limited to noise, distortion and loss. These interference factors directly affect the user's perception of audio quality and also have a significant impact on application performance.
[0003] Moreover, the quality problem of audio signals is more prominent in terms of user experience and application performance. Therefore, the audio signal enhancement method still faces a series of challenges and shortcomings. The original audio signals are subject to various complex interference situations, and need to be more intelligent and flexible to deal with various complex environments. In addition, the diversity of personalized audio processing needs also puts higher requirements on the applicability of the system, including the performance of detailed audio features such as voiced and unvoiced sounds in the audio. The performance of these detailed audio features cannot be processed accordingly. Usually, the audio signal processing method is unified, and the audio signal cannot be analyzed and adjusted in a targeted manner. Manual review and processing are required, which requires a lot of manpower and material resources. Summary of the invention
[0004] 1. Technical issues to be resolved
[0005] In view of the deficiencies in the prior art, the present invention provides a method and system for enhancing an audio signal, which solves the problems mentioned in the background technology.
[0006] (II) Technical solution
[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: an audio signal enhancement system, including an audio acquisition module, an audio quantization module, a verification module, an extraction module, a processing module and an evaluation module;
[0008] The audio acquisition module obtains the original audio signal through the audio input interface, performs fixed-cycle sampling, and composes the sampled continuous audio signal into an audio signal set;
[0009] The audio quantization module processes the audio signal set, including quantization processing, mapping continuous audio amplitudes into discrete digital values, determining quantization levels, including 16 bits and 24 bits, to form a digital audio signal set;
[0010] The verification module performs preprocessing verification on the digital audio signal set, and the verification includes signal distortion verification and signal loss verification;
[0011] The extraction module performs frame processing on the verified digital audio signal set to obtain a digital audio frame data group, and performs feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrates them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set;
[0012] The processing module normalizes and fits the first data set, the second data set, and the third data set, uses deep learning technology to establish a digital audio signal model, and obtains the audio signal evaluation index Pgzs after training and analysis;
[0013] The audio signal evaluation index Pgzs is obtained by the following calculation formula:
[0014]
[0015] Wherein, Dsxs represents the short-time energy coefficient, Glxs represents the zero-crossing rate coefficient, Plxs represents the Mel-frequency cepstrum coefficient, p1, p2 and p3 represent the proportional coefficients of the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs and the Mel-frequency cepstrum coefficient Plxs respectively, and C represents the first correction constant;
[0016] The short-time energy coefficient Dsxs is obtained by calculating the first data, and matched with a preset short-time energy threshold D to obtain a short-time energy audio solution;
[0017] The zero-crossing rate coefficient Glxs is obtained by calculating the second data set, and matched with the preset zero-crossing rate threshold G to obtain a zero-crossing rate audio solution;
[0018] The Mel-frequency cepstral coefficient Plxs is obtained by calculating the third data set;
[0019] The evaluation module compares a preset audio signal threshold value Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy.
[0020] Preferably, the audio acquisition module includes an input interface unit and a sampling unit;
[0021] The input interface unit is used to communicate and connect with an external audio device, and obtain the original audio signal through the audio input interface, and the connection includes: a microphone and an audio input device;
[0022] The sampling unit samples the acquired original audio signal at a fixed period to form discrete audio samples, and combines continuous audio samples to obtain an audio signal set.
[0023] Preferably, the audio quantization module includes a quantization unit and a quantization level determination unit;
[0024] The quantization unit performs amplitude quantization processing on the audio signal set by a linear quantization method and a set number of quantization bits, including 16 bits and 24 bits, to map the continuous audio amplitude into discrete digital values, thereby forming a digital audio signal set;
[0025] The quantization level determination unit is used to set the number of quantization bits, determine the digital representation range of each sample, and select different number of bits, including 16 bits or 24 bits, according to specific needs to define the accuracy of the digital audio signal set.
[0026] Preferably, the verification module includes a distortion verification unit and a loss verification unit;
[0027] The distortion verification unit is responsible for performing distortion verification on the digital audio signal set, including using verification technology and error coding technology to detect whether there is signal distortion;
[0028] The loss verification unit is responsible for performing loss verification on the digital audio signal set, including detecting whether there is a signal loss by comparing the received digital audio signal with a preset signal sequence.
[0029] Preferably, the extraction module comprises an extraction unit and an integration unit;
[0030] The extraction unit performs frame processing on the verified digital audio signal set to form a digital audio frame data group, and then extracts short-time energy features, zero-crossing rate features and Mel frequency cepstrum features;
[0031] The short-term energy features include: audio sample amplitude value;
[0032] The zero-crossing rate characteristics include: the number of times the amplitude value passes through zero value with a positive phase and the number of times the amplitude value passes through zero value with a negative phase;
[0033] Mel frequency cepstrum features include: Mel filter bandwidth, Mel frequency measure and output energy value of filter bank;
[0034] The integration unit integrates the extracted feature information into three data sets, including short-time energy features integrated into a first data set, zero-crossing rate features integrated into a second data set, and Mel frequency cepstrum features integrated into a third data set;
[0035] The first data set includes: an amplitude value Zfz;
[0036] The second data set includes: a positive phase frequency value Zxw and a negative phase frequency value Fxw;
[0037] The third data set includes: the total energy value Scn1 of the audio signal and the average output energy value Pjn1 of the audio signal.
[0038] Preferably, the processing module comprises a modeling unit;
[0039] The modeling unit normalizes the first data set, the second data set, and the third data set, and uses deep learning technology to establish a digital audio signal model after fitting, and obtains: short-time energy coefficient Dsxs, zero-crossing rate coefficient Glxs, and Mel-frequency cepstral coefficient Plxs after training and analysis, and then associates the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs, and the Mel-frequency cepstral coefficient Plxs to obtain: audio signal evaluation index Pgzs.
[0040] Preferably, the short-time energy coefficient Dsxs is obtained by the following calculation formula:
[0041]
[0042] Wherein, N represents the total number of digital audio sub-frame data groups, i=1 represents the first sub-frame audio sample in the digital audio sub-frame data group, i represents the specific i-th sub-frame audio sample in the total number of digital audio sub-frame data groups, i(Zfz) represents the amplitude value of the first sub-frame audio sample, and Zfz represents the amplitude value. By calculating the amplitude value Zfz of the audio sample, the expression state of the intensity fluctuation in the audio signal is obtained: the short-time energy coefficient Dsxs;
[0043] And match it with the preset short-time energy threshold D to obtain the short-time energy audio solution:
[0044] The short-time energy coefficient Dsxs is less than the short-time energy threshold D, and the result of the scheme in which there is no intensity fluctuation in the short-time energy audio window is obtained;
[0045] The short-time energy coefficient Dsxs≥short-time energy threshold D, and the result of the intensity fluctuation scheme in the short-time energy audio window is obtained;
[0046] The zero-crossing rate coefficient Glx is obtained by the following calculation formula:
[0047]
[0048] Wherein, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, sign(i-1(Zxw+Fxw)) represents the positive and negative signs of the positive phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample, and sign(i(Zxw+Fxw)) represents the positive and negative phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample. The positive and negative signs of the sum of the phase order values Fxw, specifically, when the sum of sign(i(Zxw+Fxw)) is less than 0, -1 is returned, when the sum of sign(i(Zxw+Fxw)) is equal to 0, 0 is returned, and when the sum of sign(i(Zxw+Fxw)) is greater than 0, 1 is returned. By calculating the absolute value difference between the positive phase order values Zxw and the negative phase order values Fxw of adjacent framed audio samples, the characteristic performance of the voiced and clear voices in the audio signal is obtained: the zero-crossing rate coefficient Glx;
[0049] And match it with the preset zero-crossing rate threshold G to obtain the zero-crossing rate audio solution:
[0050] Zero rate coefficient Glx<zero crossing rate threshold G, obtain the result of the voiced sound solution in the audio window;
[0051] The zero rate coefficient Glx≥zero crossing rate threshold G, and the result of the unvoiced solution in the audio window is obtained.
[0052] Preferably, the Mel-frequency cepstral coefficient Plxs is obtained by the following calculation formula:
[0053]
[0054] In the formula, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, Scnl represents the total energy value of the audio signal, Pjnl represents the average output energy value of the audio signal, log(i(Scnl)|+|i(Pjnl)) represents the logarithm of the sum of the absolute values of the total energy value Scnl of the audio signal and the average output energy value Pjnl of the audio signal, and by calculating the total energy value Scnl of the audio signal, the Mel-frequency cepstrum coefficient Plxs that simulates the perception state of human hearing to audio frequency and audio intensity is obtained.
[0055] Preferably, the evaluation module includes a storage unit and a matching unit;
[0056] The storage unit is used to store a short-time energy threshold D, a short-time energy audio scheme, a zero-crossing rate threshold G, a zero-rate audio scheme, an audio signal threshold Y, an audio signal adjustment strategy scheme, a related audio enhancement adjustment method, and a related prompt notification method;
[0057] The matching unit matches the preset relevant information with the required comparison value, including comparing the preset audio signal threshold Y with the audio signal evaluation index Pgzs, to obtain the audio signal adjustment strategy:
[0058] The audio signal evaluation index Pgzs is less than the audio signal threshold Y, and the result of the audio signal non-adjustment strategy solution is obtained;
[0059] The audio signal evaluation index Pgzs ≥ the audio signal threshold Y, and the audio signal adjustment strategy solution result is obtained. The adjustment includes: noise reduction adjustment, enhanced filtering adjustment, echo adjustment, language enhancement adjustment and spectrum smoothing adjustment.
[0060] A system for enhancing an audio signal comprises the following steps:
[0061] Step 1: The audio acquisition module obtains the original audio signal through the audio input interface, and performs fixed-cycle sampling to form an audio signal set;
[0062] Step 2: Processing the audio signal set through the audio quantization module, including quantization processing, mapping the continuous audio amplitude into discrete digital values to form a digital audio signal set;
[0063] Step 3: Pre-process and verify the digital audio signal set through the verification module, including signal distortion verification and signal loss verification;
[0064] Step 4: Performing frame processing on the verified digital audio signal set through the extraction module to obtain a digital audio frame data group, and performing feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrating them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set;
[0065] Step 5: normalize and fit the first data set, the second data set, and the third data set through the processing module, and use deep learning technology to establish a digital audio signal model. After training and analysis, obtain: audio signal evaluation index Pgzs;
[0066] Step 6: The evaluation module compares the preset audio signal threshold Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy solution.
[0067] (III) Beneficial effects
[0068] The present invention provides an audio signal enhancement method and system, which have the following beneficial effects:
[0069] (1) When the system is running, the original audio signals obtained are combined through the audio acquisition module to obtain an audio signal set, and the audio quantization module is used to process the signals to form a digital audio signal set, which is then verified, including signal distortion verification and signal loss verification. The extraction module is used to perform frame processing to obtain a digital audio frame data group, and then feature extraction and integration are performed to form a first data set, a second data set, and a third data set. The processing module is used to normalize the first data set, the second data set, and the third data set, and a deep learning technology is used to establish a digital audio signal model. After training and analysis, the following is obtained: an audio signal evaluation index Pgzs. The evaluation module compares the preset audio signal threshold Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy, so that the system can intelligently and automatically process and adjust the audio signal, which reduces the user's tedious work and improves the processing efficiency and quality of the audio signal, providing the user with a clearer and more natural auditory experience.
[0070] (2) By calculating the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs and the Mel-frequency cepstral coefficient Plxs, the key features of the audio signal are covered, providing rich information for subsequent processing. The acquisition of the short-time energy audio scheme and the zero-crossing rate audio scheme achieves a comprehensive evaluation, which provides a more accurate and intelligent solution and evaluation result for audio signal enhancement, and provides a basic support for improving the clarity and accuracy of the audio signal.
[0071] (3) Through steps 1 to 6, the audio signal obtained by the audio interface is collected to form an audio signal set. After processing, a digital audio signal set is obtained, and verification is performed, including signal distortion verification and signal loss verification. Frame processing is then performed to obtain a digital audio frame data group. After extraction, a first data set and a second data set are obtained. After normalization processing is performed, a digital audio signal model is established. After training and analysis, the audio signal evaluation index Pgzs is obtained, and compared with a preset audio signal threshold Y, an audio signal adjustment strategy is obtained, so as to achieve adaptive real-time detection of audio signals, and process them to obtain the results of the audio signal enhancement method, thereby reducing the use of human and material resources as much as possible and improving work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] Figure 1 A schematic diagram of a block diagram of an audio signal enhancement system of the present invention;
[0073] Figure 2 The figure is a schematic diagram of the steps of an audio signal enhancement method of the present invention. DETAILED DESCRIPTION
[0074] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0075] In today's digital society, with the booming development of various application scenarios such as communications, entertainment and voice recognition, the demand for audio signal processing systems is growing. In these application scenarios, the original audio signals are often interfered by various environmental factors, including but not limited to noise, distortion and loss. These interference factors directly affect the user's perception of audio quality and also have a significant impact on application performance.
[0076] Moreover, the quality problem of audio signals is more prominent in terms of user experience and application performance. Therefore, the audio signal enhancement method still faces a series of challenges and shortcomings. The original audio signals are subject to various complex interference situations, and need to be more intelligent and flexible to deal with various complex environments. In addition, the diversity of personalized audio processing needs also puts higher requirements on the applicability of the system, including the performance of detailed audio features such as voiced and unvoiced sounds in the audio. The performance of these detailed audio features cannot be processed accordingly. Usually, the audio signal processing method is unified, and the audio signal cannot be analyzed and adjusted in a targeted manner. Manual review and processing are required, which requires a lot of manpower and material resources.
[0077] Example 1
[0078] The present invention provides an audio signal enhancement system, see Figure 1 ,
[0079] It includes an audio acquisition module, an audio quantization module, a verification module, an extraction module, a processing module and an evaluation module;
[0080] The audio acquisition module obtains the original audio signal through the audio input interface, performs fixed-cycle sampling, and composes the sampled continuous audio signal into an audio signal set;
[0081] The audio quantization module processes the audio signal set, including quantization processing, mapping continuous audio amplitudes into discrete digital values, determining quantization levels, including 16 bits and 24 bits, to form a digital audio signal set;
[0082] The verification module performs preprocessing verification on the digital audio signal set, and the verification includes signal distortion verification and signal loss verification;
[0083] The extraction module performs frame processing on the verified digital audio signal set to obtain a digital audio frame data group, and performs feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrates them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set;
[0084] The processing module normalizes and fits the first data set, the second data set, and the third data set, uses deep learning technology to establish a digital audio signal model, and obtains the audio signal evaluation index Pgzs after training and analysis;
[0085] The audio signal evaluation index Pgzs is obtained by the following calculation formula:
[0086]
[0087] Wherein, Dsxs represents the short-time energy coefficient, Glxs represents the zero-crossing rate coefficient, Plxs represents the Mel-frequency cepstrum coefficient, p1, p2 and p3 represent the proportional coefficients of the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs and the Mel-frequency cepstrum coefficient Plxs respectively, and C represents the first correction constant;
[0088] Among them, 0.21≤p1≤0.35, 0.22≤p2≤0.36, 0.18≤p3≤0.29, and p1+p2+p3≤1.0;
[0089] The short-time energy coefficient Dsxs is obtained by calculating the first data, and matched with a preset short-time energy threshold D to obtain a short-time energy audio solution;
[0090] The zero-crossing rate coefficient Glxs is obtained by calculating the second data set, and matched with the preset zero-crossing rate threshold G to obtain a zero-crossing rate audio solution;
[0091] The Mel-frequency cepstral coefficient Plxs is obtained by calculating the third data set;
[0092] The evaluation module compares a preset audio signal threshold value Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy.
[0093] In this embodiment, the acquired original audio signals are combined through the audio acquisition module to obtain an audio signal set, and processed through the audio quantization module to form a digital audio signal set, and then verification is performed, including signal distortion verification and signal loss verification, and frame processing is performed through the extraction module to obtain a digital audio frame data group, and then feature extraction and integration are performed to form a first data set, a second data set and a third data set, and the first data set, the second data set and the third data set are normalized and fitted through the processing module, and a digital audio signal model is established using deep learning technology. After training and analysis, the following is obtained: an audio signal evaluation index Pgzs, and a preset audio signal threshold Y is compared with the audio signal evaluation index Pgzs through the evaluation module to obtain an audio signal adjustment strategy solution, so that the system can intelligently and automatically process and adjust the audio signal, reducing the user's tedious work, while improving the processing efficiency and quality of the audio signal, and providing the user with a clearer and more natural auditory experience.
[0094] Example 2
[0095] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the audio acquisition module includes an input interface unit and a sampling unit;
[0096] The input interface unit is used to communicate and connect with an external audio device, and obtain the original audio signal through the audio input interface, and the connection includes: a microphone and an audio input device;
[0097] The sampling unit samples the acquired original audio signal at a fixed period to form discrete audio samples, and combines continuous audio samples to obtain an audio signal set.
[0098] The audio quantization module includes a quantization unit and a quantization level determination unit;
[0099] The quantization unit performs amplitude quantization processing on the audio signal set by a linear quantization method and a set number of quantization bits, including 16 bits and 24 bits, to map the continuous audio amplitude into discrete digital values, thereby forming a digital audio signal set;
[0100] The quantization level determination unit is used to set the number of quantization bits, determine the digital representation range of each sample, and select different number of bits, including 16 bits or 24 bits, according to specific needs to define the accuracy of the digital audio signal set.
[0101] The verification module includes a distortion verification unit and a loss verification unit;
[0102] The distortion verification unit is responsible for performing distortion verification on the digital audio signal set, including using verification technology and error coding technology to detect whether there is signal distortion;
[0103] The loss verification unit is responsible for performing loss verification on the digital audio signal set, including detecting whether there is a signal loss by comparing the received digital audio signal with a preset signal sequence.
[0104] Example 3
[0105] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the extraction module includes an extraction unit and an integration unit;
[0106] The extraction unit performs frame processing on the verified digital audio signal set to form a digital audio frame data group, and then extracts short-time energy features, zero-crossing rate features and Mel frequency cepstrum features;
[0107] The short-term energy features include: audio sample amplitude value;
[0108] The zero-crossing rate characteristics include: the number of times the amplitude value passes through zero value with a positive phase and the number of times the amplitude value passes through zero value with a negative phase;
[0109] Mel frequency cepstrum features include: Mel filter bandwidth, Mel frequency measure and output energy value of filter bank;
[0110] The integration unit integrates the extracted feature information into three data sets, including short-time energy features integrated into a first data set, zero-crossing rate features integrated into a second data set, and Mel frequency cepstrum features integrated into a third data set;
[0111] The first data set includes: an amplitude value Zfz;
[0112] The second data set includes: a positive phase frequency value Zxw and a negative phase frequency value Fxw;
[0113] The third data set includes: the total energy value Scn1 of the audio signal and the average output energy value Pjn1 of the audio signal.
[0114] The processing module includes a modeling unit;
[0115] The modeling unit normalizes the first data set, the second data set, and the third data set, and uses deep learning technology to establish a digital audio signal model after fitting, and obtains: short-time energy coefficient Dsxs, zero-crossing rate coefficient Glxs, and Mel-frequency cepstral coefficient Plxs after training and analysis, and then associates the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs, and the Mel-frequency cepstral coefficient Plxs to obtain: audio signal evaluation index Pgzs.
[0116] The short-time energy coefficient Dsxs is obtained by the following calculation formula:
[0117]
[0118] Wherein, N represents the total number of digital audio sub-frame data groups, i=1 represents the first sub-frame audio sample in the digital audio sub-frame data group, i represents the specific i-th sub-frame audio sample in the total number of digital audio sub-frame data groups, i(Zfz) represents the amplitude value of the first sub-frame audio sample, and Zfz represents the amplitude value. By calculating the amplitude value Zfz of the audio sample, the expression state of the intensity fluctuation in the audio signal is obtained: the short-time energy coefficient Dsxs;
[0119] And match it with the preset short-time energy threshold D to obtain the short-time energy audio solution:
[0120] The short-time energy coefficient Dsxs is less than the short-time energy threshold D, and the result of the scheme in which there is no intensity fluctuation in the short-time energy audio window is obtained;
[0121] The short-time energy coefficient Dsxs≥short-time energy threshold D, and the result of the intensity fluctuation scheme in the short-time energy audio window is obtained;
[0122] Short-time energy audio window: After the audio signal is framed, continuous audio signal sample frames are integrated to form a short-time energy audio window;
[0123] The zero-crossing rate coefficient Glx is obtained by the following calculation formula:
[0124]
[0125] Wherein, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, sign(i-1(Zxw+Fxw)) represents the positive and negative signs of the positive phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample, and sign(i(Zxw+Fxw)) represents the positive and negative phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample. The positive and negative signs of the sum of the phase order values Fxw, specifically, when the sum of sign(i(Zxw+Fxw)) is less than 0, -1 is returned, when the sum of sign(i(Zxw+Fxw)) is equal to 0, 0 is returned, and when the sum of sign(i(Zxw+Fxw)) is greater than 0, 1 is returned. By calculating the absolute value difference between the positive phase order values Zxw and the negative phase order values Fxw of adjacent framed audio samples, the characteristic performance of the voiced and clear voices in the audio signal is obtained: the zero-crossing rate coefficient Glx;
[0126] And match it with the preset zero-crossing rate threshold G to obtain the zero-crossing rate audio solution:
[0127] Zero rate coefficient Glx<zero crossing rate threshold G, obtain the result of the voiced sound solution in the audio window;
[0128] The zero rate coefficient Glx≥zero crossing rate threshold G, and the result of the unvoiced solution in the audio window is obtained.
[0129] The Mel frequency cepstral coefficient Plxs is obtained by the following calculation formula:
[0130]
[0131] In the formula, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, Scnl represents the total energy value of the audio signal, Pjnl represents the average output energy value of the audio signal, log(i(Scnl)|+|i(Pjnl)) represents the logarithm of the sum of the absolute values of the total energy value Scnl of the audio signal and the average output energy value Pjnl of the audio signal, and by calculating the total energy value Scnl of the audio signal, the Mel-frequency cepstrum coefficient Plxs that simulates the perception state of human hearing to audio frequency and audio intensity is obtained.
[0132] In this embodiment, the key features of the audio signal are covered by calculating the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs and the Mel-frequency cepstral coefficient Plxs, providing rich information for subsequent processing. The acquisition of the short-time energy audio solution and the zero-crossing rate audio solution achieves a comprehensive evaluation, which provides a more accurate and intelligent solution and evaluation result for audio signal enhancement, and provides basic support for improving the clarity and accuracy of the audio signal.
[0133] Example 4
[0134] This embodiment is explained in Example 1, please refer to Figure 1 ,Specifically: the evaluation module includes a storage unit and a matching unit;
[0135] The storage unit is used to store a short-time energy threshold D, a short-time energy audio scheme, a zero-crossing rate threshold G, a zero-rate audio scheme, an audio signal threshold Y, an audio signal adjustment strategy scheme, a related audio enhancement adjustment method, and a related prompt notification method;
[0136] The matching unit matches the preset relevant information with the required comparison value, including comparing the preset audio signal threshold Y with the audio signal evaluation index Pgzs, to obtain the audio signal adjustment strategy:
[0137] The audio signal evaluation index Pgzs is less than the audio signal threshold Y, and the result of the audio signal non-adjustment strategy solution is obtained;
[0138] The audio signal evaluation index Pgzs≥the audio signal threshold Y, and the audio signal adjustment strategy solution result is obtained, the adjustment includes: noise reduction adjustment, enhanced filtering adjustment, echo adjustment, language enhancement adjustment and spectrum smoothing adjustment, and the short-time energy audio solution result and the zero-crossing rate audio solution result are integrated to perform corresponding feature extraction, and obtain relevant features of the audio signal, including the explosive or intensity-changing part in the audio signal and the distinction between voiced and unvoiced sounds in the audio signal, and then execute the corresponding enhancement algorithm, including: dynamic range compression adjustment, filtering algorithm adjustment and spectrum smoothing algorithm adjustment.
[0139] Example 5
[0140] A method for enhancing an audio signal, please refer to Figure 2 , specifically: including the following steps:
[0141] Step 1: The audio acquisition module obtains the original audio signal through the audio input interface, and performs fixed-cycle sampling to form an audio signal set;
[0142] Step 2: Processing the audio signal set through the audio quantization module, including quantization processing, mapping the continuous audio amplitude into discrete digital values to form a digital audio signal set;
[0143] Step 3: Pre-process and verify the digital audio signal set through the verification module, including signal distortion verification and signal loss verification;
[0144] Step 4: Performing frame processing on the verified digital audio signal set through the extraction module to obtain a digital audio frame data group, and performing feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrating them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set;
[0145] Step 5: normalize and fit the first data set, the second data set, and the third data set through the processing module, and use deep learning technology to establish a digital audio signal model. After training and analysis, obtain: audio signal evaluation index Pgzs;
[0146] Step 6: The evaluation module compares the preset audio signal threshold Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy solution.
[0147] In this embodiment, through steps one to six, the audio signal obtained by the audio interface is collected to form an audio signal set. After processing, a digital audio signal set is obtained, and verification is performed, including signal distortion verification and signal loss verification. Frame processing is then performed to obtain a digital audio frame data group. After extraction, a first data set and a second data set are obtained. After normalization processing is performed, a digital audio signal model is established. After training and analysis, the audio signal evaluation index Pgzs is obtained, and compared with a preset audio signal threshold Y, an audio signal adjustment strategy is obtained, so as to achieve adaptive real-time detection of audio signals, and process them to obtain the results of the audio signal enhancement method, thereby reducing the use of human and material resources as much as possible and improving work efficiency.
[0148] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An audio signal enhancement system, characterized in that: It includes an audio acquisition module, an audio quantization module, a verification module, an extraction module, a processing module and an evaluation module; The audio acquisition module obtains the original audio signal through the audio input interface, performs fixed-cycle sampling, and composes the sampled continuous audio signal into an audio signal set; The audio quantization module processes the audio signal set, including quantization processing, mapping continuous audio amplitudes into discrete digital values, determining quantization levels, including 16 bits and 24 bits, to form a digital audio signal set; The verification module performs preprocessing verification on the digital audio signal set, and the verification includes signal distortion verification and signal loss verification; The extraction module performs frame processing on the verified digital audio signal set to obtain a digital audio frame data group, and performs feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrates them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set; The processing module normalizes and fits the first data set, the second data set, and the third data set, uses deep learning technology to establish a digital audio signal model, and obtains the audio signal evaluation index Pgzs after training and analysis; The audio signal evaluation index Pgzs is obtained by the following calculation formula: Wherein, Dsxs represents the short-time energy coefficient, Glxs represents the zero-crossing rate coefficient, Plxs represents the Mel-frequency cepstrum coefficient, p1, p2 and p3 represent the proportional coefficients of the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs and the Mel-frequency cepstrum coefficient Plxs respectively, and C represents the first correction constant; The short-time energy coefficient Dsxs is obtained by calculating the first data, and matched with a preset short-time energy threshold D to obtain a short-time energy audio solution; The zero-crossing rate coefficient Glxs is obtained by calculating the second data set, and matched with the preset zero-crossing rate threshold G to obtain a zero-crossing rate audio solution; The Mel-frequency cepstral coefficient Plxs is obtained by calculating the third data set; The evaluation module compares a preset audio signal threshold value Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy.
2. The audio signal enhancement system according to claim 1, characterized in that: The audio acquisition module includes an input interface unit and a sampling unit; The input interface unit is used to communicate and connect with an external audio device, and obtain the original audio signal through the audio input interface, and the connection includes: a microphone and an audio input device; The sampling unit samples the acquired original audio signal at a fixed period to form discrete audio samples, and combines continuous audio samples to obtain an audio signal set.
3. The audio signal enhancement system according to claim 1, characterized in that: The audio quantization module includes a quantization unit and a quantization level determination unit; The quantization unit performs amplitude quantization processing on the audio signal set by a linear quantization method and a set number of quantization bits, including 16 bits and 24 bits, to map the continuous audio amplitude into discrete digital values, thereby forming a digital audio signal set; The quantization level determination unit is used to set the number of quantization bits, determine the digital representation range of each sample, and select different number of bits, including 16 bits or 24 bits, according to specific needs to define the accuracy of the digital audio signal set.
4. The audio signal enhancement system according to claim 1, characterized in that: The verification module includes a distortion verification unit and a loss verification unit; The distortion verification unit is responsible for performing distortion verification on the digital audio signal set, including using verification technology and error coding technology to detect whether there is signal distortion; The loss verification unit is responsible for performing loss verification on the digital audio signal set, including detecting whether there is a signal loss by comparing the received digital audio signal with a preset signal sequence.
5. The audio signal enhancement system according to claim 3, characterized in that: The extraction module includes an extraction unit and an integration unit; The extraction unit performs frame processing on the verified digital audio signal set to form a digital audio frame data group, and then extracts short-time energy features, zero-crossing rate features and Mel frequency cepstrum features; The short-term energy features include: audio sample amplitude value; The zero-crossing rate characteristics include: the number of times the amplitude value passes through zero value with a positive phase and the number of times the amplitude value passes through zero value with a negative phase; Mel frequency cepstrum features include: Mel filter bandwidth, Mel frequency measure and output energy value of filter bank; The integration unit integrates the extracted feature information into three data sets, including short-time energy features integrated into a first data set, zero-crossing rate features integrated into a second data set, and Mel frequency cepstrum features integrated into a third data set; The first data set includes: an amplitude value Zfz; The second data set includes: a positive phase frequency value Zxw and a negative phase frequency value Fxw; The third data set includes: the total energy value Scn1 of the audio signal and the average output energy value Pjn1 of the audio signal.
6. The audio signal enhancement system according to claim 5, characterized in that: The processing module includes a modeling unit; The modeling unit normalizes the first data set, the second data set, and the third data set, and uses deep learning technology to establish a digital audio signal model after fitting, and obtains: short-time energy coefficient Dsxs, zero-crossing rate coefficient Glxs, and Mel-frequency cepstral coefficient Plxs after training and analysis, and then associates the short-time energy coefficient Dsxs, the zero-crossing rate coefficient Glxs, and the Mel-frequency cepstral coefficient Plxs to obtain: audio signal evaluation index Pgzs.
7. The audio signal enhancement system according to claim 5, characterized in that: The short-time energy coefficient Dsxs is obtained by the following calculation formula: Wherein, N represents the total number of digital audio sub-frame data groups, i=1 represents the first sub-frame audio sample in the digital audio sub-frame data group, i represents the specific i-th sub-frame audio sample in the total number of digital audio sub-frame data groups, i(Zfz) represents the amplitude value of the first sub-frame audio sample, and Zfz represents the amplitude value. By calculating the amplitude value Zfz of the audio sample, the expression state of the intensity fluctuation in the audio signal is obtained: the short-time energy coefficient Dsxs; And match it with the preset short-time energy threshold D to obtain the short-time energy audio solution: The short-time energy coefficient Dsxs is less than the short-time energy threshold D, and the result of the scheme in which there is no intensity fluctuation in the short-time energy audio window is obtained; The short-time energy coefficient Dsxs≥short-time energy threshold D, and the result of the intensity fluctuation scheme in the short-time energy audio window is obtained; The zero-crossing rate coefficient Glx is obtained by the following calculation formula: Wherein, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, sign(i-1(Zxw+Fxw)) represents the positive and negative signs of the positive phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample, and sign(i(Zxw+Fxw)) represents the positive and negative phase frequency value Zxw and the negative phase frequency value Fxw of the i-1-th frame audio sample. The positive and negative signs of the sum of the phase order values Fxw, specifically, when the sum of sign(i(Zxw+Fxw)) is less than 0, -1 is returned, when the sum of sign(i(Zxw+Fxw)) is equal to 0, 0 is returned, and when the sum of sign(i(Zxw+Fxw)) is greater than 0, 1 is returned. By calculating the absolute value difference between the positive phase order values Zxw and the negative phase order values Fxw of adjacent framed audio samples, the characteristic performance of the voiced and clear voices in the audio signal is obtained: the zero-crossing rate coefficient Glx; And match it with the preset zero-crossing rate threshold G to obtain the zero-crossing rate audio solution: Zero rate coefficient Glx<zero crossing rate threshold G, obtain the result of the voiced sound solution in the audio window; The zero rate coefficient Glx≥zero crossing rate threshold G, and the result of the unvoiced solution in the audio window is obtained.
8. The audio signal enhancement system according to claim 1, characterized in that: The Mel frequency cepstral coefficient Plxs is obtained by the following calculation formula: In the formula, N represents the total number of digital audio frame data groups, i=1 represents the first frame audio sample in the digital audio frame data group, i represents the specific i-th frame audio sample in the total number of digital audio frame data groups, Scnl represents the total energy value of the audio signal, Pjnl represents the average output energy value of the audio signal, log(i(Scnl)|+|i(Pjnl)) represents the logarithm of the sum of the absolute values of the total energy value Scnl of the audio signal and the average output energy value Pjnl of the audio signal, and by calculating the total energy value Scnl of the audio signal, the Mel-frequency cepstrum coefficient Plxs that simulates the perception state of human hearing to audio frequency and audio intensity is obtained.
9. The audio signal enhancement system according to claim 1, characterized in that: The evaluation module includes a storage unit and a matching unit; The storage unit is used to store a short-time energy threshold D, a short-time energy audio scheme, a zero-crossing rate threshold G, a zero-rate audio scheme, an audio signal threshold Y, an audio signal adjustment strategy scheme, a related audio enhancement adjustment method, and a related prompt notification method; The matching unit matches the preset relevant information with the required comparison value, including comparing the preset audio signal threshold Y with the audio signal evaluation index Pgzs, to obtain the audio signal adjustment strategy: The audio signal evaluation index Pgzs is less than the audio signal threshold Y, and the result of the audio signal non-adjustment strategy solution is obtained; The audio signal evaluation index Pgzs ≥ the audio signal threshold Y, and the audio signal adjustment strategy solution result is obtained. The adjustment includes: noise reduction adjustment, enhanced filtering adjustment, echo adjustment, language enhancement adjustment and spectrum smoothing adjustment.
10. A method for enhancing an audio signal, comprising the audio signal enhancement system according to any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: The audio acquisition module obtains the original audio signal through the audio input interface, and performs fixed-cycle sampling to form an audio signal set; Step 2: Processing the audio signal set through the audio quantization module, including quantization processing, mapping the continuous audio amplitude into discrete digital values to form a digital audio signal set; Step 3: Pre-process and verify the digital audio signal set through the verification module, including signal distortion verification and signal loss verification; Step 4: Performing frame processing on the verified digital audio signal set through the extraction module to obtain a digital audio frame data group, and performing feature extraction, including short-time energy features, zero-crossing rate features and Mel-frequency cepstrum features, and then integrating them, including: integrating short-time energy feature information into a first data set, integrating zero-crossing rate feature information into a second data set, and integrating Mel-frequency cepstrum feature information into a third data set; Step 5: normalize and fit the first data set, the second data set, and the third data set through the processing module, and use deep learning technology to establish a digital audio signal model. After training and analysis, obtain: audio signal evaluation index Pgzs; Step 6: The evaluation module compares the preset audio signal threshold Y with the audio signal evaluation index Pgzs to obtain an audio signal adjustment strategy solution.
Citation Information
Patent Citations
Audio detecting method and device, electronic equipment and storage medium
CN109994127A
Multi-functional operation nursing wagon
CN111110490A
Audio playing method, earphone and computer readable storage medium
CN111885458A
Environment sound unvarnished transmission method and device applied to VR, equipment and storage medium
CN115842980A
Method for generating multi-mode vortex electromagnetic waves through single antenna
CN117744380A
