Recording de-noising method, system and equipment based on sound characteristics and medium

Abstract

The invention discloses a sound feature-based recording denoising method and system, electronic equipment and a storage medium, and relates to the field of recording processing. In the method, a to-be-denoised record is divided into multiple frames, the short-time energy and the short-time zero-crossing rate of each frame are calculated, and each frame is an audio signal with a preset time length; dividing the recording to be denoised into a voice segment and a voice-free segment according to the short-time energy and the short-time zero-crossing rate; determining frequency spectrum characteristics of the voice-free segment, and determining noise characteristics according to the frequency spectrum characteristics through a preset statistical model; and de-noising the voice segment according to the noise characteristics. By implementing the technical scheme provided by the invention, the effects of effectively reducing noise interference and improving the recording quality are achieved.

Description

Technical Field

[0001] The present application relates to the technical field of recording processing, and in particular to a recording denoising method, system, electronic device and storage medium based on sound features. Background Art

[0002] With the widespread application and popularization of recording technology, various noises will inevitably be mixed into the recording, such as background noise, current noise, etc. These noises seriously affect the quality and audibility of the recording.

[0003] Existing recording noise removal methods usually include steps such as spectrum analysis, noise estimation and noise suppression. Although these methods can reduce noise to a certain extent, they often have problems such as incomplete noise suppression and large voice distortion. Especially in complex and changeable noise environments, the noise removal effect of traditional methods is even more difficult to guarantee.

[0004] Therefore, how to more effectively estimate and suppress noise based on sound characteristics has become an important challenge facing current recording noise removal technology. Summary of the invention

[0005] The present application provides a recording denoising method, system, electronic device and storage medium based on sound features, which effectively reduce noise interference and improve recording quality.

[0006] In a first aspect of the present application, a recording denoising method based on sound features is provided, which is applied to a recording denoising platform. The recording denoising method comprises:

[0007] The recording to be denoised is divided into multiple frames, and the short-time energy and short-time zero-crossing rate of each frame are calculated, where each frame is an audio signal of a preset time length;

[0008] Dividing the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate;

[0009] Determine the frequency spectrum characteristics of the speech-free segment, and determine the noise characteristics according to the frequency spectrum characteristics through a preset statistical model;

[0010] De-noising the speech segment according to the noise characteristics.

[0011] By adopting the above technical solution, the recording to be denoised is divided into multiple frames, and the speech segments and non-speech segments are identified based on short-time energy and short-time zero-crossing rate, so that the noise area can be accurately located. The noise characteristics are determined according to the spectral characteristics of the non-speech segment, and the speech segments are subjected to targeted denoising, thereby significantly improving the clarity of the speech and making the speech content easier to understand and identify. The noise interference in the recording can be effectively removed, whether it is environmental noise, background noise or other types of noise. By reducing or eliminating these noises, the overall quality of the recording is significantly improved, making it purer, natural and real. The denoising process based on sound characteristics can adapt to different types of noise and recording environments. Whether it is static noise or dynamic noise, whether it is indoor environment or outdoor environment, this method can effectively perform denoising. This makes the recording denoising system more robust and adaptable. By dividing the recording to be denoised into multiple frames and processing them separately, the method reduces the complexity and computational complexity of the system. The processing of each frame is independent and can be carried out in parallel, thereby improving the processing speed and efficiency of the system. At the same time, the method does not require complex hardware equipment support, but only needs to be implemented at the software level. Since each frame is processed independently and quickly, this method can perform noise removal in real time or near real time. This makes it very suitable for application scenarios that require real-time processing, such as teleconferencing, online live broadcasting, etc.

[0012] Optionally, dividing the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate includes:

[0013] When the short-time energies of a first preset number of consecutive frames are all greater than a first preset short-time energy threshold, taking the start frame of the first preset number of consecutive frames as the starting point of the speech segment;

[0014] When the short-time energies of a second preset number of consecutive frames are all less than the first preset short-time energy threshold, taking the start frame of the second preset number of consecutive frames as the end point of the speech segment;

[0015] Determine a speech segment according to the starting point and the end point, classify frames in the speech segment whose short-time energy is less than a second preset short-time energy threshold into a speech-free segment, and classify frames in the speech segment whose short-time zero-crossing rate is greater than a preset short-time zero-crossing rate threshold into a speech-free segment;

[0016] Aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm, and adjusting the first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold according to the alignment result;

[0017] The recording to be denoised is re-divided according to the adjusted first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold.

[0018] By adopting the above technical solution, setting the short-time energy threshold of the first preset number of frames and the second preset number of frames in a row, the starting point and the end point of the speech segment can be more accurately identified. The embodiment of the present application can reduce the misjudgment caused by the energy fluctuation of a single frame and improve the accuracy of speech segment detection. After determining the starting point and the end point of the speech segment, some frames are further re-divided into non-speech segments according to the short-time energy and the short-time zero-crossing rate. This can more accurately identify non-speech parts with low energy or high zero-crossing rate, such as background noise or silent segments, thereby optimizing the division of speech segments and non-speech segments. By introducing a preset dynamic time warping algorithm (DTW), the divided recording is aligned with multiple known speech templates, and the thresholds of short-time energy and short-time zero-crossing rate are adjusted according to the alignment results, and the thresholds can be dynamically adjusted according to the actual speech features. This adaptive capability enables the system to better adapt to different types of speech and noise environments and improve the denoising effect. Through the refined and dynamically adjusted processing steps, the recording denoising platform can reduce the misjudgment rate and reduce the situation where a speech segment is misjudged as a non-speech segment or a non-speech segment is misjudged as a speech segment. This will further improve the accuracy of speech segment detection and noise removal effect.

[0019] Optionally, aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm includes:

[0020] Extracting a sequence with the same features from the recording to be denoised and the known speech template, wherein the features include short-time energy and short-time zero-crossing rate;

[0021] Constructing a two-dimensional DTW matrix according to a first length of the sequence of the to-be-denoised recordings and a second length of the sequence of the known speech templates;

[0022] A matching path is determined according to the target element of the DTW matrix, and the recording to be denoised is aligned with the known speech template according to the matching path.

[0023] By adopting the above technical solution, the same feature sequence (such as short-time energy and short-time zero-crossing rate) is extracted from the recording to be denoised and the known speech template, which can ensure that the comparison is based on the same features in the subsequent alignment process, thereby improving the accuracy of matching. An important feature of the DTW algorithm is that it can process sequences of different lengths. By constructing a two-dimensional DTW matrix, the algorithm can find the optimal matching path between the recording to be denoised and the known speech template, even if their lengths are different. This flexibility enables the algorithm to be applicable to various practical application scenarios. According to the matching path determined by the target element of the DTW matrix, the recording denoising platform can more accurately evaluate the similarity between the recording to be denoised and the known speech template. Based on this similarity, the recording denoising platform can further optimize the previously set thresholds of short-time energy and short-time zero-crossing rate, so that the division of speech segments and non-speech segments is more accurate. By accurately aligning the recording to be denoised and the known speech template, the recording denoising platform can more accurately identify the noise segment and the speech segment, so that the noise can be more accurately eliminated and the speech can be retained when performing the denoising process. This will significantly improve the denoising effect and make the processed recording clearer and purer. Through precise feature matching and sequence alignment, the recording denoising platform can more accurately identify speech segments and noise segments, thereby reducing the false positive rate. This will further improve the accuracy of speech segment detection and reduce the impact of false positives on denoising effects.

[0024] Optionally, determining the frequency spectrum features of the speech-free segment and determining the noise characteristics according to the frequency spectrum features by using a preset statistical model includes:

[0025] Performing a fast Fourier transform on the speech-free segment to convert the audio signal from the time domain to the frequency domain, filtering the frequency domain signal using a filter bank, taking the logarithmic energy of the output of each filter, and obtaining Mel-frequency cepstrum coefficients by discrete cosine transform;

[0026] Inputting the Mel-frequency cepstral coefficients into a trained Gaussian mixture model, and calculating the likelihood that the vector corresponding to the Mel-frequency cepstral coefficients belongs to a noise model;

[0027] When the likelihood is greater than the threshold, it is determined that the target frame corresponding to the Mel-frequency cepstral coefficient contains noise, and the mean and variance of the Mel-frequency cepstral coefficient of the target frame are obtained.

[0028] By adopting the above technical solution, the non-speech segment is converted from the time domain to the frequency domain using the fast Fourier transform, and the filter bank is used for filtering, so that the spectral characteristics of the noise signal can be accurately extracted. Subsequently, by calculating the logarithmic energy of each filter output and applying the discrete cosine transform, the Mel frequency cepstral coefficient (MFCC) is obtained, which further compresses the spectral characteristics and retains the key information, which is very important for the subsequent analysis of noise characteristics. The extracted MFCC is input into the pre-trained Gaussian mixture model, and by calculating the likelihood that the vector corresponding to the MFCC belongs to the noise model, it can be effectively identified whether the target frame contains noise. The embodiment of the present application is based on the principle of statistical learning, and can accurately distinguish between noise and speech signals, providing a reliable basis for subsequent noise removal processing. When the target frame is determined to contain noise, the MFCC mean and variance of the frame are further obtained, and this information can be used to update the noise model. This dynamic update mechanism enables the noise model to adapt to the changes in noise characteristics under different environments and conditions, and improves the robustness and adaptability of the system. Based on the accurately extracted noise spectrum characteristics and the effectively identified noise characteristics, the recording denoising platform can more accurately locate and eliminate the noise in the recording. This targeted noise reduction process can significantly improve the clarity and audibility of recordings and improve the user experience. By modeling and analyzing noise characteristics with a Gaussian mixture model, the recording noise reduction platform can reduce the number of cases where speech signals are misclassified as noise or noise is missed. This high-precision noise identification and processing capability helps improve the overall performance and reliability of the system.

[0029] Optionally, the denoising the speech segment according to the noise characteristic includes:

[0030] Calculating an over-subtraction factor and a gain compensation factor, subtracting a noise spectrum in the noise characteristic from a spectrum of each frame to obtain a first intermediate spectrum value, and multiplying the over-subtraction factor by the first intermediate spectrum value to obtain a second intermediate spectrum value;

[0031] The gain compensation factor is multiplied by the second intermediate spectrum value to obtain a final spectrum value, and the final spectrum value is converted into a time domain signal by inverse short-time Fourier transform to obtain a denoised audio signal.

[0032] By adopting the above technical solution, calculating the over-reduction factor and the gain compensation factor, and subtracting the noise spectrum from the spectrum frame by frame, the embodiment of the present application can accurately eliminate the noise component in the recording. This targeted processing method makes the noise removal more accurate and retains the quality and audibility of the original voice signal. While eliminating the noise, the voice signal affected by the noise can be compensated by the application of the gain compensation factor to reduce the voice distortion. This processing method can maintain the clarity and naturalness of the voice signal while removing the noise, and improve the user experience. Using the inverse short-time Fourier transform to convert the processed spectrum value back to the time domain signal is an efficient and accurate process. This conversion can ensure that after the noise is eliminated, the audio signal remains continuous and complete in the time domain without any breakage or distortion. Overall, this noise removal method based on noise characteristics can significantly improve the quality of the audio signal. By accurately eliminating noise, reducing voice distortion and efficient spectrum conversion, the processed audio signal has significantly improved in clarity, audibility and naturalness, providing users with a better auditory experience. The embodiment of the present application not only improves the audio quality, but also enhances the overall performance of the system. Because the processing is precise and efficient, the system can process large amounts of audio data faster and provide better services to users.

[0033] Optionally, the calculating of the over-subtraction factor and the gain compensation factor includes:

[0034] Performing Fourier transformation on the non-speech segment to calculate a first power spectrum density of the non-speech segment, and performing Fourier transformation on the speech segment to calculate a second power spectrum density of the speech segment;

[0035] Performing a Bark transformation on the first power spectral density to obtain a first Bark spectral density, and performing a Bark transformation on the second power spectral density to obtain a second Bark spectral density;

[0036] Calculating a first Bark spectral density sum of the speech-free segment exceeding a preset hearing threshold according to the first Bark spectral density, and calculating a second Bark spectral density sum of the speech-with segment exceeding a preset hearing threshold according to the second Bark spectral density;

[0037] A ratio of the first Bark spectral density sum to the second Bark spectral density sum is calculated, and the ratio is used as a gain compensation factor.

[0038] By adopting the above technical solution, Fourier transform is performed on the non-speech segment and the speech segment respectively, and their respective power spectrum densities are calculated, and further Bark transformation is performed to obtain Bark spectrum density. The embodiment of the present application can accurately analyze the characteristics of noise and speech on the spectrum. This helps to more accurately identify and process noise and retain useful information in the speech signal. The calculation of Bark spectrum density is based on the auditory characteristics of the human ear and can better reflect the human ear's perception of sound. By converting the power spectrum density into Bark spectrum density, the auditory impact of the sound signal can be more accurately evaluated, so that the subsequent noise removal processing is more in line with the auditory perception of the human ear. By calculating the sum of the Bark spectrum densities of the non-speech segment and the speech segment exceeding the preset auditory threshold, and calculating their ratio as the gain compensation factor, the embodiment of the present application can accurately quantify the impact of noise on the speech signal, and compensate the speech signal accordingly. This accurate gain compensation factor calculation can ensure that the quality and clarity of the speech signal are retained as much as possible while eliminating noise. Based on the gain compensation factor calculated above, the speech signal is subjected to noise removal processing, which can more accurately eliminate noise and reduce speech distortion. This processing method can significantly improve the quality of audio signals, making them clearer, more natural and easier to understand. Since the embodiments of the present application are based on accurate audio signal processing and spectrum analysis, they can adapt to different types of noise and speech signals, and therefore have stronger robustness. This makes the embodiments of the present application more reliable and stable in practical applications, and can cope with audio processing requirements in various complex scenarios.

[0039] Optionally, the recording denoising method includes:

[0040] Compression parameters in the DRC algorithm are set, wherein the compression parameters include a compression threshold, a compression ratio, an attack time, and a release time, and the recording to be denoised after denoising is processed by the DRC algorithm.

[0041] By adopting the above technical solution, the DRC algorithm is used to adjust the dynamic range of the audio signal, so that the amplitude change of the audio signal is smoother. By setting appropriate compression threshold, compression ratio and other parameters, the peak value of the audio signal can be limited to avoid too high or too low volume, so that the audio sounds more comfortable. During the recording process, due to environmental noise or other interference factors, there may be some noise or noise in the recording. By compressing the audio with the DRC algorithm, the amplitude of these noises or noises can be effectively reduced, making the voice or music in the audio clearer and more audible. The attack time and release time parameters in the DRC algorithm can control the rate of change of the audio signal. By setting these parameters reasonably, the audio signal can be made more stable in volume, avoid abrupt volume changes, and improve the consistency and coherence of the audio. The DRC algorithm can be dynamically adjusted according to the characteristics of the audio signal, so that the audio signal is more in line with the auditory habits of the human ear while maintaining the original characteristics. By setting the compression parameters reasonably, the audio can be made fuller and more three-dimensional in terms of volume and timbre, and the expressiveness of the audio can be enhanced. The recording processed by the DRC algorithm not only has significantly improved the sound quality, but also better meets the needs of users in terms of volume and clarity. This enables users to have a better listening experience when listening to recordings, improving user satisfaction and loyalty.

[0042] In a second aspect of the present application, a recording denoising system based on sound features is provided, comprising a calculation module, a division module, a feature module and an execution module, wherein:

[0043] A calculation module is configured to divide the recording to be denoised into multiple frames, and calculate the short-time energy and short-time zero-crossing rate of each frame, where each frame is an audio signal of a preset time length;

[0044] A division module, configured to divide the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate;

[0045] A feature module configured to determine the frequency spectrum features of the speech-free segment and determine the noise characteristics according to the frequency spectrum features through a preset statistical model;

[0046] An execution module is configured to denoise the speech segment according to the noise characteristics.

[0047] In the third aspect of the present application, an electronic device is provided, including a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes any one of the methods described above.

[0048] In a fourth aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores instructions, and when the instructions are executed, any of the methods described above is executed.

[0049] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0050] 1. By dividing the recording to be denoised into multiple frames and calculating the short-time energy and short-time zero-crossing rate of each frame, the speech segment and the non-speech segment can be accurately distinguished. Short-time energy and short-time zero-crossing rate are commonly used characteristic parameters in speech signal analysis. They can effectively reflect the energy distribution and frequency changes of speech signals in different time periods, thereby helping the algorithm to accurately identify speech segments and noise segments;

[0051] 2. After determining the non-speech segments, further determine the spectral characteristics of these segments, and determine the noise characteristics based on these spectral characteristics through a preset statistical model, which can accurately capture the spectral characteristics of the noise and provide important reference information for the subsequent denoising of the speech segments;

[0052] 3. According to the determined noise characteristics, the speech segment is subjected to noise removal processing. Since the noise characteristics have been accurately determined, the noise can be removed more specifically while retaining the useful information in the speech signal. This processing method can significantly improve the clarity and audibility of the recording and improve the user's listening experience.

[0053] 4. De-noising is performed on recordings based on sound features, so it can adapt to different types of recordings and noise environments. Whether it is a voice call recording, a music recording, or other types of recordings, as long as effective sound features can be extracted, the embodiments of the present application can achieve effective noise removal effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of a method for removing noise from a recording based on sound features disclosed in an embodiment of the present application;

[0055] Figure 2 It is a module diagram of a recording noise removal system based on sound features disclosed in an embodiment of the present application;

[0056] Figure 3 It is a structural schematic diagram of an electronic device disclosed in an embodiment of the present application.

[0057] Explanation of the reference numerals: 201, computing module; 202, partitioning module; 203, feature module; 204, execution module; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0058] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0059] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0060] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0061] This embodiment discloses a recording denoising method based on sound features, which is applied to a recording denoising platform. Figure 1 is a flow chart of a method for removing noise from a recording based on sound features disclosed in an embodiment of the present application, such as Figure 1 As shown, the recording noise removal method includes the following steps:

[0062] S110, dividing the recording to be denoised into multiple frames, calculating the short-time energy and short-time zero-crossing rate of each frame, each frame being an audio signal of a preset time length;

[0063] Recording segmentation refers to dividing a continuous audio signal into several small segments according to a certain time length, each of which is called a frame. This process is usually implemented using a window function, which is a function with a specific shape on the time axis and is used to intercept a part of the audio signal. The selection of the preset time length is usually based on the characteristics of the audio signal and the application requirements. For example, in speech recognition, the frame length is usually selected to be 10 to 30 milliseconds to ensure that sufficient speech information can be captured. Frames can be continuous or overlapping. Overlapping frames can provide more contextual information and help to analyze audio signals more accurately. Short-time energy refers to the sum of the energy of all sampling points in a frame of audio signal. It reflects the average energy level of the audio signal within the frame time. The short-time zero-crossing rate refers to the number of times the signal passes through the zero value in a frame of audio signal. It reflects the frequency change characteristics of the audio signal within the frame time.

[0064] S120, dividing the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate;

[0065] Short-time energy reflects the average energy level of the speech signal of the frame. In the speech signal, the energy of the voiced segment is usually higher than that of the unvoiced segment and the noise segment. Therefore, short-time energy can be used to distinguish the voiced area (speech segment) from the silent area (no-speech segment). For example, an energy threshold is set. When the short-time energy of a frame exceeds the threshold, the frame is considered to belong to the speech segment; otherwise, the frame is considered to belong to the no-speech segment. The short-time zero-crossing rate reflects the frequency change characteristics of the speech signal of the frame. In the speech signal, the zero-crossing rate of the speech segment is usually higher than the zero-crossing rate of the noise segment. In addition, the zero-crossing rate of the noise segment is also often high, but its energy distribution is usually more uniform. Combined with short-time energy, the short-time zero-crossing rate can be used to more accurately divide the speech segment and the no-speech segment. For example, when the short-time energy of a frame exceeds the energy threshold and its short-time zero-crossing rate is also high, it can be considered that the frame belongs to the speech segment; when the short-time energy of a frame is low and its short-time zero-crossing rate is also low, it can be considered that the frame belongs to the no-speech segment.

[0066] Optionally, dividing the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate includes:

[0067] When the short-time energies of a first preset number of consecutive frames are all greater than a first preset short-time energy threshold, taking the start frame of the first preset number of consecutive frames as the starting point of the speech segment;

[0068] When the short-time energies of a second preset number of consecutive frames are all less than the first preset short-time energy threshold, taking the start frame of the second preset number of consecutive frames as the end point of the speech segment;

[0069] Determine a speech segment according to the starting point and the end point, classify frames in the speech segment whose short-time energy is less than a second preset short-time energy threshold into a speech-free segment, and classify frames in the speech segment whose short-time zero-crossing rate is greater than a preset short-time zero-crossing rate threshold into a speech-free segment;

[0070] Aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm, and adjusting the first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold according to the alignment result;

[0071] The recording to be denoised is re-divided according to the adjusted first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold.

[0072] It is necessary to set a first preset short-time energy threshold, a second preset short-time energy threshold and a preset short-time zero-crossing rate threshold, which are usually determined based on experience or experiments. Traverse each frame of the recording to be denoised, and when the short-time energy of the first preset number of frames (for example, 3 frames or 5 frames) is greater than the first preset short-time energy threshold, the starting frame of the continuous frame is used as the starting point of the speech segment, which indicates the beginning of the speech signal. Similarly, when the short-time energy of the second preset number of frames (the second preset number can be the same as the first preset number, or different from the first preset number) is less than the first preset short-time energy threshold, the starting frame of the continuous frame is used as the end point of the speech segment, which indicates the end of the speech signal. After determining the speech segment according to the starting point and the end point, further refine the division. The frames in the speech segment with short-time energy less than the second preset short-time energy threshold (usually lower than the first preset short-time energy threshold) are divided into the speech-free segment. In addition, the frames in the speech segment with a short-time zero-crossing rate greater than the preset short-time zero-crossing rate threshold are also divided into the speech-free segment. These frames may contain non-speech sounds or noise.

[0073] In order to more accurately divide the speech segment and the non-speech segment, the dynamic time warping algorithm (DTW) can be used to align the divided recording to be denoised with multiple known speech templates. The DTW algorithm can measure the similarity between two sequences, even if they are offset or stretched in time. According to the alignment result of DTW, the matching degree between the divided speech segment and the known speech template is analyzed. If the matching degree is low, it may be caused by improper threshold setting. According to the alignment result, the first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold are dynamically adjusted. For example, if too many non-speech frames are mistakenly divided into speech segments, the thresholds of short-time energy and short-time zero-crossing rate can be increased; conversely, if too many speech frames are mistakenly divided into non-speech segments, these thresholds can be lowered. The recording to be denoised is re-divided using the adjusted threshold. Repeat the above steps of dividing the speech segment and the non-speech segment to obtain a more accurate division of the speech segment and the non-speech segment. This process can be iterated multiple times until a satisfactory division result is obtained or the preset number of iterations is reached. By dynamically adjusting the threshold, the accuracy of dividing speech segments and non-speech segments can be improved, providing a better foundation for subsequent tasks such as speech enhancement and speech recognition.

[0074] By setting a reasonable short-time energy threshold and combining the energy judgment of continuous frames, the starting point and end point of the speech signal can be accurately identified. This is of vital importance for subsequent speech processing, such as speech recognition and speech enhancement. In the speech segment, not only based on the short-time energy, but also combined with the short-time zero-crossing rate, the speech frame and the non-speech frame are further distinguished. This detailed distinction helps to improve the accuracy and efficiency of speech processing. The dynamic time warping algorithm is used to align the divided recording with the known speech template, and the thresholds of short-time energy and short-time zero-crossing rate are dynamically adjusted according to the alignment results. This adaptive method can further improve the accuracy of speech segment division and adapt it to different recording environments and speech characteristics. By combining multiple features and adaptive adjustment strategies, it can maintain high robustness under different recording conditions, and achieve more accurate speech segment division even in the case of large noise or poor speech quality. Accurate speech segment division can provide more valuable data input for subsequent speech processing, thereby improving the accuracy of speech recognition, the effect of speech enhancement, etc. This is of great significance to improving the performance of the entire speech processing system.

[0075] Optionally, aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm includes:

[0076] Extracting a sequence with the same features from the recording to be denoised and the known speech template, wherein the features include short-time energy and short-time zero-crossing rate;

[0077] Constructing a two-dimensional DTW matrix according to a first length of the sequence of the to-be-denoised recordings and a second length of the sequence of the known speech templates;

[0078] A matching path is determined according to the target element of the DTW matrix, and the recording to be denoised is aligned with the known speech template according to the matching path.

[0079] Extract sequences with the same features from the denoised recording and the known speech template. In this scenario, the features of interest are mainly short-time energy and short-time zero-crossing rate. With these two features, a two-dimensional feature sequence matrix can be constructed for each denoised recording and known speech template, where each row represents a frame and each column represents a feature (short-time energy or short-time zero-crossing rate). A two-dimensional DTW matrix is ​​constructed based on the sequence length of the denoised recording (assuming the first length) and the sequence length of the known speech template (assuming the second length). Assuming that the feature sequence length of the denoised recording is N and the feature sequence length of the speech template is M, the size of the DTW matrix is ​​N×M. Each cell (i, j) of this matrix represents the cumulative distance between the i-th frame of the denoised recording and the j-th frame of the known speech template. The calculation of the cumulative distance is based on the feature difference (such as Euclidean distance) of the corresponding frames between the two sequences and the cumulative distance of the previous frames. The DTW algorithm allows nonlinear alignment on the time axis, so the calculation of the cumulative distance takes into account "diagonal", "horizontal" and "vertical" movements.

[0080] In the DTW matrix, it is necessary to find a path from the upper left corner to the lower right corner so that the sum of the cumulative distances of all grids on this path is minimized. This path is the so-called matching path, which represents the optimal alignment between the recording to be denoised and the known voice template. Dynamic programming is usually used to determine the matching path. Starting from the upper left corner of the DTW matrix, the cumulative distance is calculated grid by grid, and the next grid to be calculated is selected according to certain rules (such as the maximum allowed oblique, horizontal and vertical movement steps) until the lower right corner is reached. The target element in the embodiment of the present application is the element in the lower right corner of the DTW matrix. Once the matching path is determined, the recording to be denoised and the known voice template can be aligned according to this path. Specifically, each frame in the recording to be denoised can be mapped to the corresponding frame in the template, thereby achieving alignment of the two on the time axis.

[0081] By extracting the same feature sequences (such as short-time energy and short-time zero-crossing rate) from the denoised recording and the known speech template, the DTW algorithm can accurately compare these sequences to find the best match between them. This accurate match is crucial for applications such as speech recognition, speech synthesis, and speech enhancement, as it ensures that the system can correctly recognize and understand the input speech signal. The DTW algorithm allows nonlinear alignment on the time axis, which means that even if the denoised recording and the known speech template differ in speed, rhythm, or pitch, the algorithm can still find the best match between them. This flexibility makes the DTW algorithm more robust and adaptable when processing speech signals with different characteristics. By aligning the denoised recording with multiple known speech templates, the recording denoising platform can compare the similarities between the recording and different templates and make more accurate decisions based on this. For example, the recording denoising platform can select the best matching speech template as the recognition result based on the alignment results, and the recording denoising platform can adjust the parameters of the denoising algorithm based on the alignment results to improve the speech quality. This alignment-based decision-making process can significantly improve the accuracy and efficiency of speech processing. Based on the alignment results, the recording denoising platform can automatically adjust the threshold settings previously used to divide speech segments. This automated adjustment can further optimize the accuracy of speech segment division and enable the recording denoising platform to adapt to different recording environments and speech characteristics. Through continuous iteration and optimization, the recording denoising platform can gradually improve its ability to process speech signals.

[0082] S130, determining the frequency spectrum characteristics of the speech-free segment, and determining the noise characteristics according to the frequency spectrum characteristics through a preset statistical model;

[0083] Spectral features describe the energy distribution of an audio signal at different frequencies. This is usually achieved through fast Fourier transform (FFT) or similar frequency domain analysis methods. Spectral features can include spectral envelope, spectral centroid, spectral roll-off, etc. Spectral envelope: describes the trend of spectral energy changing with frequency. Spectral centroid: indicates the concentrated location of energy in the spectrum, that is, the frequency where the "center of gravity" is located. Spectral roll-off: describes the energy decay rate of the spectrum from high frequency to low frequency. After obtaining the spectral features of the speechless segment, it is necessary to use a preset statistical model to determine the noise characteristics. These statistical models are usually based on the training of a large amount of known noise data and are used to describe the statistical characteristics of the noise. Gaussian model: In some cases, the spectral features of the background noise may obey a Gaussian distribution. By calculating the mean and variance of the spectral features, a Gaussian model can be fitted to describe the noise. Through the preset statistical model, the characteristics of the noise can be determined based on the spectral features of the speechless segment. This includes the spectral shape of the noise, the power spectral density, the noise level, etc. These characteristics are crucial for subsequent tasks such as noise suppression and speech enhancement, because they provide detailed information about the noise, enabling the recording denoising platform to estimate and eliminate the noise more accurately.

[0084] Optionally, determining the frequency spectrum features of the speech-free segment and determining the noise characteristics according to the frequency spectrum features by using a preset statistical model includes:

[0085] Performing a fast Fourier transform on the speech-free segment to convert the audio signal from the time domain to the frequency domain, filtering the frequency domain signal using a filter bank, taking the logarithmic energy of the output of each filter, and obtaining Mel-frequency cepstrum coefficients by discrete cosine transform;

[0086] Inputting the Mel-frequency cepstral coefficients into a trained Gaussian mixture model, and calculating the likelihood that the vector corresponding to the Mel-frequency cepstral coefficients belongs to a noise model;

[0087] When the likelihood is greater than the threshold, it is determined that the target frame corresponding to the Mel-frequency cepstral coefficient contains noise, and the mean and variance of the Mel-frequency cepstral coefficient of the target frame are obtained.

[0088] Perform a fast Fourier transform (FFT) on the non-speech segment. FFT is an efficient algorithm for converting audio signals from the time domain to the frequency domain. Through FFT, the amplitude and phase information of the audio signal at different frequencies can be obtained. Use a filter bank to filter the frequency domain signal. The filter bank usually includes a series of bandpass filters, each of which covers a specific frequency band of the audio signal spectrum. These filters can be designed with different bandwidths and center frequencies in order to capture the characteristics of the audio signal in different frequency ranges. Take the logarithmic energy of each filter output. Logarithmic energy is the logarithmic representation of the energy value of the audio signal within the filter output frequency band. By calculating the logarithmic energy, the relative energy distribution of the audio signal in different frequency bands can be obtained, which is very useful for subsequent feature extraction. The filtered logarithmic energy is converted into Mel Frequency Cepstral Coefficients (MFCC) through Discrete Cosine Transform (DCT). MFCC is a widely used audio feature that is designed based on the auditory characteristics of the human ear and can well describe the spectral envelope of the audio signal. The calculation of MFCC usually includes the following steps: mapping the logarithmic energy to the Mel frequency scale, performing discrete cosine transform on the mapped Mel frequency logarithmic energy, and selecting a part of the coefficients in the DCT result as MFCC features. The calculated MFCC features are input into the trained Gaussian mixture model (GMM). GMM is a probabilistic model that assumes that the MFCC features of the audio signal obey a mixture of multiple Gaussian distributions. By training a large amount of audio data containing noise, a GMM model that can describe the characteristics of noise can be obtained. In the GMM model, the likelihood that each MFCC feature vector belongs to the noise model is calculated. The likelihood indicates the degree of match between the feature vector and the noise model. Generally, the higher the likelihood, the more likely the feature vector is generated by noise. When the likelihood is greater than a preset threshold, it is determined that the target frame corresponding to the MFCC feature vector contains noise. This means that the audio signal of the frame is mainly composed of noise, not speech. For the target frame determined to contain noise, the mean and variance of its MFCC features are further obtained. These means and variances describe the spectral characteristics of the noise and can be used in subsequent tasks such as noise suppression or speech enhancement.

[0089] By performing fast Fourier transform and filter bank filtering on the speech-free segment, the features of the audio signal in the frequency domain can be accurately extracted. Furthermore, by calculating the logarithmic energy and performing discrete cosine transform, the Mel-frequency cepstral coefficients (MFCC) are obtained. These MFCC features can well describe the spectral envelope of the audio signal, especially the spectral characteristics of the noise. By inputting the MFCC features into the trained Gaussian mixture model (GMM), the likelihood that the MFCC feature vector belongs to the noise model can be calculated. By setting a suitable threshold, the target frame containing noise can be accurately judged. This noise detection method based on statistical models is more accurate and reliable than traditional methods. For the target frame determined to contain noise, the spectral characteristics of the noise can be quantitatively described by obtaining the mean and variance of its MFCC features. These quantitative indicators can be further used in subsequent noise suppression, speech enhancement and other processing, providing useful reference information for these tasks. By accurately extracting and describing the noise characteristics, better input can be provided for the subsequent speech processing system. For example, in a speech recognition system, the accuracy of recognition can be improved by eliminating or reducing the impact of background noise; in a speech enhancement system, the parameters of the enhancement algorithm can be adjusted according to the noise characteristics to obtain better speech quality. Since this technical process is based on a statistical model for noise detection, it has strong adaptability and robustness. It can handle different types of noise environments and adapt to different application scenarios. In addition, by updating and training the statistical model, the performance and accuracy of the system can be further improved.

[0090] S140: De-noise the speech segment according to the noise characteristics.

[0091] According to the noise characteristics obtained by analysis, a noise model can be established. This model can be a statistical model (such as a Gaussian mixture model) or other types of models used to describe and simulate the characteristics of noise. When processing a speech segment, it is necessary to use the previously established noise model to perform noise removal. This usually involves the following steps: extracting the spectral features of the speech segment (such as MFCC), matching or comparing these features with the noise model to estimate the noise components in the current speech segment, and using appropriate algorithms and techniques (such as spectral subtraction, Wiener filtering, minimum mean square error estimation, etc.) to eliminate or weaken these estimated noise components. Finally, the processed spectrum is inversely transformed (such as inverse FFT) to obtain the denoised time domain audio signal.

[0092] Optionally, the denoising the speech segment according to the noise characteristic includes:

[0093] Calculating an over-subtraction factor and a gain compensation factor, subtracting a noise spectrum in the noise characteristic from a spectrum of each frame to obtain a first intermediate spectrum value, and multiplying the over-subtraction factor by the first intermediate spectrum value to obtain a second intermediate spectrum value;

[0094] The gain compensation factor is multiplied by the second intermediate spectrum value to obtain a final spectrum value, and the final spectrum value is converted into a time domain signal by inverse short-time Fourier transform to obtain a denoised audio signal.

[0095] The over-subtraction factor is a coefficient greater than 1, which is used to more strongly reduce the noise component in the spectrum subtraction. By selecting an appropriate over-subtraction factor, the noise can be better eliminated, but it may also cause speech distortion. Therefore, the selection of the over-subtraction factor needs to find a balance between eliminating noise and maintaining speech quality. Since the over-subtraction factor may cause the overall energy of the speech signal to decrease, a gain compensation factor is required to restore the energy of the speech signal. The gain compensation factor is usually calculated based on the energy of the original speech signal and the estimated noise signal. For each frame of the audio signal (usually obtained by short-time Fourier transform (STFT)), its spectrum is subtracted from the known noise spectrum. Here, the noise spectrum is extracted from the speech-free segment and used to represent the current noise characteristics. The subtraction operation can be performed in the logarithmic spectrum domain or the linear spectrum domain, but it is usually more common in the logarithmic spectrum domain because it is more in line with the auditory characteristics of the human ear. The subtraction result (i.e., the first intermediate spectrum value) is multiplied by the over-subtraction factor. This step is to more strongly reduce the noise component, but it may also cause distortion of the speech signal. Therefore, the selection of the over-subtraction factor needs to be carefully considered. The over-subtracted spectrum value (i.e., the second intermediate spectrum value) is multiplied by the gain compensation factor to restore the energy of the speech signal. This step is to compensate for the decrease in speech signal energy caused by the over-subtraction operation. The final spectrum value (i.e., the final spectrum value) is converted back to the time domain signal through the inverse short-time Fourier transform (ISTFT). This process is the inverse operation of STFT, which is used to convert the audio signal represented by the spectrum back to the original time domain signal. The time domain signal obtained by ISTFT is the denoised audio signal. The noise component in this signal has been greatly weakened or eliminated, while the speech component has been retained.

[0096] By subtracting the known noise spectrum from the spectrum of each frame, the noise component in the audio signal can be significantly reduced. This denoising method is particularly effective when the noise characteristics are stable and are spectrally distinguishable from the speech signal. The introduction of the over-subtraction factor and the gain compensation factor enables the technology to flexibly adjust the impact on the speech signal while suppressing noise. The over-subtraction factor is used to enhance the noise suppression effect, while the gain compensation factor is used to restore the speech signal energy that may be reduced due to noise suppression, thereby maintaining the clarity and intelligibility of the speech. Although spectral subtraction may cause speech distortion in some cases, by reasonably setting the over-subtraction factor and the gain compensation factor, the technology can maximize the naturalness and intelligibility of the speech while reducing noise. This is particularly important for application scenarios that need to retain speech information. Since the technology is mainly based on spectrum analysis and mathematical operations, it has high real-time processing capabilities. This makes the technology suitable for a variety of application scenarios that require real-time noise removal, such as voice communication, speech recognition, etc. The technology is based on standard digital signal processing algorithms and statistical models, so it is easy to implement and optimize. By adjusting the algorithm parameters and model structure, the noise removal effect and speech quality can be further improved.

[0097] Optionally, the calculating of the over-subtraction factor and the gain compensation factor includes:

[0098] Performing Fourier transformation on the non-speech segment to calculate a first power spectrum density of the non-speech segment, and performing Fourier transformation on the speech segment to calculate a second power spectrum density of the speech segment;

[0099] Performing a Bark transformation on the first power spectral density to obtain a first Bark spectral density, and performing a Bark transformation on the second power spectral density to obtain a second Bark spectral density;

[0100] Calculating a first Bark spectral density sum of the speech-free segment exceeding a preset hearing threshold according to the first Bark spectral density, and calculating a second Bark spectral density sum of the speech-with segment exceeding a preset hearing threshold according to the second Bark spectral density;

[0101] A ratio of the first Bark spectral density sum to the second Bark spectral density sum is calculated, and the ratio is used as a gain compensation factor.

[0102] Fourier transform is performed on the non-speech segment (usually the pure noise part in the audio signal). Fourier transform can convert the signal from the time domain to the frequency domain, thereby obtaining the spectral representation of the signal. The power spectral density (PSD) of the non-speech segment is calculated. The power spectral density describes the power distribution of the signal at different frequencies and is an important parameter in frequency domain analysis. Similarly, the speech segment (a mixed signal containing speech and noise) is also Fourier transformed, and its power spectral density is calculated. Bark transform is a method of converting frequency to psychoacoustic scale. Since the human auditory system has different sensitivities to different frequencies, the use of the Bark scale can more accurately describe the human ear's perception of sound. In this step, the power spectral densities of the obtained non-speech segment and speech segment are Bark transformed to obtain their spectral densities on the Bark scale, namely the first Bark spectral density and the second Bark spectral density. The preset hearing threshold is the minimum sound intensity that the human ear can perceive. In this step, it is necessary to calculate the sum of the spectral densities of the non-speech segment and the speech segment that exceed this threshold on the Bark scale. Specifically, it is to traverse each value of the Bark spectral density, add up the values ​​greater than the preset hearing threshold, and obtain the first Bark spectral density sum of the speechless segment and the second Bark spectral density sum of the speech segment. Finally, the gain compensation factor is calculated based on the Bark spectral density sum of the speechless segment and the speech segment that exceeds the preset hearing threshold. The gain compensation factor is used to adjust the power spectral density of the speech segment to compensate for the speech distortion that may be caused by noise suppression. The formula for calculating the gain compensation factor is: Gain compensation factor = first Bark spectral density sum / second Bark spectral density sum. This ratio reflects the energy ratio of the speechless segment (pure noise) and the speech segment in the part that exceeds the hearing threshold.

[0103] Using Bark transformation to convert power spectrum density to Bark spectrum density can more accurately match the auditory characteristics of the human ear. Since the human ear has different sensitivities to different frequencies, the Bark scale can more accurately reflect the human ear's perception of sound, making audio processing more in line with the human ear's auditory habits. By calculating the power spectrum density of the speechless segment (pure noise) and the speech segment and performing Bark transformation on them, the distribution of noise and speech in the frequency domain can be accurately evaluated. This provides an important basis for subsequent noise suppression, allowing the algorithm to more accurately identify and suppress noise. The calculation of the gain compensation factor is based on the ratio of the sum of the Bark spectrum densities of the speechless segment and the speech segment that exceeds the preset hearing threshold. This calculation method can ensure that the quality of the speech is maintained as much as possible while suppressing noise. By adjusting the gain compensation factor, the speech distortion caused by noise suppression can be effectively compensated, making the processed speech signal clearer and more natural. This technical process is not only applicable to static noise environments, but also to dynamic noise environments. Since the gain compensation factor is calculated based on the actual noise and speech characteristics, it can be adaptively adjusted according to different audio signals and noise environments, thus achieving more flexible and effective audio processing. This technical process is based on standard digital signal processing algorithms and psychoacoustic models, so it is easy to implement and optimize. By adjusting the algorithm parameters and model structure, the effect and performance of audio processing can be further improved.

[0104] Optionally, the recording denoising method includes:

[0105] Compression parameters in the DRC algorithm are set, wherein the compression parameters include a compression threshold, a compression ratio, an attack time, and a release time, and the recording to be denoised after denoising is processed by the DRC algorithm.

[0106] Dynamic Range Compression (DRC) algorithm is an audio processing technology designed to reduce the dynamic range of audio signals. It can automatically adjust the volume according to the strength of the audio signal to ensure that the volume of the audio signal is more balanced during playback. DRC algorithm usually includes parameters such as compression threshold, compression ratio, attack time and release time, which can be adjusted according to specific needs. Compression threshold: The compression threshold determines when the audio signal starts to be compressed. When the strength of the audio signal exceeds the compression threshold, the DRC algorithm will start to adjust the volume. The setting of the compression threshold can be adjusted according to the characteristics and needs of the recording to achieve the best volume balancing effect; Compression ratio: The compression ratio determines the degree to which the audio signal is compressed after exceeding the compression threshold. The larger the compression ratio, the smaller the volume change range of the audio signal, and the audio will sound more stable. However, too high a compression ratio may cause the audio to lose its dynamic sense, so it is necessary to select a suitable compression ratio according to the actual situation; Attack time: The attack time refers to the speed at which the DRC algorithm starts to adjust the volume after the audio signal exceeds the compression threshold. A shorter attack time can make the DRC algorithm respond to changes in the audio signal faster, but it may also cause the sound quality of the audio signal to be affected. Therefore, it is necessary to select a suitable attack time according to the actual situation; Release time: The release time refers to the speed at which the DRC algorithm gradually stops adjusting the volume after the audio signal falls below the compression threshold. A longer release time can make the sound quality of the audio signal more natural, but it may also cause the volume of the audio signal to change insufficiently quickly. Therefore, it is necessary to select a suitable release time according to the actual situation. After setting the compression parameters of the DRC algorithm, it can be applied to the recording after noise removal. Through the processing of the DRC algorithm, the volume of the recording will be more balanced and the sound quality will be further improved. During the processing, the changes in the audio signal can be monitored in real time, and the parameters of the DRC algorithm can be adjusted as needed to achieve the best noise removal and sound quality optimization effect.

[0107] The core function of the DRC algorithm is to adjust the dynamic range of the audio signal to ensure that the volume of the audio signal is more balanced when it is played. This is particularly important for recording noise removal, because noise often affects the volume balance of the audio signal, and the DRC algorithm can effectively improve this, making the recording sound more stable and consistent. By adjusting the compression parameters, the DRC algorithm can reduce the interference of noise to a certain extent. For example, when there is continuous background noise in the recording, the DRC algorithm can treat it as a low-intensity signal and attenuate it during the volume adjustment process, thereby reducing the impact of noise on the recording quality. In addition to reducing noise interference, the DRC algorithm can also improve the sound quality by optimizing the dynamic range of the audio signal. For example, when there is a sudden increase in volume in the recording (such as sudden noise or screaming), the DRC algorithm can quickly reduce the volume of these parts to avoid distortion or distortion of the audio signal, thereby improving the sound quality. The compression parameters in the DRC algorithm can be adjusted according to the characteristics and needs of the recording, which makes the algorithm highly adaptable. Whether it is processing different types of recordings (such as voice, music, ambient sound, etc.), or processing noise of different intensities, the best noise removal effect can be achieved by adjusting the compression parameters. The DRC algorithm can process recordings in real time, which means that noise removal and sound quality optimization can be performed during the recording process. This is especially important for recording scenarios that require real-time feedback (such as live broadcasts, conference calls, etc.), ensuring that listeners hear high-quality audio signals at the first time.

[0108] This embodiment also discloses a recording noise removal system based on sound features. Figure 2 is a module diagram of a recording noise reduction system based on sound features disclosed in an embodiment of the present application, such as Figure 2 As shown, the recording noise removal system includes a calculation module 201, a division module 202, a feature module 203 and an execution module 204, wherein:

[0109] A calculation module 201 is configured to divide the recording to be denoised into multiple frames, and calculate the short-time energy and short-time zero-crossing rate of each frame, where each frame is an audio signal of a preset time length;

[0110] A division module 202 is configured to divide the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate;

[0111] A feature module 203 is configured to determine the frequency spectrum features of the speech-free segment and determine the noise characteristics according to the frequency spectrum features through a preset statistical model;

[0112] The execution module 204 is configured to perform noise reduction on the speech segment according to the noise characteristics.

[0113] Optionally, the division module 202 is configured to:

[0114] When the short-time energies of a first preset number of consecutive frames are all greater than a first preset short-time energy threshold, taking the start frame of the first preset number of consecutive frames as the starting point of the speech segment;

[0115] When the short-time energies of a second preset number of consecutive frames are all less than the first preset short-time energy threshold, taking the start frame of the second preset number of consecutive frames as the end point of the speech segment;

[0116] Determine a speech segment according to the starting point and the end point, classify frames in the speech segment whose short-time energy is less than a second preset short-time energy threshold into a speech-free segment, and classify frames in the speech segment whose short-time zero-crossing rate is greater than a preset short-time zero-crossing rate threshold into a speech-free segment;

[0117] Aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm, and adjusting the first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold according to the alignment result;

[0118] The recording to be denoised is re-divided according to the adjusted first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold.

[0119] Optionally, the division module 202 is configured to:

[0120] Extracting a sequence with the same features from the recording to be denoised and the known speech template, wherein the features include short-time energy and short-time zero-crossing rate;

[0121] Constructing a two-dimensional DTW matrix according to a first length of the sequence of the to-be-denoised recordings and a second length of the sequence of the known speech templates;

[0122] A matching path is determined according to the target element of the DTW matrix, and the recording to be denoised is aligned with the known speech template according to the matching path.

[0123] Optionally, the feature module 203 is configured to:

[0124] Performing a fast Fourier transform on the speech-free segment to convert the audio signal from the time domain to the frequency domain, filtering the frequency domain signal using a filter bank, taking the logarithmic energy of the output of each filter, and obtaining Mel-frequency cepstrum coefficients by discrete cosine transform;

[0125] Inputting the Mel-frequency cepstral coefficients into a trained Gaussian mixture model, and calculating the likelihood that the vector corresponding to the Mel-frequency cepstral coefficients belongs to a noise model;

[0126] When the likelihood is greater than the threshold, it is determined that the target frame corresponding to the Mel-frequency cepstral coefficient contains noise, and the mean and variance of the Mel-frequency cepstral coefficient of the target frame are obtained.

[0127] Optionally, the execution module 204 is configured to:

[0128] Calculating an over-subtraction factor and a gain compensation factor, subtracting a noise spectrum in the noise characteristic from a spectrum of each frame to obtain a first intermediate spectrum value, and multiplying the over-subtraction factor by the first intermediate spectrum value to obtain a second intermediate spectrum value;

[0129] The gain compensation factor is multiplied by the second intermediate spectrum value to obtain a final spectrum value, and the final spectrum value is converted into a time domain signal by inverse short-time Fourier transform to obtain a denoised audio signal.

[0130] Optionally, the execution module 204 is configured to:

[0131] Performing Fourier transformation on the non-speech segment to calculate a first power spectrum density of the non-speech segment, and performing Fourier transformation on the speech segment to calculate a second power spectrum density of the speech segment;

[0132] Performing a Bark transformation on the first power spectral density to obtain a first Bark spectral density, and performing a Bark transformation on the second power spectral density to obtain a second Bark spectral density;

[0133] Calculating a first Bark spectral density sum of the speech-free segment exceeding a preset hearing threshold according to the first Bark spectral density, and calculating a second Bark spectral density sum of the speech-with segment exceeding a preset hearing threshold according to the second Bark spectral density;

[0134] A ratio of the first Bark spectral density sum to the second Bark spectral density sum is calculated, and the ratio is used as a gain compensation factor.

[0135] Optionally, the recording noise reduction system includes an enhancement module, and the enhancement module is configured to:

[0136] Compression parameters in the DRC algorithm are set, wherein the compression parameters include a compression threshold, a compression ratio, an attack time, and a release time, and the recording to be denoised after denoising is processed by the DRC algorithm.

[0137] It should be noted that: when the device provided in the above embodiment realizes its function, only the division of the above functional modules is used as an example. In actual application, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0138] This embodiment also discloses an electronic device, referring to Figure 3 The electronic device may include: at least one processor 301 , at least one communication bus 302 , a user interface 303 , a network interface 304 , and at least one memory 305 .

[0139] The communication bus 302 is used to realize the connection and communication between these components.

[0140] The user interface 303 may include a display screen (Display) and a camera (Camera). The optional user interface 303 may also include a standard wired interface and a wireless interface.

[0141] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0142] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0143] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 305 may optionally be at least one storage device located away from the aforementioned processor 301. As Figure 3 As shown, the memory 305 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an application program of a recording denoising method based on sound features.

[0144] exist Figure 3 In the electronic device shown, the user interface 303 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 301 can be used to call the application program of the recording denoising method based on sound features stored in the memory 305. When executed by one or more processors 301, the electronic device executes one or more methods in the above-mentioned embodiments.

[0145] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0146] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0150] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory 305 and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory 305 includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk.

[0151] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification, those skilled in the art will easily think of other embodiments of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A recording denoising method based on sound features, characterized in that: Applied to a recording denoising platform, the recording denoising method includes: The recording to be denoised is divided into multiple frames, and the short-time energy and short-time zero-crossing rate of each frame are calculated, where each frame is an audio signal of a preset time length; Dividing the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate; Determine the frequency spectrum characteristics of the speech-free segment, and determine the noise characteristics according to the frequency spectrum characteristics through a preset statistical model; De-noising the speech segment according to the noise characteristics.

2. The recording denoising method based on sound features according to claim 1, characterized in that: The step of dividing the recording to be denoised into a speech segment and a speech-free segment according to the short-time energy and the short-time zero-crossing rate comprises: When the short-time energies of a first preset number of consecutive frames are all greater than a first preset short-time energy threshold, taking the start frame of the first preset number of consecutive frames as the starting point of the speech segment; When the short-time energies of a second preset number of consecutive frames are all less than the first preset short-time energy threshold, taking the start frame of the second preset number of consecutive frames as the end point of the speech segment; Determine a speech segment according to the starting point and the end point, classify frames in the speech segment whose short-time energy is less than a second preset short-time energy threshold into a speech-free segment, and classify frames in the speech segment whose short-time zero-crossing rate is greater than a preset short-time zero-crossing rate threshold into a speech-free segment; Aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm, and adjusting the first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold according to the alignment result; The recording to be denoised is re-divided according to the adjusted first preset short-time energy, the second preset short-time energy threshold and the preset short-time zero-crossing rate threshold.

3. The recording denoising method based on sound features according to claim 2, characterized in that: The aligning the divided recording to be denoised with a plurality of known speech templates according to a preset dynamic time warping algorithm comprises: Extracting a sequence with the same features from the recording to be denoised and the known speech template, wherein the features include short-time energy and short-time zero-crossing rate; Constructing a two-dimensional DTW matrix according to a first length of the sequence of the to-be-denoised recordings and a second length of the sequence of the known speech templates; A matching path is determined according to the target element of the DTW matrix, and the recording to be denoised is aligned with the known speech template according to the matching path.

4. The recording denoising method based on sound features according to claim 1, characterized in that: The determining of the frequency spectrum feature of the speech-free segment and determining the noise characteristic according to the frequency spectrum feature by using a preset statistical model comprises: Performing a fast Fourier transform on the speech-free segment to convert the audio signal from the time domain to the frequency domain, filtering the frequency domain signal using a filter bank, taking the logarithmic energy of the output of each filter, and obtaining Mel-frequency cepstrum coefficients by discrete cosine transform; Inputting the Mel-frequency cepstral coefficients into a trained Gaussian mixture model, and calculating the likelihood that the vector corresponding to the Mel-frequency cepstral coefficients belongs to a noise model; When the likelihood is greater than the threshold, it is determined that the target frame corresponding to the Mel-frequency cepstral coefficient contains noise, and the mean and variance of the Mel-frequency cepstral coefficient of the target frame are obtained.

5. The recording denoising method based on sound features according to claim 1, characterized in that: The denoising of the speech segment according to the noise characteristics comprises: Calculating an over-subtraction factor and a gain compensation factor, subtracting a noise spectrum in the noise characteristic from a spectrum of each frame to obtain a first intermediate spectrum value, and multiplying the over-subtraction factor by the first intermediate spectrum value to obtain a second intermediate spectrum value; The gain compensation factor is multiplied by the second intermediate spectrum value to obtain a final spectrum value, and the final spectrum value is converted into a time domain signal by inverse short-time Fourier transform to obtain a denoised audio signal.

6. The recording denoising method based on sound features according to claim 5, characterized in that: The calculation of the over-subtraction factor and the gain compensation factor includes: Performing Fourier transformation on the non-speech segment to calculate a first power spectrum density of the non-speech segment, and performing Fourier transformation on the speech segment to calculate a second power spectrum density of the speech segment; Performing a Bark transformation on the first power spectral density to obtain a first Bark spectral density, and performing a Bark transformation on the second power spectral density to obtain a second Bark spectral density; Calculating a first Bark spectral density sum of the speech-free segment exceeding a preset hearing threshold according to the first Bark spectral density, and calculating a second Bark spectral density sum of the speech-with segment exceeding a preset hearing threshold according to the second Bark spectral density; A ratio of the first Bark spectral density sum to the second Bark spectral density sum is calculated, and the ratio is used as a gain compensation factor.

7. The recording denoising method based on sound features according to claim 1, characterized in that: The recording denoising method comprises: Compression parameters in the DRC algorithm are set, wherein the compression parameters include a compression threshold, a compression ratio, an attack time, and a release time, and the recording to be denoised after denoising is processed by the DRC algorithm.

8. A recording noise removal system based on sound features, characterized in that: It includes a calculation module, a partitioning module, a feature module and an execution module, wherein: A calculation module is configured to divide the recording to be denoised into multiple frames, and calculate the short-time energy and short-time zero-crossing rate of each frame, where each frame is an audio signal of a preset time length; A division module, configured to divide the recording to be denoised into a speech segment and a non-speech segment according to the short-time energy and the short-time zero-crossing rate; A feature module configured to determine the frequency spectrum features of the speech-free segment and determine the noise characteristics according to the frequency spectrum features through a preset statistical model; An execution module is configured to denoise the speech segment according to the noise characteristics.

9. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are both used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed, the method according to any one of claims 1 to 7 is performed.

Images