Live broadcast sound card noise reduction method based on environment feature recognition

Abstract

The invention provides a live broadcast sound card noise reduction method based on environment feature recognition, relates to the technical field of noise reduction, and can realize dynamic adaptation and precise processing of multiple types of noise and reverberation factors in a live broadcast scene by constructing a noise reduction process oriented to environment feature recognition, thereby remarkably improving the definition and the reduction degree of voice signals. In particular, in a complex or violently changing live broadcast environment, intelligent identification can be carried out based on multiple parameters such as reverberation time, environmental noise energy and background spectrum entropy characteristics, and it is ensured that the noise reduction process has pertinence and real-time performance. Through a multi-stage processing strategy, reverberation tail sound is effectively suppressed, and residual noise is adaptively recognized and filtered, so that the common problem of voice distortion or detail loss in a traditional noise reduction method is avoided.

Description

Technical Field

[0001] The present invention relates to the field of noise reduction technology, and more specifically, to a live broadcast sound card noise reduction method based on environmental feature recognition. Background Art

[0002] With the rapid development of real-time audio interaction technologies such as online live streaming, remote conferencing, and voice-based social networking, the performance requirements for voice signal processing equipment, especially live streaming sound cards, continue to increase. Traditional sound cards rely solely on fixed parameter configurations and simple filter banks for noise reduction, making them difficult to adapt to complex and changing real-world scenarios, such as reverberant indoor environments, sudden noise interference, or persistent background sounds. A growing number of researchers are attempting to combine machine auditory feature analysis, acoustic modeling, and adaptive signal processing methods to achieve more flexible and high-fidelity speech enhancement systems. Among current mainstream technologies, audio processing models based on spectral subtraction, Wiener filtering, and deep neural networks have achieved good results in some scenarios, but they still have limitations under conditions such as multipath reverberation, dynamic noise interference, and fine-grained feature changes.

[0003] Specifically, existing technologies generally adopt an overall modeling strategy for reverberation suppression, ignoring the differences in reverberation time in different frequency bands, which makes it difficult to effectively distinguish between direct sound and reverberation tail, affecting audio quality and speech intelligibility. At the same time, some methods rely too much on static parameter settings and are unable to dynamically adjust the processing strategy according to the noise energy and complex background spectrum structure of the live broadcast environment, resulting in a serious performance degradation of the algorithm in low signal-to-noise ratio or structured noise scenarios. In addition, most noise reduction methods are only based on audio power spectrum information for analysis, lack the use of frequency domain spectral entropy features, and fail to fully explore the time-frequency coupling relationship of the signal. Especially in an environment where complex speech and background sound are intertwined, it is easy to make misjudgments or missed judgments, resulting in speech quality loss.

[0004] Therefore, a live sound card noise reduction solution based on environmental feature recognition is needed. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention is proposed. The present invention provides a live broadcast sound card noise reduction method based on environmental feature recognition.

[0006] According to one aspect of the present invention, a live sound card noise reduction method based on environmental feature recognition is provided, which includes: collecting reverberation time parameters, environmental noise energy and background spectral entropy characteristics in a live broadcast environment to construct an environmental feature vector; dynamically adjusting the gain control factor of an FIR filter group based on the reverberation time parameters in the environmental feature vector and with reference to the environmental noise energy, applying FIR filter groups in different frequency bands to estimate the reverberation time in frequency bands, and outputting a frequency-divided reverberation matrix; performing subspace mapping on an input audio stream based on the frequency-divided reverberation matrix and background spectral entropy characteristics, extracting and retaining the first nms direct sound component DS, and suppressing the reverberation tail part; wherein n is a constant; after preliminary noise reduction, detecting the time domain energy and frequency domain spectral entropy of DS respectively, and triggering a mark as a residual noise mark when the frequency domain spectral entropy is lower than a threshold and the time domain energy is higher than a threshold; when the residual noise mark is triggered, constructing an adaptive second-order filter based on DS, and performing local enhancement filtering in the marked segment, and finally outputting an optimized audio signal.

[0007] Furthermore, the reverberation time parameter is obtained by framing the input audio stream using the short-time energy window method, extracting the energy change curve of each frame, and estimating it based on the maximum attenuation segment fitting; the ambient noise energy is obtained by counting the time periods in which the energy of each frame is lower than the average energy by m% on the basis of the same framing, and calculating its average power value, where m is a constant; the background spectral entropy feature is obtained by calculating the normalized amplitude probability distribution based on the amplitude spectrum of each frame.

[0008] Furthermore, the reverberation time parameter, the environmental noise energy and the background spectrum entropy feature are added together in a weighted manner to generate the environmental feature vector.

[0009] Furthermore, the dynamically adjusting gain control factor of the FIR filter group includes: selecting a corresponding basic FIR filter group template FIR based on a reverberation time parameter, and presetting multiple filter group schemes for different reverberation levels; calculating a gain control factor based on ambient noise energy, and refining the overall gain of the filter group in the form of a nonlinear function; and multiplying the gain parameter of each FIR channel by the corresponding gain control factor to obtain a dynamically adjusted filter group.

[0010] Furthermore, the construction of the frequency division reverberation matrix includes: applying a time-frequency analysis method to the input audio stream to decompose it into multiple frequency band signals; inputting the signals of each frequency band into the corresponding adjusted FIR filter channel, performing independent filtering processing, and extracting the filtered frequency band signals; analyzing the energy attenuation characteristics based on the envelope curve of the filtered signals of each frequency band, and calculating the reverberation time parameters of each frequency band respectively; and combining the reverberation time parameters calculated for each frequency band in the order of the frequency bands to form a complete frequency division reverberation matrix.

[0011] Furthermore, the subspace mapping of the input audio stream includes: calculating the signal sparsity index of each frequency band based on the frequency division reverberation matrix and the background spectral entropy characteristics; determining the boundary between the direct sound subspace and the reverberation sound subspace according to the value range of the signal sparsity index in each frequency band; and using an orthogonal projection operator to perform subspace mapping processing on the input audio stream to extract the direct sound component DS.

[0012] Furthermore, the signal sparsity index of each frequency band is calculated as follows: in each frequency band, the corresponding reverberation time parameters are extracted based on the frequency division reverberation matrix, and a local frequency feature vector is jointly formed based on the background spectral entropy characteristics; for each local frequency feature vector, the sparsity index is calculated using a weighted method based on the coupling of local kurtosis and spectral entropy.

[0013] Furthermore, the division of the boundary includes: generating a frequency threshold curve for the sparsity index of all frequency bands based on a preset nonlinear mapping function, which serves as a preliminary boundary reference for the direct phonon space and the reverberation phonon space; dividing the subspace belonging to each frequency band according to the relative position relationship between the frequency threshold curve and the sparsity index, and marking the direct phonon space and the reverberation phonon space.

[0014] Furthermore, the direct sound component DS is reorganized by sliding according to the time window to generate a direct sound output stream with enhanced continuity. When a residual noise mark is detected, the direct sound component in the corresponding segment is extracted to establish a local time domain reference vector; the local energy spectral density is dynamically calculated based on the local time domain reference vector, and the center frequency and bandwidth of the second-order filter are adaptively set based on the local energy spectral density to generate a local enhancement filter; the segment corresponding to the residual noise mark is locally enhanced and filtered according to the local enhancement filter, and an intermediate optimization result is output; the intermediate optimization result and the original audio data without triggering the mark are reorganized on the time axis to output a continuous optimized audio signal.

[0015] Furthermore, the dynamic calculation of the local energy spectrum density includes: performing window function processing on the local time domain reference vector, using an exponential decay window to enhance the weight of the first-end component of the segment, and outputting a weighted local vector; performing fast Fourier transform on the weighted local vector, extracting the corresponding frequency components, and generating a local frequency spectrum; based on the local frequency spectrum, accumulating power density by frequency band, and dynamically calculating the local energy spectrum density.

[0016] Compared with the existing technology, the live sound card noise reduction method based on environmental feature recognition provided by the present invention can achieve dynamic adaptation and precise processing of multiple types of noise and reverberation factors in the live broadcast scene by constructing a noise reduction process for environmental feature recognition, thereby significantly improving the clarity and restoration of the voice signal. Especially in complex or drastically changing live broadcast environments, the present invention can perform intelligent identification based on multiple parameters such as reverberation time, environmental noise energy, and background spectral entropy characteristics to ensure that the noise reduction process is both targeted and real-time. Through a multi-stage processing strategy, the reverberation tail is effectively suppressed and the residual noise is adaptively identified and filtered out, thereby avoiding the common voice distortion or detail loss problems in traditional noise reduction methods.

[0017] This invention also offers significant advantages in the consistency and detail of noise reduction processing. It dynamically constructs local filters and performs fine frequency domain enhancement, making the final output audio signal more natural and continuous, significantly improving the listening experience. Compared to traditional solutions that rely solely on static filtering or fixed parameter models, this invention demonstrates greater environmental adaptability and speech fidelity. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work. In the drawings: Figure 1 The present invention is a flowchart of a live sound card noise reduction method based on environmental feature recognition according to an embodiment of the present invention.

[0019] Figure 2 This is a flowchart of dividing boundaries in a live sound card noise reduction method based on environmental feature recognition according to an embodiment of the present invention.

[0020] Figure 3 This is a flow chart of dynamically calculating local energy spectral density in a live sound card noise reduction method based on environmental feature recognition according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] Below, the exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0022] As mentioned in the above background technology, the existing technology generally adopts an overall modeling strategy for reverberation suppression, ignoring the differences in reverberation time in different frequency bands, which makes it difficult to effectively distinguish between direct sound and reverberation tail, affecting audio quality and speech intelligibility. At the same time, some methods rely too much on static parameter settings and cannot dynamically adjust the processing strategy according to the noise energy and complex background spectrum structure of the live broadcast environment, resulting in a serious performance degradation of the algorithm in low signal-to-noise ratio or structured noise scenarios. In addition, most noise reduction methods are only based on audio power spectrum information for analysis, lack the use of frequency domain spectral entropy features, and fail to fully explore the time-frequency coupling relationship of the signal. Especially in an environment where complex speech and background sound are intertwined, it is easy to make misjudgments or missed judgments, resulting in speech quality loss. Therefore, a live sound card noise reduction solution based on environmental feature recognition is needed.

[0023] Figure 1 Flowchart of the live sound card noise reduction method based on environmental feature recognition according to an embodiment of the present invention. Figure 1 As shown, the live sound card noise reduction method based on environmental feature recognition includes: In an embodiment of the present invention, step S1 includes: collecting reverberation time parameters, environmental noise energy and background spectral entropy characteristics in the live broadcast environment, and constructing an environmental feature vector based on a preset reference matrix.

[0024] Specifically, in the embodiments of the present invention, first, the reverberation time parameter refers to an important physical quantity that reflects the energy attenuation characteristics of sound after propagation in space. It is usually measured as the time required for the sound energy to drop by 60dB (i.e., RT60). Conventional technology often relies on impulse response experiments or known sound source testing. This is limited by its dependence on specific environmental and instrument conditions, making it difficult to apply to open and dynamically changing live broadcast environments. To this end, the present invention uses a short-time energy window method to frame the input audio stream, extract the energy change curve of each frame, and estimate it based on the maximum attenuation segment fitting. This avoids reliance on specialized test signals and improves application flexibility and real-time performance.

[0025] For example, in an embodiment of the present invention, the short-time energy window method involves segmenting the input audio stream into frames with a fixed length and overlap ratio, resulting in a series of temporally continuous audio frames. During the segmentation process, a weighted window function, such as a Hamming window or Hann window, is typically used to suppress spectral leakage and ensure the accuracy of subsequent energy estimation. The energy of each frame is calculated by summing the squared amplitudes of the sample points within that frame, forming a curve showing energy variation over time. After generating the energy curve, the present invention uses a method based on finding the maximum continuous decay segment. This method automatically locates the time interval with the largest energy drop in the energy curve, then performs a linear regression on this segment using a least-squares linear fitting algorithm. The time corresponding to a 60dB energy drop is calculated as the reverberation time parameter, RT60. Compared to existing methods using overall curve fitting or preset threshold truncation, this method offers the advantages of better adaptability to dynamic environmental changes and greater robustness to localized noise interference, effectively improving the stability and accuracy of reverberation time estimation.

[0026] Furthermore, in an embodiment of the present invention, the extraction of ambient noise energy is also statistically analyzed based on the framed audio frames. Traditional noise energy estimation methods are often based on the mean of the entire signal segment or inferring the noise floor level through spectral subtraction. These methods are easily interfered with by non-noise factors such as bursts and voice activity, resulting in unstable estimation. The ambient noise energy is obtained by calculating the average power value of the time period in which the energy of each frame is less than m% of the average energy on the same frame basis, where m is a constant.

[0027] For example, in the specific operation, the average energy of all audio frames is first calculated. Then, all frames with energy less than m% of the average value are counted. The sample data corresponding to these frames are extracted, and their average power density is further calculated as the environmental noise energy indicator. Through this method, the present invention can more accurately extract background noise characteristics in complex audio backgrounds. In particular, for live broadcast scenes with a large number of non-stationary signals (such as speech, applause, and environmental changes), it can significantly improve the accuracy and representativeness of noise energy estimation.

[0028] To avoid the problem of feature instability caused by conventional spectral entropy extraction methods, which mostly calculate entropy values ​​directly based on the amplitude-spectral modulus length, but are easily affected by different amplitude dimensions, the background spectral entropy feature described in the present invention calculates a normalized amplitude probability distribution based on the amplitude-spectral spectrum of each frame. Through this normalization process, the present invention effectively eliminates the interference of overall energy changes on the spectral entropy estimation, allowing the spectral entropy feature to more realistically reflect the complexity of the signal's spectral distribution, thereby improving sensitivity and recognition of changes in background noise patterns under different environmental backgrounds.

[0029] Finally, the reverberation time parameter, ambient noise energy, and background spectral entropy features are weighted and superimposed to generate an environmental feature vector. Compared to the method of directly stacking the original feature parameters, the introduction of weights in this invention allows for flexible adjustment of the importance of each feature according to different application requirements. For example, the weight of ambient noise energy can be appropriately increased when the background noise fluctuates drastically, while the weight of reverberation time parameters can be increased when the reverberation characteristics change significantly. This allows for dynamic adaptation and precise control of the environmental feature vector.

[0030] It should be understood that through the above steps, the present invention can accurately and stably extract preliminary feature information reflecting the acoustic state of the live broadcast environment directly based on the real-time input audio stream without the need for external dedicated measurement equipment and additional excitation signal input, thereby providing a reliable and dynamically adaptable environmental reference basis for subsequent noise suppression and other processing.

[0031] S2: According to the reverberation time parameter in the environmental feature vector, the gain control factor of the FIR filter group is dynamically adjusted with reference to the environmental noise energy. The FIR filter groups in different frequency bands are applied to estimate the reverberation time in different frequency bands and output the frequency-divided reverberation matrix.

[0032] In particular, in an embodiment of the present invention, the dynamic adjustment of the gain control factor of the FIR filter bank includes the following key operation steps: a. Based on the reverberation time parameter, a corresponding basic FIR filter bank template (FIR) is selected, and multiple filter bank schemes are preset for different reverberation levels. Conventional FIR filter bank designs are typically designed for fixed environments, not considering the dynamic changes in environmental reverberation. Therefore, once the environment changes, the performance of the original filter bank degrades. However, the present invention uses the reverberation time parameter as a classification basis to preset multiple basic FIR filter bank templates. For example, for short reverberation environments (RT60 < 0.3s), medium reverberation environments (0.3s ≤ RT60 < 0.7s), and long reverberation environments (RT60 ≥ 0.7s) (the above data is only an example and the specific settings will depend on the actual application). Different basic filter banks are designed for each of these environments. Each template is optimized based on its frequency response curve and amplitude response characteristics to adapt to the acoustic characteristics of different reverberation conditions. This basic template selection strategy ensures that the initial filter bank configuration is more closely aligned with the current actual environment, avoiding the reverberation estimation distortion caused by the one-size-fits-all use of a unified template, and improving the starting point of the filter bank's adaptability.

[0033] b. In order to further refine the performance of the filter bank under different environmental noise backgrounds, the present invention calculates the gain control factor based on the environmental noise energy and uses a nonlinear function to refine the overall gain of the filter bank.

[0034] It should be noted that traditional FIR filter design typically assumes stable or low background noise. However, in real live broadcast environments, background noise energy often fluctuates dramatically, for example, due to audience noise and equipment operation. These noise variations can interfere with reverberation estimation. Therefore, the present invention designs a mechanism for dynamically adjusting gain based on ambient noise energy. Specifically, based on the aforementioned frame segmentation, energy data for each frame is statistically analyzed. A low-energy segment is selected by setting a constant (e.g., 5%) to calculate the average power of the ambient noise. Subsequently, the gain control factor g is generated based on the ambient noise energy level and compared with a preset noise energy reference value. The calculation of g can employ nonlinear functions, such as logarithmic or exponential ones, and is not strictly limited here. For example, g = exp(α × (Pn - P0)), where Pn is the current noise energy, P0 is the reference noise energy, and α is the adjustment coefficient. By introducing a nonlinear function, the gain refinement process is sensitive to noise variations, avoiding the over- or under-gain issues associated with linear adjustment.

[0035] c. Multiply the gain parameters of each FIR channel by the corresponding gain control factor to obtain a dynamically adjusted filter bank. It can be seen that a channel refers to the sub-filters of the FIR filter bank divided for different frequency bands (such as low frequency, medium frequency, and high frequency), and each channel is responsible for processing the reverberation signal in a specific frequency range. In the present invention, the gain value of each channel not only inherits the design characteristics of the basic template, but also undergoes secondary adjustments based on the current noise environment. For example, in a high-noise environment, the gain factor of the medium- and high-frequency band channels can be appropriately lowered to suppress noise amplification; while in a low-noise environment, the gain is allowed to be appropriately increased to capture weak reverberation signals, thereby ensuring the dynamic range and accuracy of the frequency-divided reverberation estimation. Through this channel-by-channel and frequency-band gain adjustment strategy, the adaptability of the filter bank can be effectively improved, making the reverberation time estimation result more consistent with the actual sound field characteristics, greatly enhancing the reliability and stability of the system in complex and changing live broadcast environments.

[0036] In a further embodiment of the present invention, the dynamically adjusted FIR filter bank is applied to perform multi-band processing on the input audio stream, thereby completing the frequency band estimation of the reverberation time and finally outputting the frequency-divided reverberation matrix.

[0037] Preferably, the construction of the frequency division reverberation matrix includes: applying a time-frequency analysis method to the input audio stream to decompose it into multiple frequency band signals; inputting each frequency band signal into the corresponding adjusted FIR filter channel, performing independent filtering processing, and extracting the filtered frequency band signal; based on the envelope curve of the signal after filtering in each frequency band, analyzing the energy attenuation characteristics and calculating the reverberation time parameters of each frequency band respectively; and combining the reverberation time parameters calculated for each frequency band in the order of the frequency bands to form a complete frequency division reverberation matrix.

[0038] Specifically, the input audio stream is first decomposed into signals of different frequency bands using time-frequency analysis methods such as short-time Fourier transform (STFT) or wavelet transform. Each frequency band signal is then fed into the corresponding adjusted FIR channel for filtering. The filtered signal envelope curve is used to analyze the energy attenuation characteristics of each frequency band, calculate the reverberation time parameters for each frequency band, and ultimately combine them into a frequency-fraction reverberation matrix. Compared to traditional unified processing, the frequency-fraction estimation method of the present invention can reflect the differences in reverberation behavior across frequency bands, more accurately characterize the dynamic characteristics of the ambient sound field, and achieve more detailed environmental modeling and adaptive processing.

[0039] Preferably, the reverberation time parameters for each frequency band can be calculated using the following steps: Each filtered reverberation signal is obtained in each independent frequency band (e.g., divided into 1 / 3 octaves), forming a frequency-segmented reverberation data set; a sliding time window of fixed length is set for the reverberation signal in each frequency band. Within this window, the current local signal energy is calculated to produce an energy curve with time as the horizontal axis and local energy as the vertical axis. The sliding window is gradually slid across the entire signal to form a complete local energy time series curve; and the rate of decrease of the local energy is detected on this energy curve. Specifically: calculate the energy change rate between each sliding window (i.e., the local energy decline rate); mark the segments that decline continuously and whose decline rate exceeds the preset threshold as candidate energy attenuation segments; for all candidate energy attenuation segments, further screening is performed: the energy of the segment is required to decline continuously without intermediate energy rebound (i.e., the local rebound is less than the set small fluctuation tolerance threshold); the segment with the longest time span and the largest overall energy change amplitude is preferentially selected as the maximum attenuation segment; if there are multiple segments that meet the above criteria, the segment with high initial energy and low final energy is selected (to ensure the maximum dynamic range); within the determined maximum attenuation segment, the energy curve is preprocessed: first take the logarithm of the energy curve (usually in dB) to linearize the attenuation trend; then use the least squares method for linear fitting to obtain the slope of the energy change over time (i.e., the energy decay rate); sliding denoising (such as simple smoothing filtering) can be used in the fitting process to reduce the influence of local noise disturbances. Finally, based on the energy attenuation slope fitted from the maximum attenuation segment and the defined attenuation amplitude standard (usually with a 60dB energy drop as a reference), the reverberation time parameters for that frequency segment are calculated. If the actual recording environment does not allow a full 60dB drop, linear extrapolation can be performed based on the existing attenuation.

[0040] S3: Based on the frequency-division reverberation matrix and background spectral entropy characteristics, the input audio stream is subspace mapped to extract and retain the first nms direct sound component DS and suppress the reverberation tail part; where n is a constant.

[0041] In an embodiment of the present invention, the frequency division reverberation matrix and the background spectral entropy feature are first introduced as the basis for joint features to ensure that the input audio stream can achieve a more accurate separation effect of direct sound and reverberation tail in subsequent processing. The background spectral entropy feature is used to measure the spectral complexity of background noise or background sound signals. Generally, the higher the spectral entropy, the more uniform the spectrum distribution and the closer it is to white noise. The lower the spectral entropy, the higher the spectral energy concentration. By jointly modeling these two features of different dimensions, the dual effects of environmental reverberation and background noise can be fully taken into account, and the problem of insufficient separation effect caused by traditional single parameter modeling can be effectively compensated. In addition, compared with the method based on energy or time domain characteristics alone, the joint feature input adopted by the present invention can more finely reflect the sparsity difference between direct sound and reverberation sound on the frequency domain scale, enhance the accuracy and robustness of the separation process, and provide a more solid foundation for subsequent subspace mapping.

[0042] Specifically, the subspace mapping described in this step involves dividing the feature space into a region dominated by direct sound (i.e., the direct sound subspace) and a region dominated by reverberation tail (i.e., the reverberation sound subspace) according to certain rules. The signal components within each region are then extracted using a specific mapping operator. Preferably, performing subspace mapping on the input audio stream includes the following steps: A. Based on the frequency-division reverberation matrix and background spectral entropy characteristics, the signal sparsity index of each frequency band is calculated to evaluate the distribution density of direct sound and reverberation sound in the frequency domain.

[0043] The sparsity index refers to the degree of concentration of a signal's energy distribution within a certain frequency band. High sparsity indicates that the energy is concentrated in a few frequency components, while low sparsity indicates that the energy is dispersed. To accurately extract sparsity, the present invention extracts the corresponding reverberation time parameters in each frequency band based on the frequency-division reverberation matrix and, based on the background spectral entropy characteristics, jointly forms a local frequency eigenvector. For each local frequency eigenvector, a weighted method based on the coupling of local kurtosis and spectral entropy is used to calculate the sparsity index. The local frequency eigenvector comprehensively reflects the energy decay rate and spectral complexity of the frequency band and is a key input for measuring the distribution characteristics of direct sound and reverberation sound. Traditional methods typically use only reverberation time or only spectral entropy as a judgment basis, ignoring the interaction between the two in the actual environmental sound field, resulting in one-sided or unstable evaluation results. By jointly considering reverberation time and spectral entropy, the present invention can more realistically reflect the actual distribution characteristics of direct sound and reverberation sound in different frequency bands, thereby providing more discriminative reference data for subspace segmentation.

[0044] Furthermore, local kurtosis refers to the sharpness of the local energy peak of the signal. The larger the value, the higher the concentration of signal energy at a few frequency points, which is positively correlated with sparsity. The spectral entropy reflects the degree of energy dispersion. The lower the spectral entropy, the sparser the signal. Therefore, the present invention uses local kurtosis as a positive weighting factor and spectral entropy as a negative weighting factor to calculate the sparsity index through adaptive coupling weights. This weighting method not only makes full use of the complementary information of the two features, but also can adaptively adjust the respective weight ratios according to different environments, avoiding the overall mapping effect affected by the deviation of a single indicator. Compared with the conventional method of directly taking the average or single weighting, the method of the present invention can more finely distinguish the differences between the direct sound and reverberation sound components within the frequency band, thereby improving the accuracy of sparsity assessment.

[0045] B. According to the value range of the distribution density in each frequency band, the subspace segmentation threshold is adaptively set to determine the boundary between the direct sound subspace and the reverberation sound subspace.

[0046] Among them, the division of the boundary includes: generating a frequency threshold curve for the sparsity index of all frequency bands based on a preset nonlinear mapping function, which serves as a preliminary boundary reference for the direct phonon space and the reverberation phonon space; dividing the subspace belonging to each frequency band according to the relative position relationship between the frequency threshold curve and the sparsity index, and marking the direct phonon space and the reverberation phonon space.

[0047] The subspace segmentation threshold of the present invention refers to determining one or more segmentation points in the frequency domain to distinguish between areas dominated by direct sound and areas dominated by reverberation sound. Traditional methods mostly use fixed thresholds or empirical divisions, which leads to insufficient adaptability of the division results under different sound field conditions, and there are problems such as omission of direct sound or insufficient reverberation suppression. To address this shortcoming, in the present invention, by dynamically analyzing the signal sparsity of each frequency band and adjusting the threshold value in real time, the division process can adapt to changes in the current acoustic environment, ensuring that the direct sound component is retained to the greatest extent possible, while effectively suppressing the reverberation tail, thereby improving the clarity and intelligibility of the overall audio stream.

[0048] Exemplarily, in the specific operation process, the present invention first calculates the corresponding signal sparsity index for each predefined frequency band. If the sparsity value of a frequency band is high, it usually means that the direct sound component in the frequency band is dominant and the energy is concentrated; if the sparsity value is low, the reverberation component is dense and the energy distribution is relatively dispersed. After obtaining the sparsity indices of all frequency bands, the present invention maps these sparsity indices to corresponding frequency threshold values ​​according to a nonlinear mapping function to form a frequency threshold curve. This curve serves as a preliminary subspace segmentation reference line and can present a dynamic change curve rather than a fixed horizontal line according to the sparsity characteristics of each frequency band, thereby better fitting the changing characteristics of the actual acoustic environment. Furthermore, the present invention does not directly use the frequency threshold curve as the final dividing line, but instead divides all frequency bands in detail according to the relative position relationship between the frequency threshold curve and the sparsity index. Specifically, if the sparsity index of a frequency band is higher than the frequency threshold curve, the band is marked as the direct sound subspace (DS subspace), and the corresponding signal is preferentially retained. Conversely, if the sparsity index is lower than the threshold curve, it is marked as the reverberant sound subspace (R subspace), and the corresponding signal will be preferentially suppressed or eliminated by subsequent processing strategies. This partitioning strategy not only considers the absolute characteristics of each frequency band, but also comprehensively considers its relative position in the global frequency distribution, making the subspace partitioning results more detailed and resistant to noise interference, thus laying a solid foundation for subsequent subspace mapping and direct sound extraction.

[0049] It should be noted that the nonlinear mapping function may use a Sigmoid nonlinear function or an exponential function, etc., to achieve a clearer separation of direct sound and reverberation sound, and an adaptive and smooth threshold change, and the present invention does not make the sole limitation here.

[0050] C. Based on the subspace segmentation threshold, the orthogonal projection operator is used to perform subspace mapping processing on the input audio stream to extract the direct sound component DS and construct the reverberation residual component.

[0051] The orthogonal projection operator first defines the subspace basis of the direct sound and reverberant sound based on a sparsity index and threshold. It then constructs an orthogonal projection matrix for the direct sound subspace and applies it to the spectral representation of the input signal to achieve subspace separation. Compared to traditional, simple frequency masking or time window truncation methods, the orthogonal projection operator accurately extracts the target component while maintaining spectral coherence, effectively avoiding boundary effects and energy leakage, and significantly improving the purity and stability of direct sound extraction.

[0052] D. The direct sound component DS is reorganized by sliding the time window to form a direct sound output stream with enhanced continuity for subsequent secondary noise suppression.

[0053] As can be seen, the subspace mapping employed in this invention goes beyond simple threshold segmentation. It adaptively determines subspace boundaries based on the dynamic changes in energy, sparsity, and reverberation spread within the input signal's frequency band. This dynamic subspace partitioning effectively adapts to the variations in reverberation tail and direct sound components under varying recording environments and sound source conditions, significantly improving the accuracy and stability of direct sound extraction. It's also worth noting that the nms direct sound component DS mentioned in the operation refers to the primary direct sound energy component within a continuous timeframe of n milliseconds (n is a constant, typically 20ms or 30ms, adjustable depending on the scenario) from the direct sound's onset on the time axis. This limit of n milliseconds is because, generally, the primary energy of the direct sound is concentrated within a very short period after utterance, while the reverberation tail gradually thins out and diffuses. Therefore, extracting the direct sound within a fixed time window effectively eliminates reverberation interference while preserving the integrity of the primary signal characteristics.

[0054] S4: After the initial noise reduction, the time domain energy and frequency domain spectral entropy of DS are detected respectively. When the frequency domain spectral entropy is lower than the threshold And the time domain energy is higher than the threshold , the trigger mark is the residual noise mark.

[0055] It should be noted that after completing the subspace partitioning and preliminary noise reduction for the direct and reverberant sounds in step S3, some residual noise signals with complex energy characteristics may still exist. These signals typically manifest as short-term energy fluctuations or abnormal frequency domain distributions, necessitating further detection strategies to identify and label them. This step aims to accurately label potential residual noise signals using a joint detection mechanism based on time-domain energy characteristics and frequency-domain spectral entropy features, providing more reliable prior information for subsequent noise suppression or sound source reconstruction.

[0056] Specifically, after the input signal undergoes the noise reduction process in step S3, a preliminarily purified time domain signal sequence is obtained. The signal may still have several high-amplitude burst components in the time domain, and may exhibit spectral unevenness in the frequency domain. This time-frequency characteristic is widely considered to be a typical manifestation of non-stationary noise or pseudo-structured sound in engineering. Therefore, in this step, each signal segment is first short-term framed, and the corresponding time domain energy index is calculated for each frame. ,in, Indicates the The frame signal can be given by the following formula: in, For the Frame The signal value of the sampling point, is the length of each frame. It reflects the intensity characteristics of the signal in the time domain and helps to identify sudden high-energy events.

[0057] To compensate for possible misjudgments that may occur when relying solely on time-domain energy (for example, some speech signals also have high energy), the frequency-domain spectral entropy is calculated as a quantification of the signal's spectral complexity. This indicator reflects the uniformity and orderliness of the signal's frequency-domain power spectrum and is defined as follows: in, For the Frame signal The normalized power spectral density of the frequency components, is the total number of frequency components. If the spectrum energy is concentrated in a few frequency components, then The smaller the value, the more deterministic or orderly the spectrum structure of the frame signal is; on the contrary, if the spectrum is evenly distributed, then As it approaches the maximum, the signal spectrum becomes more chaotic or uncertain.

[0058] On this basis, the present invention designs a joint threshold judgment mechanism, setting two control thresholds: one is the spectrum entropy lower limit threshold , and the second is the upper threshold of time domain energy , when a frame signal meets the following two conditions: and , which means that the frame contains obvious unnatural components, which may be residual structural noise, pseudo-excitation sound or incompletely suppressed acoustic interference. Therefore, the frame signal is marked as a residual noise frame and is treated specially in subsequent processing.

[0059] To further reduce false alarm rates, the present invention incorporates a multi-frame continuity check mechanism. This means that a segment is marked as residual noise only when the aforementioned conditions are met for more than K consecutive frames (e.g., K = 3). This strategy leverages the short-lived nature of residual noise, enhancing robustness against occasional direct sound pulses or speech onsets.

[0060] S5: When the residual noise marker is triggered, an adaptive second-order filter is constructed based on DS, and local enhancement filtering is performed within the marked segment, and finally the optimized audio signal is output.

[0061] First, in order to establish the reference information required by the local filter, the DS components need to be reorganized by sliding the time window.

[0062] Preferably, step S5 includes the following steps: A. The direct sound component DS is reassembled by sliding in a time window to generate a direct sound output stream with enhanced continuity. When a residual noise mark is detected, the direct sound component in the corresponding segment is extracted to establish a local time domain reference vector.

[0063] B. Dynamically calculate the local energy spectrum density based on the local time domain reference vector, and adaptively set the center frequency and bandwidth of the second-order filter based on the local energy spectrum density to generate a local enhancement filter.

[0064] Preferably, the dynamic calculation of the local energy spectrum density includes: performing window function processing on the local time domain reference vector, using an exponential decay window to enhance the weight of the first end component of the segment, and outputting a weighted local vector; performing fast Fourier transform on the weighted local vector, extracting the corresponding frequency components, and generating a local frequency spectrum; based on the local frequency spectrum, accumulating power density by frequency band, and dynamically calculating the local energy spectrum density.

[0065] It should be noted that the local energy spectral density refers to the local power density distribution obtained by summing the frequency bands, which reflects the energy concentration of the DS component of the segment in each frequency band. In order to accurately calculate the local energy spectral density, it is first necessary to perform window function processing on the local time domain reference vector. In the present invention, an exponential decay window is particularly used, and the window function form is: ,in, Indicates the sampling point index, is the decay rate of the window function. The reason for introducing the exponential decay window is that, compared with conventional rectangular windows, Hamming windows, etc., the exponential decay window can effectively increase the weight of the components at the beginning of the segment, and the residual noise pollution is usually concentrated in the initial stage of the segment. Therefore, this weighting strategy helps to enhance the filter's sensitivity and response speed to initial noise pollution. In traditional operations, uniform weighting is often used, which results in the inability to capture the noise burst characteristics in time during filter design. The present invention uses the weighting method of the exponential decay window to enable the local energy spectral density to more accurately reflect the initial characteristics of noise pollution, thereby further improving the filter's adaptive performance.

[0066] After completing the window function processing, a fast Fourier transform is performed on it to extract its corresponding frequency components; through FFT transformation, the time-local signal can be projected into the frequency space to extract the power information of different frequency components; according to the frequency division standard, the spectrum is divided into several frequency bands with fixed bandwidth, and the power density of the corresponding frequency components is accumulated in each frequency band to form a banded local energy spectrum density curve.

[0067] Based on the above-mentioned local energy spectrum density results, the present invention further adaptively sets the parameters of the local second-order filter, including the center frequency and bandwidth. Among them, the center frequency is set to the main peak frequency of the local energy spectrum density curve, that is, the center of the frequency band with the largest energy density; the bandwidth is dynamically set according to the half-width of the main peak frequency band, which not only ensures the coverage of the contaminated frequency band, but also avoids the loss of direct sound components caused by the filter being too wide. Compared with the fixed parameter filter, this adaptive setting method can significantly improve the filtering accuracy and avoid signal attenuation or distortion caused by frequency band mismatch. Subsequently, according to the set center frequency and bandwidth parameters, a local enhancement filter is generated, usually in the form of a second-order bandpass filter; specifically, the transfer function can be expressed as: the Laplace variable is multiplied by the bandwidth parameter of the filter as the numerator; the denominator is composed of the sum of three terms, namely the square of the Laplace variable, the product of the Laplace variable and the bandwidth, and the square value of the center frequency multiplied by 2π. The transfer function value of the system can be obtained by calculation.

[0068] This filter can enhance or preserve signal components within a set frequency band, suppressing noise outside the band, achieving local optimization. Compared to traditional full-band filters, the local enhancement filter of this invention is more selective and targeted in its frequency domain response, maximizing the naturalness and clarity of DS components while effectively reducing local noise contamination.

[0069] C. Perform local enhancement filtering on the fragment corresponding to the residual noise mark according to the local enhancement filter and output the intermediate optimization result.

[0070] It is worth mentioning that the present invention adopts local processing rather than global processing in this process, which can significantly reduce the sound quality degradation and detail loss caused by global filtering, and ensure that the optimization processing is only performed within the necessary range, thereby protecting the original audio information to the greatest extent.

[0071] D. Recombining the intermediate optimization results with the original audio data without triggering markers on the timeline to output a continuous optimized audio signal. This recombining process ensures a seamless temporal connection between the optimization results and the original signal, avoiding temporal discontinuities, jumps, or distortion, further enhancing the naturalness and listening experience of the final audio output.

[0072] In summary, the live sound card noise reduction method based on environmental feature recognition based on the embodiment of the present invention is explained. By constructing a noise reduction process for environmental feature recognition, it can achieve dynamic adaptation and precise processing of multiple types of noise and reverberation factors in the live broadcast scene, thereby significantly improving the clarity and restoration of the voice signal. Especially in complex or drastically changing live broadcast environments, the present invention can perform intelligent identification based on multiple parameters such as reverberation time, environmental noise energy, and background spectral entropy characteristics to ensure that the noise reduction process is both targeted and real-time. Through a multi-stage processing strategy, the reverberation tail is effectively suppressed and the residual noise is adaptively identified and filtered out, thereby avoiding the common voice distortion or detail loss problems in traditional noise reduction methods.

[0073] This invention also offers significant advantages in the consistency and detail of noise reduction processing. It dynamically constructs local filters and performs fine frequency domain enhancement, making the final output audio signal more natural and continuous, significantly improving the listening experience. Compared to traditional solutions that rely solely on static filtering or fixed parameter models, this invention demonstrates greater environmental adaptability and speech fidelity.

[0074] In summary, the live broadcast sound card noise reduction method based on environmental feature recognition according to the embodiment of the present invention has been explained. This method determines whether the patient's puncture site has signs of infection by performing time-series analysis on high-definition images of the patient's puncture site at different time points within the target monitoring period. This can improve the accuracy and timeliness of infection monitoring, facilitate early detection of signs of infection, and thus enable timely measures to reduce the incidence and severity of infection.

Claims

1. A live broadcast sound card noise reduction method based on environmental feature recognition, characterized in that: include: Collect reverberation time parameters, ambient noise energy, and background spectral entropy characteristics in the live broadcast environment to construct an environmental feature vector; According to the reverberation time parameter in the environmental feature vector, the gain control factor of the FIR filter group is dynamically adjusted with reference to the environmental noise energy. The FIR filter groups in different frequency bands are applied to estimate the reverberation time in different frequency bands and output the frequency-divided reverberation matrix. Based on the frequency division reverberation matrix and background spectral entropy characteristics, the input audio stream is subspace mapped to extract and retain the first nms direct sound component DS and suppress the reverberation tail part; where n is a constant; After the initial noise reduction, the time domain energy and frequency domain spectral entropy of DS are detected respectively. When the frequency domain spectral entropy is lower than the threshold And the time domain energy is higher than the threshold When , the trigger mark is the residual noise mark; When the residual noise marker is triggered, an adaptive second-order filter is constructed based on DS, and local enhancement filtering is performed within the marked segment, and finally an optimized audio signal is output.

2. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 1, characterized in that: The reverberation time parameter is obtained by framing the input audio stream using the short-time energy window method, extracting the energy change curve of each frame, and estimating it based on the maximum attenuation segment fitting; The environmental noise energy is obtained by calculating the average power value of the time period in which the energy of each frame is lower than the average energy by m% on the basis of the same sub-frame, where m is a constant; the background spectral entropy feature is obtained by calculating the normalized amplitude probability distribution based on the amplitude spectrum of each frame.

3. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 2, wherein: The reverberation time parameters, environmental noise energy and background spectral entropy characteristics are weighted superposition to generate the environmental feature vector.

4. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 1, wherein: The dynamically adjusting gain control factor of the FIR filter bank includes: According to the reverberation time parameter, the corresponding basic FIR filter group template FIR is selected, and multiple filter group schemes are preset for different reverberation levels; based on the ambient noise energy, the gain control factor is calculated, and the overall gain of the filter group is refined using a nonlinear function form; the gain parameter of each FIR channel is multiplied by the corresponding gain control factor to obtain a dynamically adjusted filter group.

5. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 4, characterized in that: The construction of the frequency division reverberation matrix includes: applying a time-frequency analysis method to an input audio stream to decompose it into multiple frequency band signals; inputting each frequency band signal into a corresponding adjusted FIR filter channel, performing independent filtering processing, and extracting the filtered frequency band signal; analyzing the energy attenuation characteristics based on the envelope curve of the filtered signal of each frequency band, and calculating the reverberation time parameters of each frequency band respectively; and combining the calculated reverberation time parameters of each frequency band in the order of the frequency bands to form a complete frequency division reverberation matrix.

6. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 1, characterized in that: The subspace mapping of the input audio stream includes: calculating the signal sparsity index of each frequency band based on the frequency-division reverberation matrix and the background spectral entropy characteristics; determining the boundary between the direct sound subspace and the reverberation sound subspace according to the value range of the signal sparsity index in each frequency band; and using an orthogonal projection operator to perform subspace mapping processing on the input audio stream to extract the direct sound component DS.

7. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 6, characterized in that: The signal sparsity index of each frequency band is calculated as follows: in each frequency band, the corresponding reverberation time parameters are extracted based on the frequency division reverberation matrix, and a local frequency feature vector is jointly formed based on the background spectral entropy characteristics; for each local frequency feature vector, the sparsity index is calculated using a weighted method based on the coupling of local kurtosis and spectral entropy.

8. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 7, characterized in that: The division of the boundary includes: generating a frequency threshold curve based on a preset nonlinear mapping function for the sparsity index of all frequency bands as a preliminary boundary reference for the direct phonon space and the reverberation phonon space; dividing the subspace belonging to each frequency band according to the relative position relationship between the frequency threshold curve and the sparsity index, and marking the direct phonon space and the reverberation phonon space.

9. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 1, characterized in that: The direct sound component DS is reorganized by sliding the time window to generate a direct sound output stream with enhanced continuity. When a residual noise mark is detected, the direct sound component in the corresponding segment is extracted to establish a local time domain reference vector; the local energy spectral density is dynamically calculated based on the local time domain reference vector, and the center frequency and bandwidth of the second-order filter are adaptively set based on the local energy spectral density to generate a local enhancement filter; the segment corresponding to the residual noise mark is locally enhanced and filtered according to the local enhancement filter, and an intermediate optimization result is output; the intermediate optimization result and the original audio data without triggering the mark are reorganized on the time axis to output a continuous optimized audio signal.

10. The method for reducing noise of a live broadcast sound card based on environmental feature recognition according to claim 1, characterized in that: The dynamic calculation of local energy spectrum density includes: performing window function processing on the local time domain reference vector, using an exponential decay window to increase the weight of the first end component of the segment, and outputting a weighted local vector; performing fast Fourier transform on the weighted local vector, extracting the corresponding frequency components, and generating a local frequency spectrum; based on the local frequency spectrum, accumulating power density by frequency band, and dynamically calculating the local energy spectrum density.

Images