Intelligent dynamic speech enhancement method and system

Abstract

The inventive subject matter provides a method and system for intelligent dynamic speech enhancement of an audio source that includes performing speech detection and intelligent enhancement gain control on a multi-channel audio source input to determine a speech enhancement gain, and further includes applying the speech enhancement gain in dynamic loudness balancing performed on the multi-channel audio source input. Wherein the intelligent enhancement gain control includes setting a speech enhancement gain based on a signal power intensity ratio of a sum of the mid-channel signal and other channel signals, and setting the speech enhancement gain based on a system volume level.

Description

Technical Field

[0001] The inventive subject matter relates generally to the field of audio signal processing, and more particularly to methods and systems for intelligent dynamic speech enhancement of audio sources. Background Art

[0002] With the popularity of high-definition cable TV, online streaming, and large-screen display devices, home theater experience is becoming more and more popular in the market. These media sources are usually equipped with multi-channel audio to provide users with a more immersive surround sound experience. However, since the main purpose of movie formats is to provide highly immersive sound effects, the clarity of speech is often sacrificed in the pursuit of surround sound.

[0003] Speech enhancement technology plays a vital role in the filmmaking process and is designed to improve the quality, audibility, and clarity of dialogues in film footage.

[0004] The development of existing speech enhancement technology can be derived from surround sound technology. For example, surround sound providers such as Dolby, THX and DTS have brought multi-channel audio coding technology to achieve better spatial resolution and stereo sound experience. This technology can immerse the audience in a surround sound environment, but sometimes it causes the dialogue in the mix to be less clear. In addition, the mixing process is also included in film production. The mixer is responsible for adjusting the volume, balance and spatial positioning of different audio elements including dialogue, music and sound effects to create realistic auditory effects. However, in the pursuit of an immersive surround sound experience, the clarity of the dialogue is often affected.

[0005] Therefore, a method and system for intelligent dynamic speech enhancement for an audio source is needed to overcome the above-mentioned defects in existing solutions. Summary of the invention

[0006] On the one hand, the inventive subject matter provides a method for intelligent dynamic speech enhancement. The method for intelligent dynamic speech enhancement includes performing speech detection and intelligent enhancement gain control on a multi-channel audio source input to determine a speech enhancement gain, wherein the multi-channel audio source input includes a center channel signal and other channel signals. The method for intelligent dynamic speech enhancement also includes applying the speech enhancement gain to a dynamic loudness balance performed on the multi-channel audio source input. Wherein, the intelligent enhancement gain control includes setting the speech enhancement gain based on a signal power intensity ratio of the center channel signal to the sum of the other channel signals, and setting the speech enhancement gain based on a system volume level.

[0007] On the other hand, the inventive subject matter provides a system for intelligent dynamic speech enhancement, which includes a memory configured to store computer executable instructions and a processor configured to execute the computer executable instructions to implement a method for intelligent dynamic speech enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The inventive subject matter may be better understood by reading the following description of non-limiting embodiments with reference to the accompanying drawings, in which:

[0009] Figure 1 Schematically shows a block diagram of the intelligent speech enhancement principle according to one or more embodiments of the implementation scheme of the present subject matter;

[0010] Figure 2 A schematic block diagram illustrating the principle of speech detection according to one or more embodiments of the present subject matter is exemplified;

[0011] Figure 3 A schematic block diagram showing a principle of intelligently controlling speech enhancement gain based on speech detection according to one or more embodiments of the present subject matter is exemplified;

[0012] Figure 4 A graph exemplarily illustrating a nonlinear volume correlation function according to one or more embodiments of the present subject matter;

[0013] Figure 5 A schematic diagram illustrating a dynamic loudness balancing process according to one or more embodiments of the present subject matter;

[0014] Figure 6 A flow chart of a method for intelligent dynamic speech enhancement according to one or more embodiments of the present subject matter is schematically shown. DETAILED DESCRIPTION

[0015] It should be understood that the following description of the embodiments is given for illustrative purposes only and is not restrictive. The division of the examples in the functional blocks, modules or units shown in the drawings should not be interpreted as indicating that these functional blocks, modules or units must be implemented as physically separated units. The functional blocks, modules or units shown or described can be implemented as separate units, circuits, chips, functional blocks, modules or circuit elements. One or more functional blocks or units can also be implemented in a common circuit, chip, circuit element or unit.

[0016] The use of singular terms (such as, but not limited to, "a") is not intended to limit the number of items. The use of relational terms, such as, but not limited to, "top," "bottom," "left," "right," "upper," "lower," "downward," "upward," "side," "first," "second" ("third," etc.), "inlet," "outlet," etc., is used in the written description for clarity when specifically referring to the drawings and is not intended to limit the scope of the inventive subject matter or the appended claims unless otherwise noted. The terms "couple," "coupled," "coupled," "coupled," "coupler," and similar terms are used broadly herein and may include any method or device for fixing, bonding, adhering, fastening, attaching, associating, inserting, forming on or in, communicating with, or otherwise directly or indirectly mechanically, magnetically, electrically, chemically, operably associated with an intermediate element, one or more components, or may also include, but is not limited to, being integrally formed with another component in a unified manner. Coupling may occur in any direction, including rotationally. The terms "including" and "such as" are illustrative and non-limiting, and the word "may" means "may, but not necessarily" unless otherwise specified. Although any other language may be used in the present inventive subject matter, the embodiments shown in the drawings are examples given for purposes of illustration and explanation and are not the only embodiments of the subject matter herein.

[0017] In order to improve the quality of speech output and thus bring a better auditory experience to users, the inventive subject matter proposes a solution for actively detecting human speech based on detection confidence scores for audio sources (e.g., cinema audio sources) and intelligently and dynamically enhancing speech loudness.

[0018] One method of speech enhancement for improving the quality, audibility and clarity of dialogues in, for example, film productions utilizes static equalization techniques. This method can use a static equalizer, typically adjusting the frequency response in the range of 200Hz to 4kHz, to increase the volume and clarity of the dialogue range, thereby emphasizing the voice dialogue. However, the disadvantage of this method is that its processing of the audio is active throughout the audio clip, which can cause an unbalanced sound even in the absence of dialogue, and also amplify background noise. Another method of dynamic speech enhancement is to detect the speech signal in each time frame and apply adaptive audio processing based on the detection results. This method can enhance the sound when speech is detected to improve the clarity and intelligibility of the dialogue. However, this method requires a very accurate and fast speech detection algorithm to process the sound quickly.

[0019] However, when speech enhancement is continuously applied at a higher system volume, continuous speech enhancement has some side effects, such as it can cause perceptual imbalance and loss of dynamics in the auditory experience. In addition, in audio clips where only speech signals exist, the speech may already be clear and does not require further enhancement. Therefore, in order to obtain a reasonable audio processing and enhancement method to improve the quality of dialogues in movie products and allow the audience to better understand and appreciate the content of the movie, the disclosure of the inventive subject matter provides a method and system for further optimized and intelligent dynamic speech enhancement.

[0020] The inventive subject matter focuses on a multi-channel audio source input from which the speech and ambience channels in the audio source can be easily extracted and directly analyzed.

[0021] Figure 1 The block diagram 100 of the intelligent speech enhancement principle according to one or more embodiments of the implementation scheme of the inventive subject matter is schematically shown. For ease of understanding, the implementation of the inventive subject matter is mainly described with reference to several modules. It can be understood by those skilled in the art that the reference module description is intended to more clearly describe the present scheme, rather than for the purpose of limitation.

[0022] The audio source input that this system can process can include single channel source input, dual channel source input and multi-channel audio source input.In the example of home theater, usually the input of audio source can be considered as configured multi-channel audio source input.For example, in 5.1Dolby surround sound theater, audio source input can usually include a bass channel and five surround sound channels, and these five surround sound channels include center channel, left front channel, right front channel, left rear channel and right rear channel.In the case of multi-channel audio source input, usually most of the voice signals are present in the center channel, and other channels (i.e. left front channel, right front channel, left rear channel and right rear channel etc.) can be considered as including surrounding environment channels.Therefore in an example of the inventive subject matter, the voice detection process in the voice detection module has adopted a method for detecting the voice of the center channel.

[0023] like Figure 1 As shown, the multi-channel audio source signal first enters the intelligent dynamic speech enhancement system from the multi-channel audio source input module 102. The multi-channel audio source input can enter the speech detection module 104 for speech detection.

[0024] Figure 2 The schematic block diagram of the principle of speech detection according to one or more embodiments of the present subject matter is exemplarily shown. Figure 2 The speech detection process can be performed by Figure 1The speech detection module 104 in the embodiment of the present invention is executed. In the speech detection module 104, a center channel extraction is first performed for the multi-channel audio source input, and most of the speech signals exist in the center channel. Then, the extracted center channel signal is normalized so that the input signal is proportionally adjusted to a similar level. The normalized signal is, for example, expressed as the following equation:

[0025] x i_norm (n)=(x i (n)-μ i ) / σ i (1)

[0026] Among them, x i (n) represents the input signal of the nth sampling point in the i-th time frame, x i_norm (n) represents the output signal of the nth sampling point in the i-th time frame, that is, the normalized signal. i and σ i is the mean and variance of the input signal corresponding to the i-th time frame.

[0027] Next, a fast autocorrelation process is performed on the normalized signal and an autocorrelation result is output. For example, the fast autocorrelation process can first use a short-time Fourier transform (STFT) method to perform a Fourier transform on the normalized input signal, and perform a fast autocorrelation on the Fourier transformed signal. For example, the fast autocorrelation process is shown in the following equations (2)-(4).

[0028] X i (z) = STFT(x i_norm (n)) (2)

[0029]

[0030] C i =norm(c i (n)) (4)

[0031] Among them, X i (z) is the signal after Fourier transformation, Represents X i (z) is the conjugate, iSTFT is the inverse short-time Fourier transform, c i (n) is the autocorrelation of the signal in the i-th time frame. Next, i (n) Find the norm and get C i For example, based on the Euclidean norm, the final autocorrelation result output C is obtained. i The output of the autocorrelation result is C i represents the detection confidence, which may indicate the possibility of speech appearing in the center channel signal.

[0032] Next, in Figure 3 FIG. 1 is a block diagram showing the principle of intelligently controlling speech enhancement gain based on speech detection according to one or more embodiments of the present subject matter. This step of processing may correspond to Figure 1 The processing of the intelligent enhancement gain control module 106. As can be seen from the foregoing, the input of the intelligent enhancement gain control module is the detection confidence which is only the signal correlation result, which needs to be converted into the enhancement gain of the speech.

[0033] The dynamic range of the enhancement gain can be defined as follows:

[0034] G i =D 0 *ln(C i +D 1 ) (5)

[0035] Among them, G i represents the gain of the dynamic control module, C i Represents the aforementioned detection confidence, D 0 and D 1 is a control parameter of the dynamic gain fluctuation range, which can be a real number greater than zero; ln(·) is a natural logarithmic function.

[0036] In some examples, it is also necessary to enhance the gain G i Further processing can be performed to reduce audio distortion. For example, G i Perform smoothing.

[0037] In one or more embodiments of the inventive subject matter, for the gain G i When speech enhancement is continuously applied at a high system volume, the continuous enhancement of speech may cause side effects in the auditory experience, such as perceptual imbalance and power loss. In addition, in a scenario where only speech signals exist, there is no need to further apply gain processing to enhance the speech signals for these audio clips where the speech is already clear. Therefore, the inventive subject matter also provides intelligent logic judgment and intelligent enhancement gain processing to intelligently optimize the speech enhancement gain processing for these proposed side effects.

[0038] In one example, the intelligent logic judgment determines the setting of the speech enhancement gain by comparing the signal power strength of the center channel and other channels. Specifically, in one or more embodiments of the inventive subject matter, the energy of the other channel signals is first extracted and summed, and then compared with the energy of the center channel. For example, when there is a speech signal in the center channel and the surrounding sound of other channels is loud, the speech enhancement gain needs to be set higher; and when there is a speech signal in the center channel but the surrounding sound of other channels is weak, the speech enhancement gain needs to be set lower. As expressed in the following equation:

[0039]

[0040] Among them, G p is the gain of the intelligent voice enhancement control module, is the power ratio of the center channel to other channels, P c is the power of the center channel, P o is the sum of the powers of other channels, and α is a limiting parameter according to system configuration.

[0041] That is to say, when the signal power strength of the sum of the mid channel and other channels is relatively small, the speech enhancement gain needs to be set higher; and when the signal power strength of the sum of the mid channel and other channels is relatively large, the speech enhancement gain can be set lower.

[0042] By making logical judgments on the signal power intensity ratio of the center channel and the sum of other channels, and based on the relative intensity of the human voice dialogue in the audio clip compared to the ambient sound, the problem of perceived imbalance when enhancing audio sound can be solved. In the case where the audibility and clarity are deteriorated due to strong ambient sound and insufficient human voice, the gain of the human voice dialogue can be further intelligently optimized, allowing users to have better auditory perception.

[0043] In one example, in intelligent logic judgment, the setting of the speech enhancement gain is determined by the system volume of the audio source of a movie product, for example. The volume level of the current system can be identified, and then different speech enhancement gains can be set within the identified different volume ranges. For example, the volume level of the current system is identified. When the system volume level is within a lower range, the speech enhancement gain can be set higher; and when the system volume level is within a higher range, the speech enhancement gain should be set lower. For example, when the volume level of the current system exceeds a threshold, the speech enhancement gain can be set to 1, so that the loudness level of the speech signal remains unchanged.

[0044] Figure 4A graph of a nonlinear volume correlation function according to one or more embodiments of the present subject matter is exemplarily shown. The volume range of the speech enhancement gain can be defined as follows:

[0045] G v =f V (G i *G p ) (7)

[0046] Among them, G v represents the output of the intelligent voice enhancement control module, f V is a nonlinear volume-related function; as mentioned above, G i represents the gain of the dynamic control module, G p is the gain of the intelligent voice enhancement control module. The curve diagram of the nonlinear volume correlation function can be, for example, Figure 4 As shown in the figure, the horizontal axis represents the system volume level and the vertical axis represents the gain setting. Figure 4 It can be seen that within the range of relatively low system volume, for example, when the system volume level is less than 20, the speech enhancement gain can be set higher, for example, the gain is 6dB. As the system volume increases, when it reaches a relatively high range of the system volume, the setting of the speech enhancement gain is reduced. For example, after the system volume level exceeds 20, the curve of the nonlinear volume-related function of the speech enhancement gain drops to 4dB, and when the system volume level reaches 24, the speech enhancement gain is 3dB. When the system volume level further increases to 26, the gain setting for speech gradually drops to 1, that is, the system maintains the speech volume without enhancing it, so as to avoid the impact of audio distortion on hearing caused by excessive volume.

[0047] Finally, a soft limiter can be used for limiting. For example, the tanh function can be used as a soft limiter to ensure the enhanced gain after limiting. Within a reasonable amplitude range. The soft limiter process can be expressed as:

[0048]

[0049] Among them, α, β, and γ are limiter parameters according to the system configuration. These parameters depend on the system configuration. α can be a real number greater than zero, and β and γ can be non-zero real numbers. At this time, G vlim It is the result of intelligent enhancement gain processing and can be used as the output of the intelligent enhancement gain control module.

[0050] The above-mentioned method, by identifying the system volume level of the film product and performing logical judgment based on the intensity of the system volume in the audio clip, can solve the problem of audio distortion caused by excessive enhancement of voice dialogue when the volume is enhanced to a higher intensity, thereby further intelligently optimizing the dynamic voice gain and enabling users to obtain better auditory perception.

[0051] Through the above logical judgment of the signal power intensity ratio of the center channel and the sum of other channels and the system volume level, the processing of dynamic voice enhancement can be intelligently optimized to further improve the user experience, thereby intelligently adjusting the voice enhancement in various scenes in the diverse film product sources. Therefore, using this method can undoubtedly enable users to obtain a better viewing experience.

[0052] The intelligent dynamic processing provided in the subject matter of the present invention is all aimed at speech enhancement. Since the frequency range of human voice is basically in the intermediate frequency range, such as between 250-4000Hz, the dynamic loudness balance module 108 in the subject matter of the present invention can also focus on the intermediate frequency range of the input audio source signal for processing. Therefore, a frequency division filter can be selected to divide the input audio signal first, for example, Figure 1 In the process, the input audio signal enters the frequency division filtering module 110 for frequency division filtering to distinguish signals in different frequency ranges such as low-frequency signals, intermediate-frequency signals and high-frequency signals, and then the separated intermediate-frequency signals can enter the dynamic loudness balance module 108 for dynamic loudness balance processing. Moreover, the intelligent enhanced gain control module is also only applied to the voice signal in the intermediate frequency range in the multi-channel audio source input to output the enhanced gain, while the signals in other frequency ranges in the input signal can remain unchanged. Through the frequency division filtering processing, the intermediate frequency range signal and other high-frequency and low-frequency range signals in the multi-channel audio source input can be processed separately to avoid audio distortion in the non-voice frequency range as much as possible.

[0053] Figure 5A schematic diagram of a dynamic loudness balancing process according to one or more embodiments of the inventive subject matter is exemplarily shown. Optionally, the signal of the multi-channel audio source input after frequency division filtering can be divided into signals in the low frequency, medium frequency and high frequency ranges. The dynamic loudness balancing module can perform dynamic loudness balancing only on the signal in the medium frequency range. Among them, for the center channel signal extracted by channel extraction, the loudness of the center channel signal can be enhanced based on the intelligent enhancement gain, for example, while the loudness of other channel signals can be weakened optionally. For example, based on the intelligent enhancement gain, the signal in the center channel and the signals in other channels can be enhanced and / or weakened respectively at different ratios, and then concatenated and mixed to generate a signal output. In addition, the signals in the low frequency range and the high frequency range after the frequency division filtering in the multi-channel audio source input will not be subjected to dynamic loudness balancing processing here, but will be directly concatenated and mixed with the signal in the medium frequency range after dynamic loudness balancing to generate a signal output. For example, in the case of Figure 1 The signal output module 112 shown in FIG outputs a signal. Thus, the decrease in audibility or clarity caused by non-speech signals can be better avoided.

[0054] return Figure 1 It can be seen that the method of intelligent dynamic speech enhancement in the inventive subject matter is mainly divided into two paths for processing, and the upper path includes audio source input, speech detection and intelligent enhancement gain, and this path is regarded as a side chain processing flow for performing detection. The lower path includes audio source input, frequency division filtering and dynamic loudness balance to form the main chain processing flow. The upper path and the lower path can be performed synchronously or asynchronously, which will be determined by the capabilities and delay requirements of the actual system. This method of performing intelligent dynamic speech enhancement in a two-way manner can minimize delay and prevent audio distortion. When the two-layer path works asynchronously, the signal can pass through the entire system at a very fast speed with almost no delay. On the other hand, estimating the enhancement gain at a relatively low rate can have higher accuracy and smoothness, which significantly helps to prevent audio distortion.

[0055] Figure 6 Schematically illustrates a flow chart 600 of a method for intelligent dynamic speech enhancement according to one or more embodiments of the present subject matter. Figure 6As shown, the method for intelligent dynamic speech enhancement may include receiving a multi-channel audio source input in step S610. Then, in step S620, speech detection is performed on the multi-channel audio source input of the side chain processing flow path. Next, in step S630, intelligent enhancement gain control is performed on the detected speech to determine the speech enhancement gain. In step S640, the determined speech enhancement gain is used in the main chain processing flow to perform dynamic loudness balancing on the multi-channel audio source input, and finally, in step S650, the method for intelligent dynamic speech enhancement can provide an audio signal output of intelligent speech enhancement.

[0056] Additionally or alternatively, such as Figure 6 The steps of the method for intelligent dynamic speech enhancement shown can be implemented by one or more processors. The processor can be implemented as a microprocessor, a microcontroller, an application-specific integrated circuit (ASIC), a digital signal processor DSP, discrete logic, or a combination of these and / or other types of circuits or logic. Similarly, the implementation of Figure 6 The instructions of the method for intelligent dynamic speech enhancement shown can be stored in a memory, which can be DRAM, SRAM, flash memory, or other types of memory. Parameters (e.g., conditions and thresholds) and other data structures can be stored and managed separately, can be combined into a single memory or database, or can be logically and physically organized in many different ways. Programs and instruction sets can be parts of a single program, can be separate programs, or can be distributed across multiple memories and processors.

[0057] The method and system of intelligent dynamic voice enhancement provided by the inventive subject matter can be used not only for consumer products such as soundbars and stereo speakers, but also for applications in places such as theaters and concert halls. Compared with static audio equalization and static voice enhancement technologies, the method and system of intelligent dynamic voice enhancement provided by the inventive subject matter can improve voice clarity through intelligent gain control. In the method of intelligent dynamic voice enhancement provided by the inventive subject matter, existing voice enhancement technologies are optimized, so that dynamic voice enhancement gain control is intelligently added to applications such as theaters, and the user experience is improved for a variety of movie clips, including but not limited to: rapid implementation of voice activity detection based on the center channel; intelligent enhancement gain control that intelligently adjusts the voice enhancement gain level according to the movie content and the volume of the playback system; and multi-channel intelligent dynamic loudness balance is achieved using dual processing paths.

[0058] Examples of one or more implementations of the inventive subject matter are described in the following clauses:

[0059] Clause 1. Performing speech detection and intelligent enhancement gain control on a multi-channel audio source input to determine a speech enhancement gain, the multi-channel audio source input including a center channel signal and other channel signals;

[0060] The speech enhancement gain is applied to a dynamic loudness balance performed on the multi-channel audio source input, wherein the intelligent enhancement gain control comprises:

[0061] Setting the speech enhancement gain based on the signal power strength ratio of the center channel to the sum of the other channels, and

[0062] Sets the speech enhancement gain based on the system volume level.

[0063] Clause 2. The method for intelligent dynamic speech enhancement as described in Clause 1, wherein setting the speech enhancement gain based on the signal power strength ratio of the center channel to the sum of the other channels comprises:

[0064] When the signal power strength ratio of the mid channel to the sum of the other channels is small, setting the speech enhancement gain higher; and

[0065] When the signal power strength ratio of the mid channel to the sum of the other channels is large, the gain of the speech enhancement is set to be low.

[0066] Clause 3. The method of intelligent dynamic speech enhancement as described in clause 1 or 2, wherein setting the speech enhancement gain based on the system volume level comprises:

[0067] Identify the volume level of the system, and set different speech enhancement gains when the identified system volume level is within different volume ranges.

[0068] When the system volume level is in a lower range, the speech enhancement gain is set higher, and

[0069] When the system volume level is in a higher range, the speech enhancement gain is set lower.

[0070] Clause 4. A method for intelligent dynamic speech enhancement as described in any one of clauses 1 to 3, wherein speech detection comprises:

[0071] Extracting center channel signal from multi-channel audio source input;

[0072] performing normalization on the center channel signal; and

[0073] A fast autocorrelation is performed on the normalized center channel signal, and the result of the fast autocorrelation represents the detection confidence, which indicates the possibility of speech appearing in the center channel signal.

[0074] Clause 5. The method for intelligent dynamic speech enhancement as described in any one of clauses 1 to 4, wherein the intelligent enhancement gain control further comprises:

[0075] converting the detection confidence into a speech enhancement gain; and

[0076] Smoothing of speech enhancement gain.

[0077] Clause 6. A method for intelligent dynamic speech enhancement as described in any one of clauses 1 to 5, wherein the intelligent enhancement gain control further comprises performing soft limiting processing on the set speech enhancement gain.

[0078] Clause 7. A method of intelligent dynamic speech enhancement as described in any one of clauses 1 to 6, wherein the dynamic loudness balancing performed on the multi-channel audio source input comprises:

[0079] enhancing the loudness of the center channel signal and reducing the loudness of other channel signals based on the set speech enhancement gain; and

[0080] The enhanced center channel signal and the attenuated other channel signals are combined and mixed to generate an output signal.

[0081] Clause 8. The method of intelligent dynamic speech enhancement as described in any one of Clauses 1 to 7 further includes: performing frequency division filtering processing on the multi-channel audio source input before performing dynamic loudness balancing on the multi-channel audio source input.

[0082] Clause 9. The method of intelligent dynamic speech enhancement as described in any one of clauses 1 to 8, further comprising:

[0083] Performing dynamic loudness balancing on multi-channel audio source inputs in the mid-frequency range only; and

[0084] A multi-channel audio source input of a mid-frequency range after dynamic loudness balance is concatenated and mixed with multi-channel audio source inputs of a low-frequency range and a high-frequency range to generate an output signal.

[0085] Clause 10. A system for intelligent dynamic speech enhancement, comprising:

[0086] a memory configured to store computer-executable instructions; and

[0087] One or more processors configured to execute stored computer executable instructions to implement the method of intelligent dynamic speech enhancement described in any of clauses 1-9.

[0088] The description of the embodiment has been presented for the purpose of illustration and description. According to the above description, the embodiment can be appropriately modified and changed, or these modifications and changes can be obtained from practicing the method. For example, unless otherwise specified, one or more of the described methods can be performed by suitable devices and / or device combinations. The method can be performed by performing the stored instructions in combination with one or more logic devices (for example, processors) in combination with one or more other hardware elements (such as storage devices, memories, hardware network interfaces / antennas, switches, actuators, clock circuits, etc.). In addition to the sequence described in the application, the described method and the associated actions can also be performed in parallel and / or simultaneously in various orders. The described system is exemplary in nature and may include additional elements and / or omit elements. The subject matter of the inventive subject includes all novel and non-obvious combinations of the disclosed various systems and configurations and other features, functions and / or properties.

[0089] Elements of various real-time solutions for implementing the modules, elements and components of the methods provided by the inventive subject matter can be manufactured as one or more electronic devices residing on the same chip or in a chipset, including but not limited to fixed or programmable logic element (e.g., transistors or gates, etc.) arrays. One or more elements of various embodiments of the apparatus described herein can also be implemented in whole or in part as one or more instruction sets, which can be arranged to execute on one or more fixed or programmable logic element arrays (e.g., microprocessors, embedded processors, IP cores, digital signal processors, FPGAs, ASSPs, and ASICs, etc.).

[0090] The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

[0091] In the above, reference is made to the embodiments presented in the inventive subject matter. However, the scope of the inventive subject matter is not limited to the specifically described embodiments. On the contrary, any combination of the aforementioned features and elements, whether or not related to different embodiments, is contemplated as implementing and practicing the contemplated embodiments.

[0092] In addition, although the embodiments disclosed herein may achieve advantages over other possible solutions or over the prior art, whether or not a given embodiment achieves a particular advantage does not limit the scope of the inventive subject matter. Therefore, the foregoing aspects, features, embodiments and advantages are merely illustrative and are not considered elements or limitations of the appended claims unless expressly recited in the claims.

[0093] While the foregoing is directed to embodiments of the inventive subject matter, other and further embodiments of the inventive subject matter may be devised without departing from the basic scope of the inventive subject matter, and the scope of the inventive subject matter is determined by the claims that follow.

Claims

1. A method for intelligent dynamic speech enhancement, comprising the following steps: Performing speech detection and intelligent enhancement gain control on a multi-channel audio source input to determine a speech enhancement gain, wherein the multi-channel audio source input includes a signal of a center channel and signals of other channels; applying the speech enhancement gain to a dynamic loudness balancing performed on the multi-channel audio source input, Wherein, the intelligent enhanced gain control includes: setting the speech enhancement gain based on a signal power strength ratio of the center channel to the sum of the other channels, and The speech enhancement gain is set based on a system volume level.

2. The method of claim 1, wherein: Setting the speech enhancement gain based on the signal power strength ratio of the center channel to the sum of the other channels includes: When the signal power strength ratio of the mid channel to the sum of the other channels is small, setting the speech enhancement gain higher; and When the signal power strength ratio of the mid channel to the sum of the other channels is large, the gain of the speech enhancement is set to be low.

3. The method of claim 1, wherein: Setting the speech enhancement gain based on the system volume level includes: Identify the volume level of the system, and set different speech enhancement gains when the identified system volume level is within different volume ranges. Wherein, when the system volume level is in a lower range, the speech enhancement gain is set higher, and When the system volume level is within a higher range, the speech enhancement gain is set lower.

4. The method of claim 1, wherein: The speech detection comprises: Extracting center channel signal from multi-channel audio source input; performing normalization on the center channel signal; and A fast autocorrelation is performed on the normalized center channel signal, and a result of the fast autocorrelation represents a detection confidence, which indicates the possibility of speech appearing in the center channel signal.

5. The method of claim 4, wherein: The intelligent enhanced gain control further includes: Converting the detection confidence into a speech enhancement gain; and The speech enhancement gain is smoothed.

6. The method of claim 1, wherein: The intelligent enhancement gain control also includes performing soft limiting processing on the set speech enhancement gain.

7. The method according to claim 2 or 3, wherein: The dynamic loudness balancing performed on the multi-channel audio source input comprises: enhancing the loudness of the center channel signal and reducing the loudness of other channel signals based on the set speech enhancement gain; and The enhanced center channel signal and the attenuated other channel signals are combined and mixed to generate an output signal.

8. The method of claim 7, further comprising: Prior to the dynamic loudness balancing performed on the multi-channel audio source input, a frequency division filtering process is performed on the multi-channel audio source input.

9. The method of claim 8, further comprising: performing dynamic loudness balancing on the multi-channel audio source input only in the mid-frequency range; as well as The multi-channel audio source input in the middle frequency range after dynamic loudness balancing is coupled and mixed with the multi-channel audio source input in the low frequency range and the high frequency range to generate an output signal.

10. A system for intelligent dynamic speech enhancement, comprising: a memory configured to store computer executable instructions; as well as One or more processors are configured to execute the computer executable instructions to implement the method for intelligent dynamic speech enhancement according to any one of claims 1-9.

Images