Microphone volume control method, device, equipment and storage medium

Abstract

This application provides a microphone volume control method, apparatus, device, and storage medium. The method includes: performing voice detection on a voice signal acquired by a microphone to obtain at least two environmental indicators corresponding to the voice signal; performing voice signal preprocessing on the voice signal using a voice signal processing method corresponding to each environmental indicator to obtain at least two voice signal streams; extracting parameter features from each voice signal stream to obtain at least two parameter state streams of the voice signal; determining the digital gain adjustment amount and analog gain adjustment amount of the microphone based on the at least two parameter state streams; and adjusting the digital gain and analog gain of the microphone accordingly. This application enables flexible and adaptive adjustment of the microphone's digital and analog gain based on the environmental indicators of the microphone's current environment, resulting in smoother microphone volume and improved user experience.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and are related to but not limited to a microphone volume control method, apparatus, device and storage medium. Background Art

[0002] In the current automatic control method of microphone gain, it is usually based on a preset fixed automatic gain control algorithm to adjust the digital gain and analog gain of the microphone to achieve microphone volume control. The method in the related art does not take into account the multiple indicators that may affect the microphone volume, and cannot flexibly adjust the digital gain and model gain of the microphone according to the current environment of the microphone. Therefore, when adjusting the microphone volume, the method in the related art will show the problem of the volume being sometimes loud and sometimes soft, which greatly reduces the user experience. Summary of the invention

[0003] The embodiments of the present application provide a microphone volume control method, apparatus, device and storage medium. Since the digital gain and model gain of the microphone are flexibly adjusted adaptively according to the environmental indicators of the environment in which the microphone is currently located, the volume of the microphone after adjustment is smoother, thereby improving the user experience.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for controlling microphone volume, the method comprising:

[0006] Performing voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal;

[0007] Using a speech signal processing method corresponding to each of the environmental indicators, respectively preprocess the speech signals to obtain at least two speech signal streams;

[0008] Performing parameter feature extraction on each speech signal stream to obtain at least two parameter state streams of the speech signal;

[0009] Determining a digital gain adjustment amount and an analog gain adjustment amount of the microphone respectively according to the at least two parameter state streams;

[0010] According to the digital gain adjustment amount and the analog gain adjustment amount, the digital gain and the analog gain of the microphone are adjusted accordingly to achieve volume control of the microphone.

[0011] The present application provides a microphone volume control device, the device comprising:

[0012] A voice detection module, used to perform voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal;

[0013] A preprocessing module, configured to perform preprocessing on the voice signal in accordance with the voice signal processing method corresponding to each of the environmental indicators, and obtain at least two voice signal streams;

[0014] A parameter feature extraction module, used to extract parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal;

[0015] A determination module, configured to respectively determine a digital gain adjustment amount and an analog gain adjustment amount of the microphone according to the at least two parameter state streams;

[0016] The adjustment module is used to adjust the digital gain and the analog gain of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount, so as to achieve volume control of the microphone.

[0017] An embodiment of the present application provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor is used to execute the computer instructions to implement the above-mentioned microphone volume control method.

[0018] An embodiment of the present application provides a microphone volume control device, comprising: a memory, used to store executable instructions; and a processor, used to implement the above-mentioned microphone volume control method when executing the executable instructions stored in the memory.

[0019] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned microphone volume control method.

[0020] The embodiments of the present application have the following beneficial effects: performing voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal; adopting the voice signal processing method corresponding to each environmental indicator, performing different types of voice signal preprocessing on the voice signal to obtain at least two voice signal streams, and each type of voice signal processing method corresponds to an environmental indicator, and then determining the digital gain adjustment amount and the analog gain adjustment amount of the microphone based on the at least two voice signal streams obtained, and realizing the volume control of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount, so that the digital gain and model gain of the microphone can be flexibly and adaptively adjusted according to the environmental indicators of the current environment of the microphone, so that the microphone volume is smoother and the user experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic diagram of an optional architecture of a microphone volume control system provided in an embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of the structure of the server provided in the embodiment of the present application;

[0023] Figure 3 This is an optional flowchart of the microphone volume control method provided in an embodiment of the present application;

[0024] Figure 4 This is an optional flowchart of the microphone volume control method provided in an embodiment of the present application;

[0025] Figure 5 This is an optional flowchart of the microphone volume control method provided in an embodiment of the present application;

[0026] Figure 6 This is an optional flowchart of the microphone volume control method provided in an embodiment of the present application;

[0027] Figure 7 It is an application scenario diagram of the method of the embodiment of the present application;

[0028] Figure 8 is an architecture diagram of a microphone automatic gain control method provided in an embodiment of the present application;

[0029] Fig. 9 This is a diagram of the voice feature pool architecture provided in an embodiment of the present application;

[0030] Fig.10 is an architecture diagram of a gain control module provided in an embodiment of the present application;

[0031] Fig.11 is a schematic diagram of a two-level volume control provided in an embodiment of the present application;

[0032] Fig.12 yes Fig.11 Two-level volume control abstract transformation illustration. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0034] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0035] Among the current automatic control methods for microphone gain, one implementation method is to use voice activity detection (VAD) technology and a preset automatic gain control algorithm to adjust the digital gain and analog gain of the microphone. However, traditional VAD and fundamental frequency indicators are not comprehensively used to control the gain calculation. Another implementation method is to adjust the microphone volume and microphone enhancement according to any adjusted analog automatic gain control (AGC) gain value or digital AGC gain value during a call. If the analog AGC gain value or the digital AGC gain value is positive, first determine whether only increasing the microphone volume can meet the positive gain requirement. If so, adjust the microphone volume. If not, determine whether the microphone enhancement can be adjusted. If so, adjust the microphone volume and microphone enhancement at the same time. If the microphone volume and microphone enhancement have been adjusted to the maximum value, no adjustment is made. If the analog AGC gain value or the digital AGC gain value is negative, first determine whether only lowering the microphone volume can meet the negative gain requirement. If so, adjust the microphone volume. If not, determine whether the microphone enhancement can be adjusted. If so, adjust the microphone volume and microphone enhancement at the same time. If the microphone volume and microphone enhancement have been adjusted to the minimum value, no adjustment is made. It can be seen that this technology is an automatic control process of microphone volume and microphone enhancement.

[0036] However, the methods in the related art do not take into account the multiple environmental indicators that may affect the microphone volume, and cannot flexibly adjust the digital gain and model gain of the microphone according to the current environment of the microphone. Therefore, when adjusting the microphone volume, the methods in the related art will show the problem of the volume being sometimes loud and sometimes soft, thereby greatly reducing the user experience.

[0037] Based on the above problems existing in the related art, the microphone volume control method provided by the embodiment of the present application, first, performs voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal; and adopts the voice signal processing method corresponding to each of the environmental indicators to perform voice signal preprocessing on the voice signal collected by the microphone respectively, and obtains at least two voice signal streams correspondingly; then, parameter feature extraction is performed on each voice signal stream to obtain at least two parameter state streams of the voice signal; according to the at least two parameter state streams, the digital gain adjustment amount and the analog gain adjustment amount of the microphone are determined respectively; finally, according to the digital gain adjustment amount and the analog gain adjustment amount, the digital gain and the analog gain of the microphone are adjusted accordingly to realize the volume control of the microphone. In this way, since the possible impact of multiple environmental indicators on the microphone volume is taken into account, the digital gain and the model gain of the microphone can be flexibly and adaptively adjusted according to the environmental indicators of the current environment of the microphone, so that the microphone volume is smoother and the user experience is improved.

[0038] The following describes an exemplary application of the microphone volume control device of the embodiment of the present application. In one implementation, the microphone volume control device provided by the embodiment of the present application can be implemented as a laptop, tablet computer, desktop computer, mobile device (e.g., mobile phone, portable music player, personal digital assistant, dedicated messaging device, portable gaming device), intelligent robot, or any other terminal with voice collection and recording functions. In another implementation, the microphone volume control device provided by the embodiment of the present application can also be implemented as a server. The following describes an exemplary application of the microphone volume control device when it is implemented as a server.

[0039] See also Figure 1 , Figure 1It is an optional architecture diagram of the microphone volume control system 10 provided in the embodiment of the present application. In order to realize the volume control of the microphone, the microphone volume control system 10 provided in the embodiment of the present application includes a terminal 100, a network 200 and a server 300. The terminal 100 has a microphone 100-1, and the microphone collects voice to obtain a voice signal. The terminal 100 sends the collected voice signal to the server 300 through the network 200. The server 300 performs voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal, and adopts a voice signal processing method corresponding to each of the environmental indicators to perform different types of voice signal preprocessing on the voice signal collected by the microphone, and obtain at least two voice signal streams; wherein each type of voice signal processing method corresponds to an environmental indicator; parameter feature extraction is performed on each voice signal stream to obtain at least two parameter state streams of the voice signal; according to the at least two parameter state streams, the digital gain adjustment amount and the analog gain adjustment amount of the microphone are determined respectively. After obtaining the digital gain adjustment amount and the analog gain adjustment amount, the server 300 can adjust the digital gain and analog gain of the microphone accordingly according to the digital gain adjustment amount and the analog gain adjustment amount to achieve volume control of the microphone. The digital gain adjustment amount and the analog gain adjustment amount can also be sent to the terminal 100 so that the terminal 100 adjusts the digital gain and analog gain of the microphone accordingly according to the digital gain adjustment amount and the analog gain adjustment amount to achieve volume control of the microphone.

[0040] Figure 2 is a schematic diagram of the structure of the server 300 provided in the embodiment of the present application, Figure 2 The server 300 shown includes: at least one processor 310, a memory 350, at least one network interface 320 and a user interface 330. The various components in the server 300 are coupled together via a bus system 340. It is understood that the bus system 340 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 340 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 340 is not described in detail. Figure 2 Various buses are labeled as bus system 340 .

[0041] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0042] The user interface 330 includes one or more output devices 331 that enable presentation of media content, including one or more speakers and / or one or more visual display screens. The user interface 330 also includes one or more input devices 332, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0043] The memory 350 may be removable, non-removable or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drive, optical disk drive, etc. The memory 350 may optionally include one or more storage devices physically away from the processor 310. The memory 350 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 350 described in the embodiment of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 can store data to support various operations, and examples of these data include programs, modules, and data structures or subsets or supersets thereof, as exemplarily described below.

[0044] Operating system 351, including system programs for processing various basic system services and performing hardware-related tasks, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0045] A network communication module 352, for reaching other computing devices via one or more (wired or wireless) network interfaces 320, exemplary network interfaces 320 include: Bluetooth, Wireless Compatibility Authentication (WiFi), and Universal Serial Bus (USB);

[0046] The input processing module 353 is used to detect one or more user inputs or interactions from one of the one or more input devices 332 and translate the detected inputs or interactions.

[0047] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2A microphone volume control device 354 stored in the memory 350 is shown, and the microphone volume control device 354 may be a microphone volume control device in the server 300, which may be software in the form of a program and a plug-in, etc., and includes the following software modules: a speech detection module 3540, a preprocessing module 3541, a parameter feature extraction module 3542, a determination module 3543, and an adjustment module 3544. These modules are logical, and therefore can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0048] In other embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the microphone volume control method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs) or other electronic components.

[0049] The microphone volume control method provided by the embodiment of the present application will be described below in conjunction with the exemplary application and implementation of the server 300 provided by the embodiment of the present application. Figure 3 , Figure 3 This is an optional flow chart of the microphone volume control method provided in the embodiment of the present application. Figure 3 The steps shown are explained.

[0050] Step S301: perform voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal.

[0051] Here, speech detection is used to detect the environmental sound in the speech signal. For example, environmental indicators include but are not limited to environmental sounds such as background noise, echo and howling. Therefore, speech detection includes noise detection, echo detection and howling detection. By performing speech detection on the speech signal, the environmental sounds such as noise, echo and howling in the speech signal are obtained.

[0052] In some embodiments, when detecting noise in a speech signal, any noise detection method may be used. For example, the speech signal may be firstly detected for active speech, so that according to the fluctuation of energy in the speech signal, when the energy is large and the fluctuation is drastic, it is considered that there is speech, and the speech at this time includes both the normal speech of the user (i.e., the basic sound) and noise, so it is necessary to further extract and process the noise. Then, according to the changing law of energy fluctuation in the speech signal, the noise segment in the speech signal may be determined, and the noise energy spectrum may be estimated, so as to determine the noise signal in the speech signal, that is, to realize the noise detection in the speech signal.

[0053] In some embodiments, when detecting an echo in a voice signal, any echo detection method may be used. For example, when it is detected that the terminal is in hands-free mode during a call, and if voice is detected by active voice detection at this time, that is, based on the fluctuation of energy in the voice signal, it is considered that voice exists when the energy is large and the fluctuation is drastic. Then, it can be considered that the voice signal at this time includes an echo.

[0054] In some embodiments, when detecting howling in a voice signal, any echo detection method can be used. For example, a howling detector based on a neural network can be used for howling detection, or the distance between the two terminals of the call can be detected. When the distance between the two terminals is less than a distance threshold and both terminals are in a hands-free state, an acoustic loop will appear between the two terminals to form a howling. At this time, it can be detected that the voice signal includes howling.

[0055] Step S302: adopting a speech signal processing method corresponding to each of the environmental indicators to perform speech signal preprocessing on the speech signal respectively, and correspondingly obtaining at least two speech signal streams.

[0056] Here, each type of speech signal processing method corresponds to an environmental indicator, and the environmental indicators include but are not limited to environmental sounds such as background noise, echo and howling. In the embodiment of the present application, each speech signal stream corresponds to a speech signal processing method, and after processing the speech signal by a speech signal processing method, a corresponding speech signal stream is obtained. It should be noted that multiple types of speech signal processing methods can also be used to process the speech signal at the same time to obtain a corresponding speech signal stream.

[0057] In the embodiment of the present application, the speech signal processing method may include but is not limited to at least one of the following: DC removal filtering, echo cancellation, noise suppression, and howling suppression. When the speech signal processing method is DC removal filtering, the speech signal is subjected to DC removal filtering to obtain a speech signal stream after DC removal filtering; when the speech signal processing method is echo cancellation, the speech signal is subjected to echo cancellation to obtain a speech signal stream after echo cancellation; when the speech signal processing method is noise suppression, the speech signal is subjected to noise suppression to obtain a speech signal stream after noise suppression.

[0058] The voice signal processing method in the embodiment of the present application is to preprocess the voice signal, that is, the voice signal preprocessing can be performed after the voice signal is collected, and then the microphone gain control is performed based on the preprocessed voice signal.

[0059] In an embodiment of the present application, the voice signal may be a voice signal with a certain duration. In an embodiment of the present application, a voice frame may be used as a unit, and a frame of voice may be processed each time to achieve volume control of the microphone. In a voice frame, a frame of voice may have multiple voice samples, and each voice sample is a sample parameter of an audio waveform in the time domain. A frame of voice corresponds to a certain duration, and may be a voice sample within a period of time. For example, a voice frame of 10 milliseconds may be used, and a voice sample of 10 milliseconds may be obtained to obtain a voice signal, and then the voice signal may be processed and calculated.

[0060] Step S303: extract parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal.

[0061] Here, parameter feature extraction refers to feature extraction of the speech signal stream after each speech signal processing method, so as to obtain the current state parameters corresponding to each speech signal stream in different states, wherein the current state parameters are parameters used to describe the different information and different states contained in the speech signal.

[0062] In an embodiment of the present application, the speech signal processing method may include but is not limited to at least one of the following: DC removal filtering processing, echo cancellation processing and noise suppression processing. Correspondingly, the speech signal stream may include but is not limited to at least one of the following: a speech signal stream after DC removal filtering, a speech signal stream after echo cancellation and a speech signal stream after noise suppression.

[0063] When the voice signal stream is a voice signal stream after DC filtering, when performing parameter feature extraction on the voice signal stream, it can be a truncation detection on the voice signal stream after DC filtering to obtain a truncation mark of the voice signal stream after DC filtering. For example, it can be detected whether the amplitude of the voice signal stream after DC filtering has exceeded the representation range of a 16-bit integer, thereby causing truncation distortion. If it exceeds, the truncation mark can be set to a first truncation mark, and if it does not exceed, the truncation mark can be set to a second truncation mark. In some embodiments, when performing parameter feature extraction on the voice signal stream, it can also be a time domain energy envelope calculation on the voice signal stream after DC filtering to obtain the energy of the voice signal stream; or, a VAD detection of microphone recording is performed on the voice signal stream after DC filtering to obtain a VAD result. After obtaining the truncation mark, energy and VAD result, at least one of the truncation mark, energy and VAD result is determined as the parameter state stream of the voice signal.

[0064] When the speech signal stream is a speech signal stream after echo cancellation, when performing parameter feature extraction on the speech signal stream, fundamental pitch extraction can be performed on the speech signal stream after echo cancellation to obtain the fundamental pitch frequency of the speech signal stream after echo cancellation; and the obtained fundamental pitch frequency is determined as the parameter state stream of the speech signal stream after echo cancellation.

[0065] When the speech signal stream is a speech signal stream after noise suppression, when performing parameter feature extraction on the speech signal stream, it can be that howling detection is performed on the speech signal stream after noise suppression to obtain a howling mark of the speech signal stream after noise suppression; and VAD detection of the second type of signal is performed on the speech signal stream after noise suppression to obtain a VAD result of the speech signal stream after noise suppression; and at least one of the howling mark and the VAD result is determined as the parameter state stream of the speech signal stream after noise suppression.

[0066] Step S304: determining a digital gain adjustment amount and an analog gain adjustment amount of the microphone respectively according to at least two parameter state streams.

[0067] Here, after determining the parameter state stream, the adjustment method and adjustment amount are determined according to the current state parameter corresponding to the parameter state stream, wherein each current state parameter corresponds to a digital gain adjustment amount and / or an analog gain adjustment amount. The digital gain adjustment amount and the analog gain adjustment amount are target amounts for adjusting the digital gain and analog gain of the microphone, and the magnitude of the adjustment when the microphone is gain controlled can be determined according to the digital gain adjustment amount and the analog gain adjustment amount.

[0068] Step S305 , adjusting the digital gain and the analog gain of the microphone accordingly according to the digital gain adjustment amount and the analog gain adjustment amount, so as to achieve volume control of the microphone.

[0069] Here, after the digital gain adjustment amount and the analog gain adjustment amount are determined, the digital gain and the analog gain are adjusted according to the target amounts corresponding to the digital gain adjustment amount and the analog gain adjustment amount. It should be noted that the adjustment process of the digital gain and the analog gain can be carried out in parallel, that is, after the digital gain adjustment amount and the analog gain adjustment amount are determined, the digital gain and the analog gain can be adjusted at the same time, or after either the digital gain or the analog gain is adjusted, the other can be adjusted. In the adjustment process, if the digital gain and the analog gain are adjusted successively, if the microphone volume of either party after adjustment has met the requirements, that is, when the microphone volume reaches smoothness, the gain of the other party may not be adjusted.

[0070] In some embodiments, the value after the microphone gain is adjusted may be a preset value, that is, a target volume of the microphone is preset. Thus, when determining the digital gain adjustment amount and the analog gain adjustment amount of the microphone, the digital gain adjustment amount and the analog gain adjustment amount may be determined respectively according to at least two parameter state streams and the preset value.

[0071] In some embodiments, when the digital gain and analog gain of the microphone are adjusted accordingly according to the digital gain adjustment amount and the analog gain adjustment amount, the digital gain and the analog gain can be adjusted in parallel, or the digital gain and the analog gain can be adjusted in sequence, or only one of the digital gain and the analog gain can be adjusted. Here, parallel adjustment means that the digital gain and the analog gain are adjusted simultaneously according to the determined digital gain adjustment amount and the analog gain adjustment amount; when the digital gain and the analog gain are adjusted in sequence, any one of the digital gain and the analog gain can be adjusted first, for example, the digital gain, and after the digital gain is adjusted according to the digital gain adjustment amount, the analog gain is continued to be adjusted according to the analog gain adjustment amount. In some embodiments, the microphone volume detection can also be performed after the digital gain (or analog gain) is adjusted. When the detection result shows that the adjusted microphone volume meets the preset conditions or reaches the preset target volume, the adjustment of the analog gain (or digital gain) is stopped, that is, the volume control of the microphone is achieved only by adjusting any one of the digital gain and the analog gain.

[0072] In some embodiments, the digital gain and the analog gain can also be adjusted according to a certain adjustment ratio. For example, after determining the digital gain adjustment amount and the analog gain adjustment amount, the digital gain adjustment amount and the analog gain adjustment amount are weighted according to a preset adjustment ratio to determine the final adjustment amount of the digital gain and the analog gain, and then the digital gain and the analog gain are adjusted according to the calculated final adjustment amount. In other embodiments, when determining the digital gain adjustment amount and the analog gain adjustment amount, the ratio between the digital gain adjustment amount and the analog gain adjustment amount can be considered, that is, the determined digital gain adjustment amount and the analog gain adjustment amount have the preset adjustment ratio, so that when adjusting the digital gain and the analog gain, they can be adjusted according to the preset adjustment ratio.

[0073] In some embodiments, the digital gain adjustment amount and the analog gain adjustment amount may have a positive proportional relationship or an inverse proportional relationship. When the digital gain adjustment amount and the analog gain adjustment amount have a positive proportional relationship, the digital gain adjustment amount is used to adjust the digital gain of the increased microphone, and at the same time, the analog gain adjustment amount is used to adjust the analog gain of the increased microphone; or, the digital gain adjustment amount is used to adjust the digital gain of the decreased microphone, and at the same time, the analog gain adjustment amount is used to adjust the analog gain of the decreased microphone. When the digital gain adjustment amount and the analog gain adjustment amount have an inverse proportional relationship, the digital gain adjustment amount is used to adjust the digital gain of the increased microphone, and at the same time, the analog gain adjustment amount is used to adjust the analog gain of the decreased microphone; or, the digital gain adjustment amount is used to adjust the digital gain of the decreased microphone, and at the same time, the analog gain adjustment amount is used to adjust the analog gain of the increased microphone.

[0074] In some embodiments, after determining the digital gain adjustment amount and the analog gain adjustment amount, the digital gain adjustment amount and the analog gain adjustment amount can be divided into multiple sub-adjustment amounts according to a certain division method, and then the digital gain and the analog gain are adjusted according to the divided sub-adjustment amounts. For example, the digital gain adjustment amount and the analog gain adjustment amount can be divided according to equal division intervals or unequal division intervals.

[0075] For example, if the currently determined digital gain adjustment amount is 32db and the analog gain adjustment amount is 40db, the digital gain adjustment amount can be divided into 4 sub-adjustments {+5db; +7db; +9db; +11db}, and the analog gain adjustment amount can be divided into 4 sub-adjustments {+10db; +10db; +10db; +10db}, then, when adjusting, the digital gain can be first increased by 5db, then by 7db, then by 9db, then by 11db, and then the analog gain can be increased by 10db, then by 10db, then by 10db, then by 10db, then by 10db. Alternatively, in other embodiments, when adjusting, the digital gain can also be first increased by 5db, then by 10db, then by 7db, then by 10db, then by 9db, then by 10db, then by 11db, and then by 10db. Moreover, during the adjustment process, the microphone volume can be tested after each adjustment of the sub-adjustment amount. When the test result shows that the current adjusted microphone volume meets the preset conditions or reaches the preset target volume, the adjustment of the analog gain and the digital gain is stopped.

[0076] The embodiments of the present application can be applied to the following scenarios: when the terminal collects a voice signal in the current environment through a microphone, the terminal sends the collected voice signal to the server, and the server adopts the method provided in the embodiments of the present application, adopts at least two different types of voice signal processing methods, and performs different types of voice signal preprocessing on the voice signal collected by the microphone, and obtains at least two corresponding voice signal streams, then performs parameter feature extraction on each voice signal stream to obtain at least two parameter state streams of the voice signal, and determines the digital gain adjustment amount and analog gain adjustment amount of the microphone according to the at least two parameter state streams, after obtaining the digital gain adjustment amount and the analog gain adjustment amount, the server adjusts the digital gain and analog gain of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount to achieve volume control of the microphone, or the server sends the digital gain adjustment amount and the analog gain adjustment amount to the terminal, so that the terminal adjusts the digital gain and analog gain of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount to achieve volume control of the microphone.

[0077] The microphone volume control method provided in the embodiment of the present application performs different types of voice signal preprocessing on the voice signals collected by the microphone to obtain at least two voice signal streams, and each type of voice signal processing method corresponds to an environmental indicator, and then determines the digital gain adjustment amount and analog gain adjustment amount of the microphone based on the at least two voice signal streams obtained, and implements volume control of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount. In this way, the digital gain and model gain of the microphone can be flexibly and adaptively adjusted according to the environmental indicators of the current environment of the microphone, so that the microphone volume is smoother and the user experience is improved.

[0078] It should be noted that in other embodiments, the microphone volume control method can also be implemented by the terminal, that is, the terminal obtains the voice signal collected by the microphone itself, and adopts at least two different types of voice signal processing methods, respectively performs different types of voice signal preprocessing on the voice signal collected by the microphone, and obtains at least two voice signal streams; wherein each type of voice signal processing method corresponds to an environmental indicator; then, parameter feature extraction is performed on each voice signal stream to obtain at least two parameter state streams of the voice signal; according to the at least two parameter state streams, the digital gain adjustment amount and the analog gain adjustment amount of the microphone are respectively determined; finally, according to the digital gain adjustment amount and the analog gain adjustment amount, the digital gain and the analog gain of the microphone are adjusted accordingly to realize the volume control of the microphone. In this scenario, the terminal autonomously realizes the volume control process of the microphone, and there is no need to transmit data with the server, which can greatly save bandwidth consumption and reduce the amount of computing of the server. In addition, the volume control of the microphone by the terminal can perform real-time analysis on the voice signal collected by the microphone, thereby avoiding the problem of inaccurate volume control caused by data transmission failure or invalid data transmission.

[0079] In some embodiments, the speech signal processing method includes but is not limited to at least one of the following: DC removal filtering processing, echo cancellation processing and noise suppression processing.

[0080] Figure 4 is an optional flow chart of the microphone volume control method provided in the embodiment of the present application, such as Figure 4 As shown, in some embodiments, when the speech signal processing method includes a DC removal filtering process, the speech signal stream includes a speech signal stream after the DC removal filtering; step S303 can be implemented by the following steps:

[0081] Step S401, performing truncation detection on the speech signal stream after the DC removal filtering to obtain a truncation mark of the speech signal stream.

[0082] Here, truncation detection is used to detect whether the amplitude of the voice signal collected by the microphone has exceeded the representation range of the 16-bit integer, thereby causing truncation distortion. This situation usually indicates that the microphone hardware gain is too large and the hardware volume setting needs to be reduced as soon as possible. Truncation detection can be implemented in the following way: Since the representation range of 16-bit integers is between -32768 and 32767, if the absolute value of the signal amplitude of N voice samples in a certain frame of input voice signal is greater than a certain threshold, for example, the threshold can be 32760, then this section of voice signal can be marked as truncation, and the truncation flag Satuate=1, otherwise the truncation flag Satuate=0.

[0083] Step S402, performing time domain energy envelope calculation on the speech signal stream after the DC removal filtering to obtain the energy of the speech signal stream.

[0084] Here, the time domain energy envelope calculation is performed by dividing the speech signal into smaller speech segments, for example, dividing the original 10 millisecond speech signal (i.e., speech frame) into 5 millisecond segments, and calculating the sum of the squares (i.e., energy) of each speech sample in these speech segments, thereby obtaining the energy of the speech signal stream.

[0085] Step S403, performing VAD detection of microphone recording on the voice signal stream after the DC removal filtering to obtain a first VAD value of the voice signal stream.

[0086] Here, VAD detection refers to active voice detection, also known as voice endpoint detection and voice boundary detection. The purpose of VAD detection is to identify and eliminate long periods of silence from the voice signal stream, so as to save voice channel resources without reducing service quality. In the embodiment of the present application, VAD detection is implemented based on the energy fluctuation of the voice signal stream. If the energy is large and the fluctuation is violent, it is considered that there is voice, so the first VAD value of the voice signal stream is the VAD value of the presence of voice; if the energy is small and the fluctuation is gentle, it is considered that there is no voice, that is, it is considered that the current period is silent, so the first VAD value of the voice signal stream is the VAD value of the absence of voice.

[0087] Step S404: determining at least one of the truncation flag, the energy and the first VAD value as a parameter state flow.

[0088] Please continue to refer to Figure 4 In some embodiments, when the voice signal processing method includes echo cancellation processing, the voice signal stream includes a voice signal stream after echo cancellation; step S303 can be implemented by the following steps:

[0089] Step S405: extract the fundamental pitch of the speech signal stream after echo cancellation to obtain the fundamental pitch frequency of the speech signal stream.

[0090] When a person speaks, the vocal cords vibrate and generate harmonics. The frequency of the vocal cord vibration is the fundamental frequency, and sound signals such as noise do not have a fundamental frequency. Therefore, fundamental frequency detection can be used to detect whether there is a fundamental frequency in the speech signal stream.

[0091] Step S406: determine the fundamental frequency as a parameter state stream.

[0092] In some embodiments, when performing echo cancellation processing on a speech signal, the method may further include the following steps: Step S11, when it is detected that the speech signal contains a first type of signal, determining energy fluctuations of the first type of signal.

[0093] Here, the first type of signal may be a far-end signal, wherein the far-end signal refers to a voice signal of the other end of the call. In the embodiment of the present application, when someone is speaking at the far end, the far-end signal will also be collected and saved and used as a reference signal for echo cancellation.

[0094] Step S12: determining a second VAD value of the speech signal stream according to the energy fluctuation.

[0095] Here, based on the energy fluctuation of the far-end signal, it is determined whether there is a voice signal in the far-end signal. If there is a voice signal in the far-end signal, the second VAD value can be set to the VAD value of the far-end voice signal; if there is no voice signal in the far-end signal, the second VAD value can be set to the VAD value without the far-end voice signal.

[0096] Step S13: when the first type of signal forms an echo signal in the speech signal, determining an echo state flag of the speech signal according to the echo signal.

[0097] Here, if the remote voice signal is played through the speaker of the terminal and then collected by the microphone of the terminal itself, an echo signal will be formed. In the embodiment of the present application, the echo state flag of the voice signal collected by the microphone can be determined based on the echo signal.

[0098] Step S14: determine at least one of the second VAD value and the echo state flag as a parameter state stream.

[0099] Please continue to refer to Figure 4 In some embodiments, when the speech signal processing method includes noise suppression processing, the speech signal stream includes a speech signal stream after noise suppression, and step S303 can be implemented by the following steps:

[0100] Step S407: perform howling detection on the noise-suppressed speech signal stream to obtain a howling flag of the speech signal stream.

[0101] Here, howling detection is used to detect whether the voice signal stream contains an obvious howling signal. For example, when two calling terminals are in the same room and at least one terminal is in hands-free mode, an acoustic loop is easily formed, thereby forming a howling signal. When a howling signal is detected, the howling flag is the first howling flag, and when no howling signal is detected, the howling flag is the second howling flag.

[0102] Step S408: Perform VAD detection of the second type of signal on the speech signal stream after noise suppression to obtain a third VAD value of the speech signal stream.

[0103] Here, the second type of signal may be a near-end signal, wherein the VAD detection module of the near-end signal is the same as the VAD detection of the microphone recording mentioned above, and is based on the signal energy fluctuation to detect whether there is a voice signal in the audio signal. The difference is that here the VAD detection of the near-end signal is performed on the voice signal stream after noise suppression, so the voice signal stream here has eliminated most of the noise, so when the third VAD value is the VAD value of the voice signal, it is usually only the near-end voice input, when there is no voice input, the third VAD value is the VAD value of the voice signal.

[0104] Step S409: determine at least one of the howling flag and the third VAD value as a parameter state stream.

[0105] In some embodiments, when the howling flag of the voice signal stream is determined, howling suppression may also be performed according to the howling flag of the voice signal stream, wherein the howling suppression includes the following steps:

[0106] Step S21: when the howling flag is the first howling flag, determining the distribution law of the howling energy of the speech signal on the frequency spectrum.

[0107] Step S22: reducing the gain of the frequency band with howling energy according to the distribution rule to suppress the howling energy.

[0108] Step S23: when the howling flag is the second howling flag, cancel the suppression of the gain of the frequency band.

[0109] based on Figure 4 , Figure 5 is an optional flow chart of the microphone volume control method provided in the embodiment of the present application, such as Figure 5 As shown, in some embodiments, after determining the first VAD value, the method may further include the following steps:

[0110] Step S501: determining at least one noise segment in a speech signal according to a first VAD value.

[0111] Step S502: Estimating the noise energy of each noise segment to obtain a noise energy spectrum of the corresponding noise segment.

[0112] Step S503: subtract the noise energy spectrum of each noise segment from the speech signal spectrum corresponding to the speech signal to obtain a denoised speech energy spectrum.

[0113] Step S504: performing time-frequency transformation processing on the denoised speech energy spectrum to obtain a speech signal stream after noise suppression.

[0114] In the embodiment of the present application, when the first VAD value is determined, noise suppression processing can also be performed according to the first VAD value. For example, the noise energy spectrum can be estimated, and then the noise energy spectrum can be subtracted from the microphone signal spectrum, and the remaining speech energy spectrum can be reconstructed into a denoised speech waveform through time-frequency transformation.

[0115] Correspondingly, after obtaining the noise-suppressed speech signal stream, parameter feature extraction is performed on the noise-suppressed speech signal stream to obtain at least two parameter state streams of the speech signal, that is, executing step S407.

[0116] based on Figure 4 , Figure 6 is an optional flow chart of the microphone volume control method provided in the embodiment of the present application, such as Figure 6 As shown, in step S304, determining the digital gain adjustment amount of the microphone according to at least two parameter state flows can be achieved by the following steps:

[0117] Step S601, obtaining the current digital gain of the microphone.

[0118] Step S602: When the third VAD value is the first preset value, the root mean square energy of the speech signal in a preset time period is counted.

[0119] Step S603: if the root mean square energy is less than the first energy threshold, the current digital gain of the microphone is increased by the first preset gain to obtain a digital gain adjustment amount.

[0120] In the embodiment of the present application, the gain may be gradually increased multiple times, each time the first preset gain is increased until the digital gain upper limit or the digital gain target value is reached. At this time, the sum of the increased multiple first preset gains is determined as the digital gain adjustment amount.

[0121] Furthermore, when determining the digital gain adjustment amount of the microphone, the following steps may also be included:

[0122] Step S604: when the howling flag is the first howling flag, stop calculating the root mean square energy until the howling flag is the second howling flag.

[0123] Step S605: when the howling flag is the second howling flag, continue to update the root mean square energy.

[0124] Please continue to refer to Figure 6 In some embodiments, determining the analog gain adjustment amount of the microphone according to at least two parameter state streams in step S304 can be implemented by the following steps:

[0125] Step S606, obtaining the current analog gain of the microphone.

[0126] Step S607: when the first VAD value is the second preset value and the fundamental frequency is greater than zero, the smoothed speech segment energy is calculated according to the energy of the speech signal stream.

[0127] Step S608: when the energy of the smoothed speech segment is greater than the second energy threshold, the current analog gain of the microphone is reduced by the second preset gain to obtain an analog gain adjustment amount.

[0128] In the embodiment of the present application, the second preset gain can be gradually reduced multiple times, each time until the analog gain lower limit or the analog gain target value is reached. At this time, the sum of the reduced multiple second preset gains is determined as the analog gain adjustment amount.

[0129] Furthermore, when determining the analog gain adjustment amount of the microphone, the following steps may also be included:

[0130] Step S609: when it is determined according to the parameter state flow that there is an echo in the speech signal, it is prohibited to increase the current analog gain of the microphone.

[0131] In the embodiment of the present application, after determining the digital gain adjustment amount and the analog gain adjustment amount of the microphone, the analog gain of the microphone is adjusted accordingly according to the analog gain adjustment amount in step S305, which can be achieved by any one of the following methods:

[0132] Method 1 includes the following steps:

[0133] Step S31, mapping the analog gain adjustment amount to a preset gain interval to obtain a mapping value of the analog gain adjustment amount. Step S32, using the mapping value to adjust the analog gain of the microphone.

[0134] Method 2 includes the following steps:

[0135] Step S33, performing two-level gain mapping on the analog gain adjustment amount to obtain a first-level gain adjustment amount corresponding to the microphone enhancement and a second-level gain adjustment amount corresponding to the microphone array. Step S34, adjusting the microphone enhancement of the microphone according to the first-level gain adjustment amount. Step S35, adjusting the microphone array of the microphone according to the second-level gain adjustment amount.

[0136] In an embodiment of the present application, when adjusting the microphone volume, the microphone enhancement and the microphone array can be adjusted at the same time. In this way, the microphone volume can be adjusted stably and orderly in combination with the first-level gain adjustment amount and the second-level gain adjustment amount, so that the adjusted microphone volume is smoother and better meets the user's usage needs.

[0137] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0138] In voice communication systems, voice signals are collected through microphone hardware, and the collected voice volumes vary. If the voice volume is too loud, it will cause voice distortion or affect the effects of other modules (for example, if the echo volume recorded by the microphone is too loud, there will be large nonlinear distortion, resulting in unclean echo cancellation); if the voice volume is too low, the receiver will not be able to hear clearly. Therefore, how to control the recording volume within a reasonable range and adjust it smoothly requires comprehensive consideration of the comprehensive impact of various voice processing modules, while accurately distinguishing various signals such as human voice, background noise, echo, howling, etc., and making full use of the capabilities provided by the device hardware or the underlying audio interface to achieve better results.

[0139] In order to solve the above problems, an embodiment of the present application provides a microphone volume control method, which is an automatic gain control method for a microphone, and the method includes systematically considering the influence of various signals such as VAD, echo, howling, etc., and integrating the fundamental pitch of the human voice and other characteristics to better distinguish between noise and human voice. For the adjustment of analog gain (hardware volume) and digital gain (software volume), in order to make the overall volume adjustment smoother, a new logic is designed to more accurately coordinate the changes in analog gain and digital gain. Finally, a complete gain control system is formed to achieve the purpose of optimizing the control of the microphone volume.

[0140] It should be noted that in the actual application scenario of microphone volume adjustment, digital gain mainly adjusts the pulse amplitude of the digital-to-analog conversion input. If the amplitude of the digital gain is too small, it will cause an increase in lost bit errors. If the amplitude of the digital gain is too large, the digital signal signal-to-noise ratio will also deteriorate due to severe data pulse clipping and increased noise pulses, which will also cause an increase in interference bit errors. The optimal value of the digital gain is to adjust the upper limit within the domain value range of the digital circuit input. The size of the digital gain will not increase or decrease the value of the output audio power, but only affect the working state of the decoder. The analog gain mainly adjusts the signal strength of the linear amplification input. The size of the analog gain directly affects the value of the output audio power within a certain range. A larger analog gain input value is conducive to improving the output signal-to-noise ratio and will also increase the output power year-on-year. However, when the input is too large, the output power increases slowly, while the distortion rises sharply. The optimal adjustment value of the analog gain should make the output voltage peak within the linear range of the amplifier.

[0141] At present, the use of voice transmission over Internet Protocol (VoIP) based on IP is becoming more and more common. The call application runs on terminals with different operating systems, such as iOS / Mac / Android / Windows, etc. Figure 7 , which is an application scenario diagram of the method of the embodiment of the present application. The method of the embodiment of the present application can be applied to Figure 7 Any one of the calling devices 701 to 704.

[0142] Common applications that support VoIP include WeChat and QQ voice calls. Different terminals may have different initial microphone volume settings, resulting in loud microphones on some terminals and soft microphones on others. If the microphone hardware volume (also called microphone analog gain) is too low, the sound recorded by the microphone will be too low for the other party to hear. Even if the digital gain (also called digital volume or software volume) is increased, the required volume cannot be reached. Increasing the digital gain will also amplify the noise (even after passing through the noise suppression module, a small amount of noise may remain), reducing the auditory comfort. If the microphone volume is too high, the maximum value of the audio sample will exceed the digital representation range (for example, the value range of a 16-bit integer is between -32768 and 32767), causing truncation distortion (also called saturation distortion). In addition, if the microphone volume is too high, nonlinear distortion (such as truncation distortion and hardware distortion) will be serious, which will cause the performance of echo cancellation to deteriorate.

[0143] The automatic gain control provided in the embodiment of the present application needs to comprehensively consider various acoustic scenarios such as speech, noise, echo, howling, etc. to avoid incorrect adjustment of the volume.

[0144] The embodiment of the present application provides a complete microphone automatic gain control system to achieve the purpose of automatically controlling the digital gain and analog gain of the microphone at the transmitting end, thereby keeping the microphone volume at a reasonable level. Figure 8 is an architecture diagram of a microphone automatic gain control method provided in an embodiment of the present application, such as Figure 8 As shown, the audio acquisition hardware interface 801 collects the speaker's voice signal, and may also collect the background noise of the surrounding environment (such as office air conditioning, or the sound of car engines on the roadside, etc.). If you are talking to other people, there may also be echo signals (for example, the sound played by your own speaker in hands-free mode will enter the microphone again) and howling signals (for example, when there are two terminals A and B in the same conference room during a multi-terminal call, a person speaks to the microphone of terminal A, and the sound is played from the speaker of terminal B and then circulated into the microphone of terminal A). The voice is sent to the voice processing system in the form of frames, for example, every 20 milliseconds is a frame, and the voice processing system processes each frame of voice signal in turn. A gain control module 802 is required to accurately detect the volume of the voice and automatically adjust the analog gain (i.e., hardware volume) and digital gain (i.e., software volume) according to the size of the voice volume.

[0145] Please continue to refer to Figure 8 , the audio acquisition hardware interface 801 provides a program interface for obtaining and setting microphone-related hardware parameters through software. The DC removal filter module 803 removes the DC component in the audio signal collected by the microphone through a preset filter to avoid interference with voice detection and subsequent processing. The echo cancellation module 804 eliminates the echo signal that may exist in the microphone signal to prevent the far-end speaker from hearing his own voice. The noise suppression module 805 suppresses the background noise in the microphone signal to improve the auditory comfort of the listener. The howling suppression module 806 suppresses the howling sound that may exist in the microphone signal (for example, when the two call terminals are very close and hands-free, an acoustic loop will appear between the two call terminals, thereby forming acoustic feedback or howling). The gain control module 802 adjusts the amplitude of the audio signal according to the acoustic parameters, that is, the digital volume or digital gain, and calculates the gain of the microphone hardware, and obtains or sets the hardware volume (that is, analog gain) through the audio acquisition hardware interface 801. Finally, the encoding and sending module 807 compresses and encodes the audio signal and sends it to the receiving party through the network for decoding and playback.

[0146] Figure 8 The solid line in the middle represents the speech signal flow, and the dotted line represents the parameter state flow. Figure 8 The microphone hardware status pool 808 is used to store status information from the hardware, including the current gain value of the hardware microphone, whether two-level gain control is supported, how many steps there are in each level of gain, how many dB of gain each step represents, and so on. Figure 8The speech feature pool 809 in is used to calculate and store speech features.

[0147] Fig. 9 This is a diagram of the voice feature pool architecture provided by the embodiment of the present application, combined with Figure 8 and Fig. 9 As shown:

[0148] The speech signal stream (1) in the speech feature pool is the speech signal after the DC filtering, which is used for truncation detection, calculation of time domain energy envelope and VAD detection of microphone recording signals.

[0149] Among them, the truncation detection module 901 is used to detect whether the amplitude of the microphone recording signal has exceeded the representation range of the 16-bit integer, thereby causing truncation distortion. This situation usually indicates that the microphone hardware gain is too large, and the hardware volume setting needs to be reduced as soon as possible. A simple truncation detection method is: since the representation range of 16-bit integers is between -32768 and 32767, if in a certain frame of input speech, there are N speech samples whose signal amplitudes have an absolute value greater than a certain threshold, such as N=10, and the threshold can be 32760, for example, then this section of speech signal can be marked as truncation, and the truncation flag Satuate=1 is set, otherwise the truncation flag Satuate=0.

[0150] The module 902 for calculating the time domain energy envelope divides the speech frame into smaller speech segments, such as a segment of 5 milliseconds, and calculates the sum of squares (ie, energy) E(t) of each speech sample in the speech segment.

[0151] The VAD detection module 903 of the microphone recording signal is used to perform active voice detection (VAD, Voice Activity Detection) on the signal collected by the microphone, generally based on the energy fluctuations in the audio signal. When the energy is large and the fluctuations are violent, it is considered that there is voice, and VAD1=1 (where VAD1 corresponds to the first VAD value mentioned above), otherwise it is considered that there is no voice, and VAD1=0. Note that at this time, in addition to being able to detect the voice segment, if the background noise is not stable, such as the sound of a keyboard nearby, or the sound of opening and closing a door, it is easy to detect as VAD1=1. In addition, if there is a strong echo signal, such as the voice of the far-end speaker played in the speaker in the hands-free mode and entering the microphone again, it will also be detected as VAD1=1. Taking the energy envelope of the echo signal into consideration means that the volume needs to be reduced when the echo is too large, which is the expected behavior.

[0152] The speech signal stream (2) in the speech feature pool is the microphone signal after echo cancellation. The fundamental frequency extraction module 904 uses this signal to detect the fundamental frequency. Since the vocal cords vibrate and generate harmonics when people speak, the frequency of the vocal cord vibration is the fundamental frequency. Many noises do not have a fundamental frequency or the fundamental frequency is not obvious, such as instantaneous noise such as opening and closing doors or knocking on tables and keyboards. Therefore, making full use of fundamental frequency detection can eliminate most of the interference of noise on VAD detection and find the speech segment more accurately. The fundamental frequency pitch of an average person is between 50 and 500 Hz, and most people are around one or two hundred Hz. If the fundamental frequency is not detected, the fundamental frequency pitch is set to 0.

[0153] The speech signal stream (3) in the speech feature pool is the microphone signal after noise suppression, and is used for howling detection and VAD detection of near-end signals.

[0154] Among them, the howling detection module 905 detects whether the audio signal contains an obvious howling signal. For example, when two calling terminals are in the same room and at least one terminal is in hands-free mode, an acoustic loop is easily formed, thereby forming a howling. The howling signal may cause the speech energy to be misestimated, thereby having a negative impact on the automatic gain control. The howling detection module can be a howling detector based on a neural network, such as a recurrent neural network (RNN) howling detector trained in advance. When howling is detected, the howling flag howlFlag = 1, and if no howling is detected, howlFlag = 0.

[0155] The VAD detection module 906 of the near-end signal is the same as the VAD detection module 903 of the microphone recording signal. Both detect whether there is a voice signal in the audio signal based on the signal energy fluctuation. The difference is that since the echo and most of the noise have been eliminated in the voice signal stream (3) here, when the VAD flag VAD2=1 (where VAD2 corresponds to the third VAD value mentioned above), it is generally only an indication of the presence of voice at the near end. When there is no voice at the near end and only an echo, VAD2=0. Since noise elimination is not easy to eliminate completely, for non-stationary noise, such as keyboard sounds, VAD2 here is also likely to be equal to 1.

[0156] The parameter state stream (4) in the speech feature pool comes from the echo cancellation module. In general echo cancellation algorithms, the echo state 907 and the VAD 908 of the far-end signal are calculated. When someone speaks at the far end, the far-end speech signal will be saved in the echo cancellation module as a reference signal for the echo cancellation algorithm. By detecting the energy fluctuation of the far-end signal, it can be detected whether there is a speech signal in the far-end signal. If so, VAD3=1 (where VAD3 corresponds to the second VAD value mentioned above), otherwise VAD3=0. If the far-end speech signal is played out through the speaker and collected by its own microphone, an echo signal is formed. The echo cancellation module will eliminate the echo and detect whether there is an echo at present. If so, the echo state EchoState=1 is set, otherwise EchoState=0.

[0157] exist Fig. 9 On the right side of , the parameter state flow (5) is the VAD detection result VAD1 of the microphone recording, which can be used in the noise suppression algorithm. For example, in the noise segment, the noise energy spectrum can be estimated and then subtracted from the microphone signal spectrum. The remaining speech energy spectrum can be reconstructed into the denoised speech waveform through time-frequency transformation.

[0158] exist Fig. 9 On the right side of , the parameter state stream (6) is the howling flag howlFlag of the howling detection. When howlFlag=1, it means there is howling at this time. The howling suppression module can use this information to suppress howling, for example, to detect the distribution of howling energy on the spectrum and reduce the gain of the frequency band with howling energy, that is, to suppress the howling energy to reduce the howling sound. When howlFlag=0, the suppression of the frequency band is gradually cancelled so that the frequency band gain returns to normal.

[0159] exist Fig. 9 On the right side of , the parameter state stream (7) includes all the features calculated in the speech feature pool mentioned above, and these features can be used for automatic gain control. The automatic gain control module is described in detail below.

[0160] Fig.10 is a schematic diagram of a gain control module provided in an embodiment of the present application, such as Fig.10 As shown, the digital gain control module 1001 can adopt a traditional digital gain control algorithm. When the VAD mark of the near-end signal in the voice feature pool shows that there is voice, that is, when VAD2=1, the root mean square energy (RMSLevel) of the voice signal in the recent period, such as 0.5 seconds, is counted. If the root mean square energy is less than the target volume, the digital gain is gradually increased until the preset digital gain upper limit is reached, such as amplification of no more than 30dB. If it is greater than the target volume, the digital gain is gradually reduced until the preset digital gain lower limit, such as -10dB, is reached.

[0161] When the digital gain control module 1001 increases or decreases the digital gain, if there is a howling signal, that is, when howlFlag in the speech feature pool is 1, the RMS energy value of the speech signal is stopped from being calculated until the howling signal disappears and the RMS energy of the speech signal is continued to be updated.

[0162] When the digital gain control module 1001 increases or decreases the digital gain, if there is a far-end speech signal, that is, when VAD3=1 in the speech feature pool, the digital gain is increased or decreased at a slower speed to avoid the negative impact caused by the erroneous speech RMS energy estimation caused by the failure to eliminate the clean echo.

[0163] The digital gain control module 1001 outputs a digital audio stream after volume adjustment.

[0164] Please continue to refer to Fig.10 , Fig.10 The analog gain calculation module 1002 in the example calculates the target analog gain A according to the information provided by the parameter state flow (7), and the specific implementation may include:

[0165] When the microphone recording VAD in the speech feature pool detects speech and the pitch extraction module detects the presence of pitch, that is, when VAD1=1 and pitch>0, the energy E(t) of the 5-ms speech segment calculated in the speech feature pool is used to calculate a smoothed speech segment energy using the following formula (1-1):

[0166] E m (t) = p*E m (t-1)+(1-p)*E(t) (1-1);

[0167] Among them, E m (t) represents the energy of the smoothed speech segment; E m (t-1) represents the energy of the smoothed speech segment obtained from the previous speech segment; p is a decimal less than 1, for example, it can be 0.95. m When (t) is greater than a preset energy upper limit, the target analog gain A is gradually reduced; when E m When (t) is less than a lower energy limit, the target analog gain A is gradually increased.

[0168] When the truncation detection module in the speech feature pool detects truncation, that is, Satuate=1, the target analog gain A is reduced frame by frame at a faster speed. For example, under normal circumstances, the target analog gain A at time t is calculated by the following formula (1-2):

[0169] A(t) = A(t-1)-1 (1-2);

[0170] Wherein, A(t-1) represents the analog gain of the previous frame.

[0171] When there is truncation, the following formula (1-3) can be used for calculation:

[0172] A(t)=A(t-1)-2 (1-3).

[0173] When the echo state in the speech feature pool is 1, ie, EchoState=1, in order to avoid the residual echo causing the speech energy to be estimated too small, which leads to the erroneous increase of the analog gain, the analog gain is only allowed to decrease and is not allowed to increase further.

[0174] When the howling detection state in the speech feature pool is 1, that is, howlFlag = 1, in order to avoid the howling sound causing the speech energy to be misestimated, the update of E is stopped at this time. m (t).

[0175] Fig.10 The analog gain conversion module 1003 in the example is used to convert the calculated target analog gain A into a hardware analog volume value range V. For example, if the target analog gain A calculated by the analog gain calculation module 1002 is in the range of integers between 0 and 300, and the actual hardware gain control interface accepts a numerical range V of integers between 0 and 100, then one way is to evenly map the values ​​of 0 to 256 in A to 0 to 100 (here 256 can be a preset value, and of course other values ​​can be taken), that is, 256 is set as the upper limit of A, and the part exceeding 256 can be ignored, because the digital gain can compensate for this part of the volume. Finally, let V = 100*A / 256.

[0176] Another implementation of the analog gain conversion module 1003 is: In some communication terminals, there is sometimes a two-stage gain control mechanism, such as Fig.11 The two-level volume control diagram in FIG. 1 is a schematic diagram of the microphone enhancement 1101 (i.e., the first level volume control) and the microphone array 1102 (i.e., the second level volume control) in the Windows system. Fig.11 The two-level volume control is abstracted as Fig.12 The first-stage gain 1201 is divided into three levels (there may be more levels in practice), each level represents 12dB gain (it may also be 10dB or other values), and the total is 36dB. The second-stage gain 1202 is divided into 100 levels. If a total of 60dB gain is provided, each level represents 0.6dB.

[0177] To increase the volume control range, Fig.12The two gains in are controlled. For example, when the volume is still not loud enough after the second-stage gain 1202 has increased to 100, if the first-stage gain 1201 is still at a lower gear, the first-stage gain 1201 can be increased by one gear. In order to make the volume transition smoothly, while increasing the first-stage gain 1201 by one gear, the second-stage gain 1202 needs to be reduced to a switching point. For example, for the above example, each gear of the first-stage gain 1201 represents 12dB, and each step of the second stage represents 0.6dB, then the switching point S of the second-stage gain 1202 is at (100-12dB / 0.6)=80. Therefore, smooth gain adjustment can be achieved through the following logic:

[0178]

[0179] After the above logic processing, the final calculated second-stage analog gain ranges from 0 to 256, which also needs to be converted into a number between 0 and 100 acceptable to the hardware interface, that is, V = 100*A / 256.

[0180] When the first stage gain needs to be adjusted or the second stage gain adjustment exceeds a certain amount, such as 2dB, Figure 8 The gain control module 802 in the system sends the new analog gain value, i.e., the parameter state stream (8), to the hardware through the audio acquisition hardware interface, thereby adjusting the analog gain. The new hardware configuration, i.e., the parameter state stream (9), is then sent to the microphone hardware state pool for storage and reference by the gain control algorithm.

[0181] The microphone volume control method provided in the embodiment of the present application includes systematically considering the influence of various signals such as VAD, echo, howling, etc., and integrating the fundamental pitch of the human voice and other characteristics to better distinguish between noise and human voice. For the adjustment of analog gain (hardware volume) and digital gain (software volume), in order to make the overall volume adjustment smoother, a new logic is designed to more accurately coordinate the changes in analog gain and digital gain. Finally, a complete gain control system is formed to achieve the purpose of optimizing the microphone volume.

[0182] The following is a description of an exemplary structure of the microphone volume control device 354 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software module stored in the microphone volume control device 354 of the memory 350 may be the microphone volume control device in the server 300, and the device includes:

[0183] A speech detection module 3540 is used to perform speech detection on the speech signal collected by the microphone to obtain at least two environmental indicators corresponding to the speech signal; a preprocessing module 3541 is used to adopt a speech signal processing method corresponding to each of the environmental indicators to perform different types of speech signal preprocessing on the speech signal to obtain at least two speech signal streams; a parameter feature extraction module 3542 is used to extract parameter features on each speech signal stream to obtain at least two parameter state streams of the speech signal; a determination module 3543 is used to determine the digital gain adjustment amount and the analog gain adjustment amount of the microphone according to the at least two parameter state streams; an adjustment module 3544 is used to adjust the digital gain and the analog gain of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount to achieve volume control of the microphone.

[0184] In some embodiments, the speech signal processing method includes DC removal filtering processing; the speech signal stream includes a speech signal stream after DC removal filtering; the parameter feature extraction module is also used to: perform truncation detection on the speech signal stream after DC removal filtering to obtain a truncation mark of the speech signal stream; perform time domain energy envelope calculation on the speech signal stream after DC removal filtering to obtain the energy of the speech signal stream; perform VAD detection of microphone recording on the speech signal stream after DC removal filtering to obtain a first VAD value of the speech signal stream; and determine at least one of the truncation mark, the energy and the first VAD value as the parameter state stream.

[0185] In some embodiments, the device also includes: determining at least one noise segment in the speech signal based on the first VAD value; a noise segment determination module, used to estimate the noise energy of each noise segment to obtain the noise energy spectrum of the corresponding noise segment; a deletion module, used to subtract the noise energy spectrum of each noise segment from the speech signal spectrum corresponding to the speech signal to obtain a denoised speech energy spectrum; a time-frequency transformation module, used to perform time-frequency transformation processing on the denoised speech energy spectrum to obtain a noise-suppressed speech signal stream; correspondingly, the parameter feature extraction module is also used to: perform parameter feature extraction on the noise-suppressed speech signal stream to obtain at least two parameter state streams of the speech signal.

[0186] In some embodiments, the speech signal processing method also includes echo cancellation processing; the speech signal stream includes a speech signal stream after echo cancellation; the parameter feature extraction module is also used to: perform fundamental tone extraction on the speech signal stream after echo cancellation to obtain the fundamental tone frequency of the speech signal stream; and determine the fundamental tone frequency as the parameter state stream.

[0187] In some embodiments, when performing the echo cancellation processing on the speech signal, the device includes: an energy fluctuation determination module, which is used to determine the energy fluctuation of the first type of signal when a first type of signal is detected in the speech signal; a second VAD value determination module, which is used to determine the second VAD value of the speech signal stream according to the energy fluctuation; an echo state flag determination module, which is used to determine the echo state flag of the speech signal according to the echo signal when the first type of signal forms an echo signal in the speech signal; and a parameter state flow determination module, which is used to determine at least one of the second VAD value and the echo state flag as the parameter state flow.

[0188] In some embodiments, the speech signal processing method also includes noise suppression processing; the speech signal stream includes a speech signal stream after noise suppression; the parameter feature extraction module is also used to: perform howling detection on the speech signal stream after noise suppression to obtain a howling sign of the speech signal stream; perform VAD detection of a second type of signal on the speech signal stream after noise suppression to obtain a third VAD value of the speech signal stream; and determine at least one of the howling sign and the third VAD value as the parameter state stream.

[0189] In some embodiments, the device further includes: a distribution law determination module, which is used to determine the distribution law of the howling energy of the speech signal on the frequency spectrum when the howling mark is a first howling mark; a howling energy suppression module, which is used to reduce the gain of the frequency band with the howling energy according to the distribution law to suppress the howling energy; and a cancellation module, which is used to cancel the suppression of the gain of the frequency band when the howling mark is a second howling mark.

[0190] In some embodiments, the determination module is also used to: obtain the current digital gain of the microphone; when the third VAD value is a first preset value, count the root mean square energy of the speech signal within a preset time period; if the root mean square energy is less than a first energy threshold, increase the current digital gain of the microphone by a first preset gain to obtain the digital gain adjustment amount.

[0191] In some embodiments, the determination module is further configured to: when the howling flag is the first howling flag, stop calculating the root mean square energy until the howling flag is the second howling flag; and when the howling flag is the second howling flag, continue updating the root mean square energy.

[0192] In some embodiments, the determination module is also used to: obtain the current analog gain of the microphone; when the first VAD value is a second preset value and the fundamental frequency is greater than zero, calculate the smoothed speech segment energy according to the energy of the speech signal stream; when the smoothed speech segment energy is greater than a second energy threshold, reduce the current analog gain of the microphone by a second preset gain to obtain the analog gain adjustment amount.

[0193] In some embodiments, the adjustment module is further used to: map the analog gain adjustment amount to a preset gain interval to obtain a mapping value of the analog gain adjustment amount; and use the mapping value to adjust the analog gain of the microphone.

[0194] In some embodiments, the adjustment module is also used to: perform two-level gain mapping on the analog gain adjustment amount to obtain a first-level gain adjustment amount corresponding to the microphone enhancement and a second-level gain adjustment amount corresponding to the microphone array; adjust the microphone enhancement of the microphone according to the first-level gain adjustment amount; and adjust the microphone array of the microphone according to the second-level gain adjustment amount.

[0195] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0196] The embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method of the embodiment of the present application.

[0197] The present application embodiment provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the present application embodiment, for example, Figure 3 The method shown.

[0198] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0199] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0200] As an example, executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file storing other programs or data, such as one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one location, or on multiple computing devices distributed at multiple locations and interconnected by a communication network.

[0201] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A microphone volume control method, characterized in that: The method comprises: Performing voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal; Using a speech signal processing method corresponding to each of the environmental indicators, respectively preprocess the speech signals to obtain at least two speech signal streams; Performing parameter feature extraction on each speech signal stream to obtain at least two parameter state streams of the speech signal; Determining a digital gain adjustment amount and an analog gain adjustment amount of the microphone respectively according to the at least two parameter state streams; According to the digital gain adjustment amount and the analog gain adjustment amount, the digital gain and the analog gain of the microphone are adjusted accordingly to achieve volume control of the microphone.

2. The method according to claim 1, characterized in that: The speech signal processing method includes a DC removal filtering process; the speech signal stream includes a speech signal stream after DC removal filtering; The step of extracting parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal includes: Performing truncation detection on the voice signal stream after the DC removal filtering to obtain a truncation mark of the voice signal stream; Performing time domain energy envelope calculation on the voice signal stream after the DC removal filtering to obtain the energy of the voice signal stream; Performing VAD detection of microphone recording on the voice signal stream after the DC removal filtering to obtain a first VAD value of the voice signal stream; At least one of the truncation flag, the energy, and the first VAD value is determined as the parameter state stream.

3. The method according to claim 2, characterized in that The method further comprises: Determining at least one noise segment in the speech signal according to the first VAD value; Performing noise energy estimation on each noise segment to obtain a noise energy spectrum of the corresponding noise segment; Subtracting the noise energy spectrum of each noise segment from the speech signal spectrum corresponding to the speech signal to obtain a denoised speech energy spectrum; Performing time-frequency transformation on the denoised speech energy spectrum to obtain a speech signal stream after noise suppression; Correspondingly, parameter feature extraction is performed on the noise-suppressed speech signal stream to obtain at least two parameter state streams of the speech signal.

4. The method according to claim 2, characterized in that: The speech signal processing method further includes echo cancellation processing; the speech signal stream includes a speech signal stream after echo cancellation; The step of extracting parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal further includes: Performing pitch extraction on the voice signal stream after echo cancellation to obtain the pitch frequency of the voice signal stream; The fundamental frequency is determined as the parameter state stream.

5. The method according to claim 4, characterized in that When performing the echo cancellation process on the voice signal, the method further includes: When it is detected that the speech signal contains a first type of signal, determining energy fluctuation of the first type of signal; Determining a second VAD value of the speech signal stream according to the energy fluctuation; When the first type of signal forms an echo signal in the speech signal, determining an echo state flag of the speech signal according to the echo signal; At least one of the second VAD value and the echo state flag is determined as the parameter state stream.

6. The method according to claim 4, characterized in that The speech signal processing method further includes noise suppression processing; the speech signal stream includes a speech signal stream after noise suppression; The step of extracting parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal further includes: Performing howling detection on the noise-suppressed speech signal stream to obtain a howling flag of the speech signal stream; Performing VAD detection of the second type of signal on the noise suppressed speech signal stream to obtain a third VAD value of the speech signal stream; At least one of the howling flag and the third VAD value is determined as the parameter state stream.

7. The method according to claim 6, characterized in that The method further comprises: When the howling flag is the first howling flag, determining a distribution rule of the howling energy of the speech signal on the frequency spectrum; According to the distribution rule, reducing the gain of the frequency band having the howling energy to suppress the howling energy; When the howling flag is the second howling flag, the suppression of the gain of the frequency band is cancelled.

8. The method according to claim 7, characterized in that Determining a digital gain adjustment amount of the microphone according to the at least two parameter state streams includes: Obtaining the current digital gain of the microphone; When the third VAD value is the first preset value, counting the root mean square energy of the speech signal within a preset time period; If the root mean square energy is less than the first energy threshold, the current digital gain of the microphone is increased by a first preset gain to obtain the digital gain adjustment amount.

9. The method according to claim 8, characterized in that Determining the digital gain adjustment amount of the microphone according to the at least two parameter state streams, further comprising: When the howling flag is the first howling flag, stop calculating the root mean square energy until the howling flag is the second howling flag; When the howling flag is the second howling flag, the root mean square energy continues to be updated.

10. The method according to claim 6, characterized in that Determining an analog gain adjustment amount of the microphone according to the at least two parameter state streams includes: Obtaining the current analog gain of the microphone; When the first VAD value is a second preset value and the fundamental frequency is greater than zero, calculating the smoothed speech segment energy according to the energy of the speech signal stream; When the energy of the smoothed speech segment is greater than a second energy threshold, the current analog gain of the microphone is reduced by a second preset gain to obtain the analog gain adjustment amount.

11. The method according to any one of claims 1 to 10, characterized in that: According to the analog gain adjustment amount, the analog gain of the microphone is correspondingly adjusted, including: Mapping the analog gain adjustment amount to a preset gain interval to obtain a mapping value of the analog gain adjustment amount; The mapping value is used to adjust the analog gain of the microphone.

12. The method according to any one of claims 1 to 10, characterized in that: According to the analog gain adjustment amount, the analog gain of the microphone is correspondingly adjusted, including: Performing two-stage gain mapping on the analog gain adjustment amount to obtain a first-stage gain adjustment amount corresponding to microphone enhancement and a second-stage gain adjustment amount corresponding to the microphone array; adjusting the microphone boost of the microphone according to the first-stage gain adjustment amount; The microphone array of the microphone is adjusted according to the second-stage gain adjustment amount.

13. A microphone volume control device, characterized in that: The device comprises: A voice detection module, used to perform voice detection on the voice signal collected by the microphone to obtain at least two environmental indicators corresponding to the voice signal; A preprocessing module, configured to perform voice signal preprocessing on the voice signal respectively using a voice signal processing method corresponding to each of the environmental indicators, and correspondingly obtain at least two voice signal streams; A parameter feature extraction module, used to extract parameter features from each speech signal stream to obtain at least two parameter state streams of the speech signal; A determination module, configured to respectively determine a digital gain adjustment amount and an analog gain adjustment amount of the microphone according to the at least two parameter state streams; The adjustment module is used to adjust the digital gain and the analog gain of the microphone according to the digital gain adjustment amount and the analog gain adjustment amount, so as to achieve volume control of the microphone.

14. A microphone volume control device, characterized in that: include: A memory for storing executable instructions; The processor is used to implement the microphone volume control method according to any one of claims 1 to 12 when executing the executable instructions stored in the memory.

15. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute the executable instructions to implement the microphone volume control method according to any one of claims 1 to 12.

Images