Audio channel selection method, device, equipment and storage medium

Abstract

The embodiment of the present application provides a method, device, equipment and storage medium for selecting an audio channel, which relates to a cascaded expansion microphone. The method includes: performing a speech enhancement operation on the audio signal of the cascaded expansion microphone to obtain a speech-enhanced audio signal; obtaining energy smoothing information for the speech-enhanced audio signal; and determining a target audio channel for audio output according to the energy smoothing information. By determining the optimal audio channel output based on the energy smoothing information after speech enhancement, the final effect of the cascaded expansion microphone can be enhanced on the basis of integrating the current audio algorithm and the speech enhancement scheme, thereby realizing the selection of the optimal audio channel.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a method for selecting an audio channel, a device for selecting an audio channel, a corresponding electronic device, and a corresponding computer storage medium. Background Art

[0002] With the popularization of audio and video communication systems, they are widely used in audio and video conferencing. With the widespread application of audio and video conferencing, in order to cover large conference rooms that can accommodate more participants, different conference terminals are becoming more and more common. One of the forms includes a cascade mode, which can be used to connect pickup devices distributed in different locations in the conference room through a cascade method to further reduce the pickup distance based on distributed spatial coverage, thereby achieving the effect of covering the entire conference room.

[0003] The cascading mode can solve the problem of poor far-field sound pickup that is common in large conference rooms. However, the output channel ultimately used in the cascading mode may not be the optimal audio channel. Summary of the invention

[0004] In view of the above problems, embodiments of the present application are proposed to provide a method for selecting an audio channel, a device for selecting an audio channel, a corresponding electronic device, and a corresponding computer storage medium that overcome the above problems or at least partially solve the above problems.

[0005] The present application embodiment discloses a method for selecting an audio channel, which involves cascading extension microphones. The method includes:

[0006] Performing a speech enhancement operation on the audio signal of the cascade extension microphone to obtain a speech-enhanced audio signal;

[0007] Acquire energy smoothing information of the audio signal after speech enhancement;

[0008] A target audio channel for audio output is determined according to the energy smoothing information.

[0009] Optionally, the cascaded extension microphone includes at least two extension microphones; and the performing a speech enhancement operation on the audio signal of the cascaded extension microphone includes:

[0010] The audio signal of each expansion microphone in the cascaded expansion microphone is obtained, and a voice enhancement operation is performed on the audio signal of each expansion microphone respectively.

[0011] Optionally, performing a speech enhancement operation on the audio signal of the cascaded expansion microphone to obtain a speech-enhanced audio signal includes:

[0012] Performing time domain conversion on the audio signal of the cascade extension microphone to obtain frequency domain information;

[0013] Estimating the interference sound energy based on the frequency domain information;

[0014] The interference sound energy is used to remove the interference part in the audio signal to obtain a speech-enhanced audio signal.

[0015] Optionally, the estimating based on the frequency domain information to obtain the interference sound energy further includes:

[0016] Calculating the frequency domain energy of the speech signal of the cascade extension microphone using the frequency domain information;

[0017] The estimating based on the frequency domain information to obtain the interference sound energy includes:

[0018] Acquire a speech model for speech enhancement, and use the speech model to estimate the proportion of the interference part to the audio signal based on the frequency domain information;

[0019] The interfering sound energy is estimated from the frequency domain energy using the ratio information.

[0020] Optionally, the using the interfering sound energy to remove the interference part in the audio signal to obtain the audio signal after speech enhancement includes:

[0021] Subtracting the interfering sound energy from the frequency domain energy to obtain the audio energy;

[0022] The signal corresponding to the audio energy is determined to be a speech-enhanced audio signal.

[0023] Optionally, the acquiring energy smoothing information of the speech-enhanced audio signal includes:

[0024] Acquire the audio energy of the speech-enhanced audio signal, and acquire the target audio energy of a preset frequency band range in the audio energy;

[0025] An energy smoothing operation is performed on the target audio energy to obtain an energy smoothing spectrum for the audio energy.

[0026] Optionally, the energy smoothing information includes an energy smoothing spectrum for audio energy; and determining a target audio channel for audio output according to the energy smoothing information includes:

[0027] From the energy smoothing spectrum, obtaining audio energy of different audio channels in different frequency bands within a preset frequency band range; wherein different extension microphones in the cascaded extension microphones have corresponding audio channels;

[0028] The audio energies of different audio channels in different frequency bands are compared to determine a target audio channel for audio output among the multiple audio channels.

[0029] Optionally, comparing audio energies of different audio channels in different frequency bands to determine a target audio channel for audio output among the multiple audio channels includes:

[0030] Obtaining audio energy of different audio channels in each frequency band;

[0031] Scoring the audio channels based on the audio energy of the different audio channels in each frequency band;

[0032] A target audio channel is determined from among the plurality of audio channels based on the scoring result.

[0033] Optionally, scoring the audio channels based on audio energy of the different audio channels in each frequency band includes:

[0034] Based on the audio energy of different audio channels in each frequency band, each audio channel is scored in segments;

[0035] The step of determining a target audio channel from a plurality of audio channels based on the scoring result comprises:

[0036] The score values ​​of each audio channel in different frequency bands are accumulated, and an audio channel whose score value exceeds a preset score threshold is determined from the multiple score values ​​as a target audio channel.

[0037] The embodiment of the present application further discloses a device for selecting an audio channel, which relates to a cascaded expansion microphone, and the device comprises:

[0038] A speech enhancement module, used for performing a speech enhancement operation on the audio signal of the cascade extension microphone to obtain a speech-enhanced audio signal;

[0039] An energy smoothing information acquisition module, used to acquire energy smoothing information of the audio signal after speech enhancement;

[0040] The target audio channel determination module is used to determine a target audio channel for audio output according to the energy smoothing information.

[0041] Optionally, the cascaded extension microphone includes at least two extension microphones; and the voice enhancement module includes:

[0042] The speech enhancement submodule is used to obtain the audio signal of each extension microphone in the cascaded extension microphones and perform speech enhancement operations on the audio signal of each extension microphone respectively.

[0043] Optionally, the speech enhancement submodule includes:

[0044] A frequency domain information acquisition unit, configured to perform time domain conversion on the audio signal of the cascade extension microphone to obtain frequency domain information;

[0045] An interference sound energy generating unit, configured to obtain interference sound energy by estimating based on the frequency domain information;

[0046] The audio signal acquisition unit is used to remove the interference part in the audio signal by using the interference sound energy to obtain the audio signal after speech enhancement.

[0047] Optionally, the speech enhancement submodule further includes:

[0048] A frequency domain energy calculation unit, configured to calculate the frequency domain energy of the speech signal of the cascade extension microphone using the frequency domain information;

[0049] The interference sound energy generation unit may include the following subunits:

[0050] A speech model acquisition subunit, used to acquire a speech model for speech enhancement, and to obtain information on the proportion of the interference part to the audio signal by using the speech model based on the frequency domain information;

[0051] The interference sound energy generating subunit is used to estimate the interference sound energy from the frequency domain energy using the ratio information.

[0052] Optionally, the energy smoothing information acquisition module includes:

[0053] A target audio energy acquisition submodule, used to acquire the audio energy of the speech-enhanced audio signal, and to acquire the target audio energy of a preset frequency band range in the audio energy;

[0054] The energy smoothing operation submodule is used to perform an energy smoothing operation on the target audio energy to obtain an energy smoothing spectrum for the audio energy.

[0055] Optionally, the energy smoothing information includes an energy smoothing spectrum for audio energy; and the target audio channel determination module includes:

[0056] An audio energy acquisition submodule, used for acquiring audio energy of different frequency bands of different audio channels in a preset frequency band range from the energy smoothing spectrum; wherein different extension microphones in the cascaded extension microphones have corresponding audio channels;

[0057] The audio energy comparison submodule is used to compare the audio energies of different audio channels in different frequency bands and determine a target audio channel for audio output among multiple audio channels.

[0058] Optionally, the audio energy comparison submodule includes:

[0059] An audio energy acquisition unit, used to acquire audio energy of different audio channels in each frequency band;

[0060] an audio channel scoring unit, configured to score the audio channels based on the audio energy of the different audio channels in each frequency band;

[0061] The target audio channel determining unit is configured to determine a target audio channel from among the multiple audio channels based on the scoring result.

[0062] Optionally, the audio channel scoring unit includes:

[0063] A segment scoring subunit, used for performing segment scoring on each audio channel based on the audio energy of different audio channels in each frequency band;

[0064] The target audio channel determination unit may include the following subunits:

[0065] The target audio channel determination subunit is used to accumulate the score values ​​of each audio channel in different frequency bands, and determine, from multiple score values, an audio channel whose score value exceeds a preset score threshold as a target audio channel.

[0066] An embodiment of the present application further discloses an electronic device, including: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of any one of the audio channel selection methods when executed by the processor.

[0067] The embodiment of the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the audio channel selection methods are implemented.

[0068] The embodiments of the present application include the following advantages:

[0069] In an embodiment of the present application, on the basis of adopting the original audio algorithm, speech enhancement is performed on the audio signal of the cascaded expansion microphone, that is, an additional speech enhancement operation is performed to extract the clear speech part in the audio signal as much as possible, and then energy smoothing information of the audio signal after speech enhancement is obtained, and the best target audio channel among multiple audio channels is judged based on the concise energy smoothing information, so as to adopt the determined target audio channel for audio output, and by determining the best audio channel output based on the energy smoothing information after speech enhancement, the final effect of the cascaded expansion microphone can be enhanced on the basis of integrating the current audio algorithm and the speech enhancement scheme, and the selection of the best audio channel can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 is a flowchart of a method for selecting an audio channel according to an embodiment of the present application;

[0071] Figure 2is a flowchart of another audio channel selection method embodiment of the present application;

[0072] Figure 3 is a schematic diagram of the implementation process of audio channel selection provided in an embodiment of the present application;

[0073] Figure 4 This is an application scenario diagram of audio channel selection provided in an embodiment of the present application;

[0074] Figure 5 It is a structural block diagram of an embodiment of an audio channel selection device of the present application. DETAILED DESCRIPTION

[0075] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0076] To facilitate those skilled in the art to understand the present application, the following explains the terms or nouns involved in the following embodiments of the present application:

[0077] Cascade mode: defined as two or more pickup devices connected to each other in a certain way.

[0078] Extension microphone: a sound pickup device extending from the audio and video equipment terminal host.

[0079] Sound pickup equipment: refers to the equipment responsible for collecting sound, which may include microphones and other devices.

[0080] Intelligent speech enhancement: usually refers to the speech enhancement operation on a single channel, including noise removal, reverberation removal and other operations, so as to obtain a clearer speech part.

[0081] Denoising: Also known as noise elimination, the noisy audio received by the pickup device is removed through an algorithm module to retain the fidelity of the original voice signal as much as possible.

[0082] De-reverberation: The audio with reverberation received by the pickup device is removed through an algorithm module to achieve the effect of the original audio without reverberation.

[0083] Time-to-frequency conversion: used to determine the frequency domain information of a local area of ​​a time-varying signal.

[0084] 3A algorithm: In the embodiments of the present application, it refers to 3A audio processing technology, which is a collective name for three audio algorithms: Acoustic Echo Cancellation (AEC), Automatic Noise Suppression (ANS), and Automatic Gain Control (AGC). In the process of processing audio data, preprocessing is required after the audio data is collected, and the 3A algorithm is a key technology for preprocessing.

[0085] Due to the limited sensitivity of microphones and the fact that long-distance sound pickup will be disturbed by noise and reverberation, large conference rooms usually have the problem of poor far-field sound pickup. In related technologies, not only can the problem of far-field sound pickup be solved from the software level, but also a product-level solution to solve the problem of far-field sound pickup can be proposed at the hardware level, such as improving the sound pickup quality of large conference rooms by cascading expansion microphones. Among them, the cascade expansion microphone method can mainly be manifested as connecting the pickup devices distributed in different positions of the conference room in a cascade manner to achieve the effect of covering the entire conference room, and such dispersed sound pickup devices can be called expansion microphones. The advantage of cascade expansion microphones is that they can effectively cover large / extra-large conference rooms through the extended sound pickup capabilities. Based on the distributed coverage of the space, the sound pickup distance is further shortened. And due to the ease of installation and portability of cascade expansion microphones, the common problem of poor far-field sound pickup in large conference rooms can be solved at a low cost.

[0086] The cascade mode can solve the problem of poor far-field sound pickup that is common in large conference rooms, but the output channel finally used in the cascade mode may not be the best audio channel, that is, the cascade expansion microphone method used in large conference rooms may further cause the problem of how to select the best audio channel as the final output channel. In related technologies, the energy of the audio signal in the cascade expansion microphone is usually used for judgment, but since the audio signal collected by the expansion microphone may be interfered by noise and reverberation, the simple energy judgment method cannot achieve the selection of the best audio channel.

[0087] One of the core ideas of the embodiments of the present application is the proposed automatic audio channel selection scheme, which can mainly determine the best audio channel output based on the energy smoothing information after speech enhancement. It can perform speech enhancement operations on the audio signals of the cascaded extension microphones, and realize speech enhancement for each extension microphone in the cascaded extension microphones by adopting a single-channel intelligent noise reduction and dereverberation scheme, so as to extract the clear speech part in the collected audio signal as much as possible, and then use the concise energy smoothing information to judge the best target audio channel from multiple audio channels, so as to use the determined target audio channel for audio output, that is, the best pickup channel is determined based on the energy of the extracted clear speech part, avoiding The final audio channel is prevented from being excessively disturbed by noise and reverberation, so that the determined target audio channel is more accurate; and based on the method of using energy for judgment after speech enhancement, additional speech enhancement operations can be performed on the audio signal of the cascaded expansion microphone on the basis of the original audio algorithm, that is, it can continue to use the current audio algorithm, the module calculation amount is small, and the final effect of the cascaded expansion microphone is enhanced while integrating the speech enhancement solution. Under the existing algorithm framework, the purpose of fast fusion and high progress of final audio channel selection can be achieved, the anti-interference robustness of the cascaded expansion microphone is improved, and the cascaded expansion microphone can select the target audio channel for the final output audio, that is, the best audio channel.

[0088] Reference Figure 1 , shows a flowchart of an embodiment of a method for selecting an audio channel of the present application, which involves cascading expansion microphones and may specifically include the following steps:

[0089] Step 101, performing a speech enhancement operation on the audio signal of the cascade extension microphone to obtain a speech-enhanced audio signal;

[0090] Far-field sound pickup in a large space often has poor sound pickup quality due to the limited sensitivity of the microphone and the fact that long-distance sound pickup is inevitably interfered by noise and reverberation. Cascaded expansion microphones can shorten the sound pickup distance based on distributed coverage of the space to solve the problem of far-field sound pickup. However, even if the sound pickup distance is reduced, the cascaded expansion microphone will still be interfered by noise and reverberation during the sound pickup process, and thus when the cascaded expansion microphone selects the target audio channel for the final audio output, its selection of the target audio channel will also be interfered by noise and reverberation.

[0091] In order to prevent the cascaded expansion microphone from being disturbed by too much noise and reverberation when selecting the target audio channel, i.e., the optimal audio channel, for the final audio output, a speech enhancement operation may be performed on the audio signal of the cascaded expansion microphone before determining the target audio channel, so as to extract the clear speech part of the collected audio signal as much as possible, i.e., to reduce the noise and reverberation interference in the audio signal as much as possible.

[0092] Among them, the cascaded expansion microphone serves as a sound pickup device, and the audio signal of the cascaded expansion microphone can be an audio signal picked up or collected by the cascaded expansion microphone. This audio signal is collected by the sound pickup device extending from the audio and video device terminal host, and can include an electrical signal with regular frequency and amplitude changes of sound waves with voice, music and sound effects, such as a voice signal.

[0093] The speech enhancement operation performed on the audio signal can be expressed as extracting effective speech signals from the noise background when the speech signal is interfered by various noises, suppressing and reducing the noise and reverberation interference, that is, extracting the pure original speech part from the noisy audio as much as possible. Then the speech enhancement operation performed is essentially a noise reduction and dereverberation operation. The audio signal before speech enhancement may include the signal of the speech part, and the signal containing noise or other interference parts, and the obtained audio signal after speech enhancement usually contains the proposed fresh speech part signal, that is, the remaining audio signal after the noise reduction and dereverberation operations are performed on the audio signal.

[0094] Exemplarily, assuming a real-time voice communication scenario, such as a conference room scenario, the noise reduction operation performed can be manifested as removing common noises in the environment and conference room, such as the sound generated by the operation of the air conditioner, the clicking of the keyboard and mouse or other sounds generated, the knocking of the table, the sound of water cups, tables and chairs, etc.; the dereverberation operation performed can be manifested as removing the reverberation effect caused by the enclosed space of the conference room, such as the sense of spatial distance. Both noise and reverberation will cause the quality of speech to drop sharply. The noise reduction and dereverberation operations can eliminate these two effects, making the audio speech clearer and improving the speech quality.

[0095] In practical applications, cascaded extension microphones refer to two or more sound pickup devices extending from an audio and video device terminal host that are interconnected in a certain manner, that is, the cascaded extension microphone may include at least two extension microphones, and each extension microphone has a corresponding different sound pickup channel, and the sound pickup channel correspondingly has an audio channel for audio output. In order to facilitate subsequent channel selection from the audio channels of multiple extension microphones, when performing a voice enhancement operation on the audio signal, the audio signal of each extension microphone in the cascaded extension microphone can be obtained, and a voice enhancement operation is performed on the audio signal of each extension microphone, that is, noise reduction and dereverberation operations are performed on the audio signal collected by each extension microphone to ensure that the audio signal of each extension microphone contains a pure original voice part.

[0096] Step 102, obtaining energy smoothing information of the audio signal after speech enhancement;

[0097] The selection of the target audio channel for the final audio output can be achieved by determining the energy of the audio signal after speech enhancement in different extension microphones. At this time, energy smoothing information for the audio signal after speech enhancement can be obtained, so that the best target audio channel can be determined from multiple audio channels based on the concise energy smoothing information.

[0098] In practical applications, the energy smoothing information of the audio signal obtained may be an energy smoothing spectrum of the audio signal, wherein the energy of the audio signal is embodied in the form of an energy spectrum, which may be obtained based on the amplitude processing of the audio signal, and mainly may be obtained by performing a square operation on the amplitude spectrum of the audio signal.

[0099] In a specific implementation, for the energy smoothing spectrum of the audio signal, the energy spectrum of the audio signal can be obtained by performing an energy smoothing operation on the energy spectrum, and then the energy spectrum after the amplitude of the audio signal in the energy spectrum is smoothed can be called the energy smoothing spectrum. The smoothing operation can also be called filtering or combined as smoothing filtering, that is, the amplitude of the audio signal in the energy spectrum is subjected to low-frequency enhanced spatial domain filtering, so as to further filter the noise after the audio signal is speech enhanced, and further reduce the selection of the target audio channel by noise and reverberation.

[0100] Step 103: Determine a target audio channel for audio output according to the energy smoothing information.

[0101] After performing speech enhancement operation on the audio signal of the cascaded extension microphone, the best target audio channel can be determined from multiple audio channels based on concise energy smoothing information, so as to adopt the determined target audio channel for audio output, that is, the best pickup channel can be determined based on the energy of the extracted clear speech part, avoiding excessive interference of noise and reverberation on the finally determined audio channel, so that the determined target audio channel is more accurate.

[0102] Among them, the cascaded extension microphone includes at least two extension microphones, each of which has a corresponding different sound pickup channel, so the sound pickup channel correspondingly has an audio channel for audio output, that is, each extension microphone has its corresponding audio channel. In practical applications, after the extension microphone as a sound pickup device picks up the audio of the sound pickup object in the space, it is necessary to output the picked-up audio signal to the other end of the audio and video device terminal host. At this time, it is necessary to determine the audio channel used when outputting the audio to the other end, that is, to specifically determine the audio channel of a certain extension microphone in the cascaded extension microphone as the target audio channel for audio output, which can be specifically manifested as the selection of the audio channel of a certain extension microphone, so that after multiple cascaded extension microphones pick up the audio, the audio channel of a certain extension microphone determined is used for audio transmission.

[0103] It should be noted that when the audio channel of a determined extension microphone is used for audio output, the output audio may be an audio signal collected by the extension microphone through the sound pickup channel, and usually the extension microphone may be an extension microphone with a shorter sound pickup distance from the sound pickup object. In other cases, the determination of the extension microphone may also be related to the cascading method of the extension microphones. For example, the extension microphone with a short sound pickup distance from the sound pickup object may not be selected when the combination is cascaded. This is not limited in the embodiments of the present application.

[0104] In an embodiment of the present application, on the basis of adopting the original audio algorithm, speech enhancement is performed on the audio signal of the cascaded expansion microphone, that is, an additional speech enhancement operation is performed to extract the clear speech part in the audio signal as much as possible, and then energy smoothing information of the audio signal after speech enhancement is obtained, and the best target audio channel among multiple audio channels is judged based on the concise energy smoothing information, so as to adopt the determined target audio channel for audio output, and by determining the best audio channel output based on the energy smoothing information after speech enhancement, the final effect of the cascaded expansion microphone can be enhanced on the basis of integrating the current audio algorithm and the speech enhancement scheme, and the selection of the best audio channel can be achieved.

[0105] Reference Figure 2 , shows a flowchart of another audio channel selection method embodiment of the present application, involving cascaded expansion microphones, which may specifically include the following steps:

[0106] Step 201, extracting audio energy from each extension microphone;

[0107] At present, the audio algorithms in real-time communication devices (such as the classic 3A algorithm) are implemented in the frequency domain. Audio energy can basically refer to the frequency domain energy. This energy can be used to reflect the audio energy picked up by each channel of the expansion microphone. When the target audio channel for outputting the audio is determined later, that is, when selecting the audio channel of a certain expansion microphone, it can be determined based on the audio energy of each expansion microphone in the cascaded expansion microphone.

[0108] The audio energy of each expansion microphone is the energy of the audio signal after speech enhancement in each expansion microphone, that is, the energy of the audio signal after noise reduction and reverberation processing. The speech enhancement solution can be integrated on the basis of the original audio algorithm. The module has a small amount of calculation and can achieve the purpose of rapid fusion under the existing algorithm framework, thereby improving the anti-interference robustness of the cascade expansion microphone. The audio energy obtained at this time does not include the energy of the interference part, so as to avoid the cascade expansion microphone from being interfered by too much noise and reverberation when selecting the target audio channel for the final output audio, that is, the best audio channel.

[0109] In one embodiment of the present application, it is first necessary to obtain the audio signals of each expansion microphone in the cascaded expansion microphone, and perform speech enhancement operations on the audio signals of each expansion microphone respectively to obtain the speech-enhanced audio signal, so as to extract the clear speech part in the collected audio signal as much as possible, that is, to reduce the noise and reverberation interference in the audio signal as much as possible, and determine the audio energy in each expansion microphone, wherein the signal corresponding to the audio energy can be the speech-enhanced audio signal.

[0110] In practical applications, such as Figure 3 As shown, the audio signal of the cascade expansion microphone can be transformed in the time domain to obtain frequency domain information, wherein the time domain transformation is mainly used to determine the frequency domain information of the local area of ​​the time-varying signal, and the frequency domain information is mainly used to represent the signal distribution of the speech part and other noise or interference parts in the speech signal and the frequency information of the signal, etc. Then, the interference sound energy can be estimated based on the frequency domain information, so that the interference sound energy can be used to remove the interference part in the audio signal to obtain the audio signal after speech enhancement.

[0111] like Figure 3 As shown, in addition to estimating the interfering sound energy based on the frequency domain information, the frequency domain information can also be used to calculate the frequency domain energy of the audio signal for the cascaded expansion microphone, wherein since the frequency domain information can be used to represent the signal distribution of the speech part and other noise or interference parts in the audio signal and the frequency information of the signal, the calculated frequency domain energy may include the energy of the speech part in the audio signal and the energy of the interfering sound, etc.

[0112] In practical applications, in order to obtain the interfering sound energy that needs to be subtracted, the process of estimating the interfering sound energy based on the frequency domain information can be specifically implemented based on the intelligent speech enhancement module. The intelligent speech enhancement module can be expressed as a speech model for speech enhancement, and this speech model is a single-channel speech enhancement model implemented by deep learning / neural network. At this time, a speech model for speech enhancement can be obtained, and the obtained speech model can be a neural network model obtained after training based on a large amount of data. The large amount of data used for training can be sample data for frequency domain information. This sample data can contain information about the speech part and the interference part in a certain frequency domain information. Then, the proportion of the interference part information in the frequency domain information can be determined.

[0113] When training a speech model for speech enhancement, sample data of frequency domain information can be used as input of the model, and the proportion of the interference part information to the frequency domain information can be used as output for training; when using the speech model for speech enhancement, the speech model can be used to estimate the proportion of the interference part to the audio signal based on the frequency domain information, that is, the frequency domain information used to estimate the energy of the interfering sound can be input into the speech model, and the speech model can accurately estimate the proportion information of the interfering part, such as the proportion of on-site noise and reverberation in the picked up audio signal (such as maskratio ratio masking), so as to use the obtained proportion information to estimate the audio energy of the speech part and the interfering sound energy of the interfering part (including noise and reverberation) from the frequency domain energy.

[0114] Since the signal corresponding to the audio energy is the audio signal after speech enhancement, the energy of the speech part in the audio signal can be extracted, that is, the audio energy in each extension microphone can be determined. Then, after the interference sound energy is estimated based on the frequency domain information using the speech model, the interference sound energy can be subtracted from the frequency domain energy to obtain the audio energy, and the signal corresponding to the audio energy can be determined to be the audio signal after speech enhancement.

[0115] Step 202, performing energy smoothing operation on target audio energy in a preset frequency band range in each extension microphone to obtain energy smoothing information for the cascaded extension microphones;

[0116] After obtaining the audio energy in each of the cascaded extension microphones, the audio energy in each of the extension microphones, that is, the energy of the audio signal after speech enhancement, can be determined to select the target audio channel for the final audio output. At this time, energy smoothing information of the audio energy can be obtained so that the best target audio channel can be determined from multiple audio channels based on the concise energy smoothing information.

[0117] In practical applications, the energy smoothing information of the audio signal obtained can be an energy smoothing spectrum for the audio signal. The audio energy can be reflected in the form of an energy spectrum. For the energy smoothing spectrum of the audio signal, the energy spectrum of the audio signal can be obtained by performing an energy smoothing operation on the energy spectrum. Then, the energy spectrum after the amplitude of the audio signal in the energy spectrum is smoothed can be called an energy smoothing spectrum. The smoothing operation can also be called filtering or smoothing filtering, that is, the amplitude of the audio signal in the energy spectrum is subjected to spatial domain filtering for low-frequency enhancement, so as to further filter the noise after the audio signal is speech enhanced, and further reduce the selection of the target audio channel by noise and reverberation.

[0118] In the process of smoothing the audio energy, not all frequency bands of the audio energy are smoothed. Since the entire algorithm framework is performed in the frequency domain, each frequency band is independent of each other. Some frequency bands may be dominated by interference sounds, such as high-frequency bands. Figure 3 As shown, it mainly performs energy smoothing on specific frequency bands in each extension microphone, that is, performs energy smoothing operation on target audio energy in a preset frequency band range.

[0119] Specifically, the audio energy of the audio signal after speech enhancement can be obtained, and the target audio energy of a preset frequency band range in the audio energy can be obtained. This preset frequency band range can be expressed as a frequency band where speech energy is concentrated, for example, 500 to 3500 Hz. Then, an energy smoothing operation can be performed on the target audio energy to obtain an energy smoothed spectrum for the audio energy. At this time, the energy smoothed spectrum obtained is a frequency band where the audio energy is concentrated, so as to focus more concentratedly on the part of the audio signal, thereby further reducing the selection of the target audio channel by noise and reverberation.

[0120] Step 203: using the energy smoothing information of the cascade extension microphone, determine a target audio channel from multiple audio channels.

[0121] After performing speech enhancement operation on the audio signal of the cascaded extension microphone, the best target audio channel can be determined from multiple audio channels based on concise energy smoothing information, so as to adopt the determined target audio channel for audio output, that is, the best pickup channel can be determined based on the energy of the extracted clear speech part, avoiding excessive interference of noise and reverberation on the finally determined audio channel, so that the determined target audio channel is more accurate.

[0122] Each extended microphone has a corresponding different sound pickup channel, and the sound pickup channel has an audio channel for audio output, that is, each extended microphone has its corresponding audio channel. Each extended microphone can pick up audio signals of different frequency bands, and different channels have different audio energies in different frequency bands, that is, different channel energies.

[0123] When determining the target audio channel for the final audio output, the audio energy of different audio channels in different frequency bands within the preset frequency band range may be obtained from the energy smoothing spectrum, and then the target audio channel for the audio output among the multiple audio channels may be determined by comparing the audio energy of different audio channels in different frequency bands, i.e., channel energy. The comparison of channel energy may be specifically performed by scoring the audio channels based on the audio energy of different audio channels in each frequency band, and determining the target audio channel from the multiple audio channels based on the scoring result.

[0124] The scoring of the audio energy of different audio channels in each frequency band can be achieved by performing segmented scoring on each audio channel based on the audio energy of different audio channels in each frequency band, and the determination of the target audio channel can be achieved by accumulating the score values ​​of each audio channel in different frequency bands, and determining the audio channel with a score value exceeding a preset score threshold or the highest score value from multiple score values ​​as the target audio channel.

[0125] For example, Figure 3 As shown, in the process of scoring the audio channels, the audio energy of different audio channels can be compared in each frequency band, and the audio channel with the highest audio energy value in the same frequency band can be voted or scored, that is, for any frequency band, the audio channel with the highest audio energy value can be used as the best channel in this frequency band, and voted or scored, and then the votes or scores of each audio channel in all frequency bands can be accumulated, and the audio channel with a number of votes or a score greater than a preset score threshold or with the highest number of votes is the best channel, that is, the target audio channel for outputting audio in the end. It should be noted that the preset score threshold can be set according to actual needs, and this is not limited in the embodiments of the present application.

[0126] Among them, the cascaded extension microphone includes at least two extension microphones, each of which has a corresponding different sound pickup channel, so the sound pickup channel correspondingly has an audio channel for audio output, that is, each extension microphone has its corresponding audio channel. In practical applications, after the extension microphone as a sound pickup device picks up the audio of the sound pickup object in the space, it is necessary to output the picked-up audio signal to the other end of the audio and video device terminal host. At this time, it is necessary to determine the audio channel used when outputting the audio to the other end, that is, to specifically determine the audio channel of a certain extension microphone in the cascaded extension microphone as the target audio channel for audio output, which can be specifically manifested as the selection of the audio channel of a certain extension microphone, so that after the multiple cascaded extension microphones pick up the audio, the audio channel of the determined extension microphone is used for playback.

[0127] Illustratively, when the audio channel of a determined extension microphone is used for audio output, the output audio may be an audio signal collected by the extension microphone through the sound pickup channel, and usually the extension microphone may be an extension microphone with a shorter sound pickup distance from a sound pickup object. In other cases, the determination of the extension microphone may also be related to the cascading method of the extension microphones. For example, the extension microphone with a short sound pickup distance from the sound pickup object may not be selected when the combination is cascaded. This is not limited in this regard by the embodiments of the present application.

[0128] In an embodiment of the present application, on the basis of adopting the original audio algorithm, speech enhancement is performed on the audio signal of the cascaded expansion microphone, that is, an additional speech enhancement operation is performed to extract the clear speech part in the audio signal as much as possible, and then energy smoothing information of the audio signal after speech enhancement is obtained, and the best target audio channel among multiple audio channels is judged based on the concise energy smoothing information, so as to adopt the determined target audio channel for audio output, and by determining the best audio channel output based on the energy smoothing information after speech enhancement, the final effect of the cascaded expansion microphone can be enhanced on the basis of integrating the current audio algorithm and the speech enhancement scheme, and the selection of the best audio channel can be achieved.

[0129] Reference Figure 4 , shows an application scenario diagram of audio channel selection provided by an embodiment of the present application, which can be applied in real-time voice communication scenarios, such as conference room application scenarios. In this conference room application scenario, a real-time communication device is involved. This real-time communication device can be an integrated communication device, including a host device and a cascaded expansion microphone. The cascaded expansion microphone can include at least two expansion microphones, usually 2-5, depending on the actual situation. The expansion microphone used can be a microphone device or a microphone array, and the cascade mode of the expansion microphone can be wired or wireless, so as to facilitate the distributed coverage of the space and further shorten the pickup distance.

[0130] The applied conference scene can be presented as a conference scene covered by cascaded expansion microphones, and the communication all-in-one device can be specifically presented as a conference audio device. Among them, the expansion microphone, as a sound pickup device, can pick up the audio signal in the conference room space. At this time, the picked up audio signal can include the voice part of the picked up object, such as the speaker's voice, and the interference part (including noise and reverberation) in the conference room. After the expansion microphone picks up these audio signals, the picked up audio signal can be processed by the host device in the all-in-one device. The processing process includes the pre-processing process performed by the 3A algorithm and the audio codec processing process, so as to transmit the processed audio information to the other end of the meeting through the network. However, when the expansion microphone does not process the audio signal well, especially in the case of large environmental noise, it will directly affect the effect of picking up the voice of the picked up object, resulting in the other end of the meeting being unable to hear the other party's voice clearly. At this time, it is necessary to select the audio channel used for audio output to the other end of the meeting. It should be noted that the audio collection operation of the sound collection device on the sound collection objects in the conference space is performed with the knowledge and permission of all the sound collection objects in the conference space.

[0131] The selection of the audio channel used for audio output to the other end of the conference can be specifically achieved by the host device determining the target audio channel. The starting point of the host device in selecting the target audio channel is not to select the audio channel close to the noise source, but to select the audio channel close to the picked-up object. However, when the noise energy generated by the noise source may be greater than the audio energy of the picked-up object, the host device is required to extract the energy of the voice part of the audio signal (i.e., the audio energy), and then compare the extracted audio energy in different frequency bands, and then determine the target audio channel that can reflect high voice quality.

[0132] Specifically, the extraction of audio energy can be manifested as a speech enhancement operation performed on the audio signal. The speech enhancement operation performed is essentially a noise reduction and dereverberation operation. In the scenario of a conference room, the noise reduction operation performed can be manifested as removing common noises in the environment and conference room, such as the sound generated by the operation of the air conditioner, the clicking sound of the keyboard and mouse, or other sounds generated, the sound of knocking on the table, the sound of water cups, tables and chairs, etc.; the dereverberation operation performed can be manifested as removing the reverberation effect caused by the enclosed space of the conference room, such as the sense of spatial distance. Both noise and reverberation will cause the quality of speech to drop sharply. The noise reduction and dereverberation operations can eliminate these two effects, making the audio speech clearer and improving the speech quality.

[0133] Then, the energy smoothing information of the extracted audio energy can be used to determine the target audio channel from multiple audio channels. Specifically, the audio energy of different audio channels can be compared in each frequency band, and the audio channel with the highest audio energy value in the same frequency band can be voted or scored, that is, for any frequency band, the audio channel with the highest audio energy value can be used as the best channel in this frequency band, and voted or scored, and then the votes or scores of each audio channel in all frequency bands can be accumulated, and the audio channel with a number of votes or scores greater than a preset score threshold or with the highest number of votes is the best channel, that is, the target audio channel for outputting audio in the end.

[0134] In actual applications, after the expansion microphone as a sound pickup device picks up the audio of the sound pickup object in the space, the picked up audio signal needs to be output to the other end of the audio and video device terminal host. At this time, it is necessary to determine the audio channel used to output the audio to the other end, that is, to specifically determine the audio channel of a certain expansion microphone in the cascaded expansion microphones as the target audio channel for audio output. This can be specifically manifested as the selection of the audio channel of a certain expansion microphone, so that after multiple cascaded expansion microphones pick up the audio, the audio channel of the determined expansion microphone is used for audio transmission.

[0135] For example, when the audio channel of a certain extended microphone is used for audio output, the output audio may be an audio signal collected by the extended microphone through the sound pickup channel, and usually the extended microphone may be an extended microphone with a shorter sound pickup distance from the sound pickup object, for example Figure 4As shown, the extension microphone that is shorter than the pickup object is the extension microphone B, and the extension microphone that is longer than the pickup object is the extension microphone C. The extension microphone B is closer to the pickup object. The closer the pickup distance is, the higher the signal-to-noise ratio is. The stronger the direct audio sound of the pickup object is than the reverberation sound, and the higher the voice quality / clarity is. The audio of the pickup channel of the extension microphone C is mostly nearby noise source signals, such as keyboard and mouse sounds. The energy of the noise source, that is, the interference sound energy, may be greater than the audio energy of the pickup object. The voice quality transmitted to the other end will be very poor. In this case, the target audio channel determined may be the extension microphone B, not the extension microphone C.

[0136] It should be noted that the automatic audio channel selection scheme proposed in the embodiment of the present application can be applied not only in conference scenarios, but also in any scenario where audio is picked up, and is not limited to the scheme of using cascaded expansion microphones for picking up audio. The embodiment of the present application does not impose any restrictions on this.

[0137] In the embodiment of the present application, the best audio channel output is determined based on the energy smoothing information after speech enhancement, which can perform speech enhancement operation on the audio signal of the cascaded extension microphone, and realize speech enhancement for each extension microphone in the cascaded extension microphone by adopting a single-channel intelligent noise reduction and dereverberation solution, so as to extract the clear speech part in the collected audio signal as much as possible, and then use the concise energy smoothing information to judge the best target audio channel from multiple audio channels, so as to use the determined target audio channel for audio output, that is, the best pickup channel is determined based on the energy of the extracted clear speech part, so as to avoid the final determined audio channel being affected by Excessive interference from noise and reverberation makes the determined target audio channel more accurate; and based on the method of using energy for judgment after speech enhancement, additional speech enhancement operations can be performed on the audio signal of the cascaded expansion microphone on the basis of the original audio algorithm, that is, it can continue to use the current audio algorithm, the module calculation amount is small, and the final effect of the cascaded expansion microphone is enhanced while integrating the speech enhancement solution. Under the existing algorithm framework, the purpose of fast fusion and high progress in the final audio channel selection can be achieved, the anti-interference robustness of the cascaded expansion microphone is improved, and the cascaded expansion microphone can select the target audio channel for the final output audio, that is, the best audio channel.

[0138] It should be noted that, for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the described order of actions, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.

[0139] Reference Figure 5 , shows a structural block diagram of an embodiment of an audio channel selection device of the present application, which involves cascaded expansion microphones and may specifically include the following modules:

[0140] A speech enhancement module 501 is used to perform a speech enhancement operation on the audio signal of the cascade extension microphone to obtain a speech-enhanced audio signal;

[0141] An energy smoothing information acquisition module 502 is used to acquire energy smoothing information of the audio signal after speech enhancement;

[0142] The target audio channel determination module 503 is configured to determine a target audio channel for audio output according to the energy smoothing information.

[0143] In an embodiment of the present application, the cascaded extension microphone includes at least two extension microphones; the speech enhancement module 501 may include the following submodules:

[0144] The speech enhancement submodule is used to obtain the audio signal of each extension microphone in the cascaded extension microphones and perform speech enhancement operations on the audio signal of each extension microphone respectively.

[0145] In one embodiment of the present application, the speech enhancement submodule may include the following units:

[0146] A frequency domain information acquisition unit, configured to perform time domain conversion on the audio signal of the cascade extension microphone to obtain frequency domain information;

[0147] An interference sound energy generating unit, configured to obtain interference sound energy by estimating based on the frequency domain information;

[0148] The audio signal acquisition unit is used to remove the interference part in the audio signal by using the interference sound energy to obtain the audio signal after speech enhancement.

[0149] In one embodiment of the present application, the speech enhancement submodule may further include the following units:

[0150] A frequency domain energy calculation unit, configured to calculate the frequency domain energy of the speech signal of the cascade extension microphone using the frequency domain information;

[0151] The interference sound energy generation unit may include the following subunits:

[0152] A speech model acquisition subunit, used to acquire a speech model for speech enhancement, and to obtain information on the proportion of the interference part to the audio signal by using the speech model based on the frequency domain information;

[0153] The interference sound energy generating subunit is used to estimate the interference sound energy from the frequency domain energy using the ratio information.

[0154] In one embodiment of the present application, the energy smoothing information acquisition module 502 may include the following submodules:

[0155] A target audio energy acquisition submodule, used to acquire the audio energy of the speech-enhanced audio signal, and to acquire the target audio energy of a preset frequency band range in the audio energy;

[0156] The energy smoothing operation submodule is used to perform an energy smoothing operation on the target audio energy to obtain an energy smoothing spectrum for the audio energy.

[0157] In an embodiment of the present application, the energy smoothing information includes an energy smoothing spectrum for audio energy; the target audio channel determination module 503 may include the following submodules:

[0158] An audio energy acquisition submodule, used for acquiring audio energy of different frequency bands of different audio channels in a preset frequency band range from the energy smoothing spectrum; wherein different extension microphones in the cascaded extension microphones have corresponding audio channels;

[0159] The audio energy comparison submodule is used to compare the audio energies of different audio channels in different frequency bands and determine a target audio channel for audio output among multiple audio channels.

[0160] In one embodiment of the present application, the audio energy comparison submodule may include the following units:

[0161] An audio energy acquisition unit, used to acquire audio energy of different audio channels in each frequency band;

[0162] an audio channel scoring unit, configured to score the audio channels based on the audio energy of the different audio channels in each frequency band;

[0163] The target audio channel determining unit is configured to determine a target audio channel from among the multiple audio channels based on the scoring result.

[0164] In one embodiment of the present application, the audio channel scoring unit may include the following subunits:

[0165] A segment scoring subunit, used for performing segment scoring on each audio channel based on the audio energy of different audio channels in each frequency band;

[0166] The target audio channel determination unit may include the following subunits:

[0167] The target audio channel determination subunit is used to accumulate the score values ​​of each audio channel in different frequency bands, and determine, from multiple score values, an audio channel whose score value exceeds a preset score threshold as a target audio channel.

[0168] In an embodiment of the present application, on the basis of adopting the original audio algorithm, speech enhancement is performed on the audio signal of the cascaded expansion microphone, that is, an additional speech enhancement operation is performed to extract the clear speech part in the audio signal as much as possible, and then energy smoothing information of the audio signal after speech enhancement is obtained, and the best target audio channel among multiple audio channels is judged based on the concise energy smoothing information, so as to adopt the determined target audio channel for audio output, and by determining the best audio channel output based on the energy smoothing information after speech enhancement, the final effect of the cascaded expansion microphone can be enhanced on the basis of integrating the current audio algorithm and the speech enhancement scheme, and the selection of the best audio channel can be achieved.

[0169] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0170] The present application also provides an electronic device, including:

[0171] The present invention comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, each process of the above-mentioned audio channel selection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0172] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, each process of the above-mentioned audio channel selection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0173] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0174] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0175] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0176] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0177] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0178] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0179] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0180] The above is a detailed introduction to an audio channel selection method, an audio channel selection device, a corresponding electronic device and a corresponding computer storage medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for a person skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for selecting an audio channel, characterized in that: It relates to a cascade expansion microphone, and the method comprises: Performing a speech enhancement operation on the audio signal of the cascade expansion microphone to obtain a speech-enhanced audio signal; the speech-enhanced audio signal is obtained by removing an interference part of the audio signal based on interference sound energy, wherein the interference sound energy is estimated based on frequency domain information, and the frequency domain information is obtained based on time domain conversion of the audio signal of the cascade expansion microphone; Acquire energy smoothing information of the audio signal after speech enhancement; the energy smoothing information is a frequency band where audio energy is concentrated; A target audio channel for audio output is determined according to the energy smoothing information.

2. The method according to claim 1, characterized in that The cascaded expansion microphone includes at least two expansion microphones; the voice enhancement operation on the audio signal of the cascaded expansion microphone includes: The audio signal of each expansion microphone in the cascaded expansion microphone is obtained, and a voice enhancement operation is performed on the audio signal of each expansion microphone respectively.

3. The method according to claim 1, characterized in that The method further comprises: Calculating the frequency domain energy of the speech signal of the cascade extension microphone using the frequency domain information; Acquire a speech model for speech enhancement, and use the speech model to estimate the proportion of the interference part to the audio signal based on the frequency domain information; The interfering sound energy is estimated from the frequency domain energy using the ratio information.

4. The method according to claim 3, characterized in that The performing a speech enhancement operation on the audio signal of the cascade extension microphone to obtain a speech-enhanced audio signal includes: Subtracting the interfering sound energy from the frequency domain energy to obtain the audio energy; The signal corresponding to the audio energy is determined to be a speech-enhanced audio signal.

5. The method according to claim 1, characterized in that: The obtaining of energy smoothing information of the audio signal after speech enhancement includes: Acquire the audio energy of the speech-enhanced audio signal, and acquire the target audio energy of a preset frequency band range in the audio energy; An energy smoothing operation is performed on the target audio energy to obtain an energy smoothing spectrum for the audio energy.

6. The method according to claim 1 or 5, characterized in that: The energy smoothing information includes an energy smoothing spectrum for audio energy; and determining a target audio channel for audio output according to the energy smoothing information includes: From the energy smoothing spectrum, obtaining audio energy of different audio channels in different frequency bands within a preset frequency band range; wherein different extension microphones in the cascaded extension microphones have corresponding audio channels; The audio energies of different audio channels in different frequency bands are compared to determine a target audio channel for audio output among the multiple audio channels.

7. The method according to claim 6, characterized in that The step of comparing audio energies of different audio channels in different frequency bands to determine a target audio channel for audio output among the multiple audio channels includes: Obtaining audio energy of different audio channels in each frequency band; Scoring the audio channels based on the audio energy of the different audio channels in each frequency band; A target audio channel is determined from among the plurality of audio channels based on the scoring result.

8. The method according to claim 7, characterized in that Scoring the audio channels based on the audio energy of the different audio channels in each frequency band includes: Based on the audio energy of different audio channels in each frequency band, each audio channel is scored in segments; The step of determining a target audio channel from a plurality of audio channels based on the scoring result comprises: The score values ​​of each audio channel in different frequency bands are accumulated, and an audio channel whose score value exceeds a preset score threshold is determined from the multiple score values ​​as a target audio channel.

9. A device for selecting an audio channel, characterized in that: It relates to a cascade expansion microphone, and the device comprises: A speech enhancement module, configured to perform a speech enhancement operation on the audio signal of the cascade expansion microphone to obtain a speech-enhanced audio signal; the speech-enhanced audio signal is obtained by removing an interference part of the audio signal using interference sound energy, wherein the interference sound energy is estimated based on frequency domain information, and the frequency domain information is obtained based on time domain conversion of the audio signal of the cascade expansion microphone; An energy smoothing information acquisition module, used to acquire energy smoothing information of the audio signal after speech enhancement; the energy smoothing information is a frequency band where audio energy is concentrated; The target audio channel determination module is used to determine a target audio channel for audio output according to the energy smoothing information.

10. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for selecting an audio channel according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for selecting an audio channel according to any one of claims 1 to 8 are implemented.

Images