Zone-based sound pickup method and apparatus, device, and readable storage medium
By employing a deep learning-based regional sound pickup method, utilizing microphone arrays and pre-trained models, the problem of traditional beamforming technology being unable to distinguish between speakers and interfering voices was solved, enabling clear transmission of voice signals in remote conferences and improving conference quality.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- GUANGZHOU SHIYUAN ELECTRONICS CO LTD
- Filing Date
- 2026-01-14
- Publication Date
- 2026-07-23
AI Technical Summary
Traditional beamforming-based area pickup technology cannot accurately distinguish between the speaker's voice signal and interfering voices, resulting in remote participants hearing noisy sounds and affecting the quality of remote meetings.
A deep learning-based regional sound pickup method is adopted, which uses a microphone array for sound pickup. The complex spectral weights of the microphone signal are determined by a pre-trained regional sound pickup model, which preserves the target signal within the target pickup area, suppresses reverberation effects, and shields interfering human voices and noise.
It achieves accurate estimation of speech signals within the target pickup area, shields against noise and interference with human voices, and improves the clarity and quality of speech in remote conferences.
Smart Images
Figure CN2026072650_23072026_PF_FP_ABST
Abstract
Description
Area pickup method, apparatus, equipment and readable storage medium
[0001] This application claims priority to Chinese Patent Application No. 2025100565611, filed on January 14, 2025, entitled “A method, apparatus, device and readable storage medium for area pickup”, the entire contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of voice acquisition technology, and in particular to a method, apparatus, device, and readable storage medium for regional voice pickup. Background Technology
[0003] With the rapid development of internet technology and the limitations of physical venues, remote conferencing is being used more and more widely.
[0004] During the meeting, the local audio pickup device uses microphone array beamforming technology to enhance the picked-up audio signal and send the enhanced audio signal to the remote end, enabling remote participants to better listen to and understand the speaker's speech.
[0005] However, traditional beamforming-based area pickup cannot accurately distinguish between the speaker's voice signal and interfering voices, resulting in remote participants hearing noisy sounds and seriously affecting the quality of remote meetings. Summary of the Invention
[0006] This application provides a method, apparatus, device, and readable storage medium for regional sound pickup. The electronic device uses deep learning to shield noise and interfering human voices outside the pickup area, thereby estimating the speech signal of the target pickup area, so that remote participants can hear accurate speech signals and achieve the goal of improving the quality of remote meetings.
[0007] In a first aspect, embodiments of this application provide a regional sound pickup method, applied to an electronic device having a microphone array, the method comprising:
[0008] The microphone signals of each microphone are obtained by using each microphone in the microphone array to pick up sound. The microphone signals are noisy reverberation signals. The noisy reverberation signals are formed by adding reverberation effects to the target signal in the target pickup area of the target space, interfering with human voices and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voices is located outside the target pickup area, and the sound source of noise is located at any position in the target space.
[0009] Determine the complex spectrum of the signals from each microphone;
[0010] The complex spectrum of each microphone signal is input into a pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise.
[0011] The target signal within the target pickup area is estimated based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
[0012] Secondly, embodiments of this application provide a regional sound pickup device, which is integrated on an electronic device having a microphone array, and the device includes:
[0013] The sound pickup module is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.
[0014] The determination module is used to determine the complex spectrum of each microphone signal;
[0015] The processing module is used to input the complex spectrum of each microphone signal into a pre-trained regional sound pickup model, so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise.
[0016] An estimation module is used to estimate the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
[0017] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the electronic device implements the method described in the first aspect or various possible implementations of the first aspect.
[0018] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the method described in the first aspect or various possible implementations of the first aspect.
[0019] Fifthly, embodiments of this application provide a computer program product comprising a computing program, wherein when the computer program is executed by a processor, it implements the method described in the first aspect or various possible implementations of the first aspect.
[0020] The regional sound pickup method, apparatus, device, and readable storage medium provided in this application embodiment include an electronic device with a microphone array and a pre-trained regional sound pickup model deployed on the electronic device. The electronic device uses each microphone in the microphone array to pick up the microphone signal, which is a noisy reverberation signal. Then, the electronic device determines the complex spectrum of each microphone signal, inputs the complex spectrum into the regional sound pickup model to obtain the weights of each complex spectrum, and estimates the target signal within the target pickup area based on the complex spectrum of each microphone and its weights. Using this scheme, the electronic device uses the regional sound pickup model to determine the weights of the complex spectrum of each microphone signal. These weights are used to retain the target signal within the target pickup area, suppress reverberation, shield noise in the target space, and block interfering human voices outside the target pickup area. This makes the estimated target signal closer to the actual target signal, achieving the goal of accurately distinguishing noise, the target signal within the target area, and interfering human voices outside the target area. This allows remote participants to hear accurate speech signals, thereby improving the quality of remote meetings. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 is a schematic diagram of a sound pickup area pointing directly forward;
[0023] Figure 2A is a schematic diagram of a 30-degree pickup zone in the regional pickup method provided in the embodiment of this application;
[0024] Figure 2B is a schematic diagram of a 60-degree pickup zone in the area pickup method provided in the embodiment of this application;
[0025] Figure 2C is a schematic diagram of the 90-degree pickup zone in the area pickup method provided in the embodiment of this application;
[0026] Figure 3 is a flowchart of the area sound pickup method provided in an embodiment of this application;
[0027] Figure 4 is a schematic diagram of the regional sound pickup method provided in an embodiment of this application;
[0028] Figure 5 is a schematic diagram of the directional adjustable method in the area pickup method provided in the embodiment of this application;
[0029] Figure 6 is another flowchart of the area pickup method provided in the embodiment of this application;
[0030] Figure 7 is a schematic diagram of a regional sound pickup device provided in an embodiment of this application;
[0031] Figure 8 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. Detailed Implementation
[0032] In remote audio and video calls in conference rooms or open spaces, the traditional approach is to enhance the picked-up audio signal using beamforming technology and then send the enhanced audio signal to the remote end so that remote participants can better hear and understand the required audio information.
[0033] Zoned sound pickup refers to preserving the voice of a speaker within a designated area while blocking various noises and interfering voices outside the pickup area. For an example, please refer to Figure 1. Figure 1 is a schematic diagram of zoned sound pickup pointing directly forward. Referring to Figure 1, the microphone array 11 of the electronic device is used for sound pickup, and the participant 13 is located within the pickup area 12. The purpose of zoned sound pickup is to preserve the sound within the pickup area 12 while blocking noise and interfering voices outside the pickup area 12. The noise includes both noise within and outside the pickup area 12. The sound within the pickup area 12 includes the voice of the participant or the sound played from the conference tablet, etc.
[0034] However, traditional beamforming technology does not distinguish between areas within and outside the pickup zone. Instead, it enhances the picked-up voice signal, that is, it enhances both the voice signal within the pickup zone 12 and the interfering voices outside the pickup zone 12. This results in the inability to achieve a precise spatial shielding effect, which in turn causes remote participants to hear noisy sounds, seriously affecting the quality of remote meetings.
[0035] In addition, some regional voice pickup algorithms can only suppress interfering human voices and noise outside the pickup area, but have no obvious suppression effect on noise within the pickup area. This leads to the need to add speech denoising algorithms to suppress noise and improve speech quality. The process is cumbersome, consumes a lot of computing resources of electronic devices, and affects the performance of electronic devices.
[0036] Based on this, embodiments of this application provide a method, apparatus, device, and readable storage medium for regional sound pickup. The electronic device uses deep learning to shield noise and interfering human voices outside the pickup area. While shielding interfering human voices, it suppresses noise inside and outside the target pickup area, thereby accurately estimating the speech signal of the target pickup area, enabling remote participants to hear accurate speech signals and achieving the goal of improving the quality of remote meetings.
[0037] The area sound pickup method provided in this application is applicable to various electronic devices with microphone arrays, including but not limited to smart interactive flat panels, mobile phones, tablets, computers, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, personal digital assistants (PDAs), in-vehicle devices, wearable devices, etc. Below, taking a smart interactive flat panel as an example, a detailed description will be provided of the electronic device used to execute the area sound pickup method of this application.
[0038] In meeting or teaching settings, electronic devices are large interactive whiteboards or electronic blackboards, such as 55-inch, 65-inch, 86-inch or even larger.
[0039] The embodiments of this application do not limit the use of microphone arrays, such as linear array microphones, circular array microphones, stereo array microphones, or planar array microphones, on electronic devices.
[0040] Pre-trained regional sound pickup models are deployed on electronic devices. The number of regional sound pickup models can be one, two, or three, etc. For example, an electronic device may deploy a 30-degree regional sound pickup model, a 60-degree regional sound pickup model, and a 90-degree regional sound pickup model, that is, a total of three regional sound pickup models are deployed. Here, 30 degrees, 60 degrees, or 90 degrees refer to the angle of the sound pickup area. The larger the angle of the sound pickup area, the larger the area of the sound pickup area.
[0041] Alternatively, the area pickup model can be deployed on a server. Electronic devices send the microphone signals from each microphone to the server, which processes the microphone signals to accurately estimate the speech signal of the target pickup area and sends it to the remote end, so that remote participants can hear the accurate speech signal.
[0042] Figure 2A is a schematic diagram of a 30-degree pickup area in the area pickup method provided in the embodiment of this application; Figure 2B is a schematic diagram of a 60-degree pickup area in the area pickup method provided in the embodiment of this application; and Figure 2C is a schematic diagram of a 90-degree pickup area in the area pickup method provided in the embodiment of this application.
[0043] Please refer to Figures 2A-2C. The default pickup area is a fan-shaped area directly in front of the microphone array, with the center of the microphone array as the reference. This area is also called the preset pickup area. The purpose of zone pickup is to preserve the speaker's voice within the pickup area while blocking various noises and interfering voices outside the pickup area.
[0044] When three pickup patterns are deployed on an electronic device, the user can select the size of the pickup area according to their needs. For example, the electronic device displays an interface for the user to select the angle of the pickup area; through this interface, the user can select a pickup area of 30 degrees, 60 degrees, or 90 degrees. The default angle of the pickup area is 60 degrees, but this embodiment of the application is not limited to this.
[0045] Taking a 60-degree pickup area selected by the user as an example, please refer to Figure 2B. During the meeting, the electronic device uses each microphone in the microphone array to pick up the microphone signal. Then, the electronic device performs a short-time Fourier transform (STFT) on each microphone signal to obtain the complex spectrum of each microphone signal. The weight of each complex spectrum is determined using the regional pickup model corresponding to the 60-degree pickup area. Based on this weight, the purpose of preserving the speaker's voice within the pickup area and blocking various noises and interfering voices outside the pickup area is achieved.
[0046] In addition to selecting the angle of the pickup area, the user can also adjust its position. Taking a 30-degree pickup area as an example, the electronic device displays an adjustment interface for adjusting the position of the target pickup area. This interface has a sliding slider; when the user slides the slider, the electronic device responds by adjusting the position of the target pickup area. After adjustment, the position of the target pickup area is shown in the dotted area of Figure 2A. Clearly, relative to the preset pickup area located directly in front of the microphone array, the target pickup area is obtained by shifting the preset pickup area by a certain angle. The target pickup area and the preset pickup area have the same area and shape. The dotted line in Figure 2A represents the centerline of the pickup area, and the angle between the two centerlines is 15 degrees. The target pickup area is obtained by rotating the preset pickup area 15 degrees to the right, with an offset angle of 15 degrees. It can be understood that when the offset angle is 0 degrees, the target pickup area and the preset pickup area completely overlap.
[0047] When the offset angle is not 0, the propagation distance of the microphone signal changes compared to the preset pickup area, resulting in a time delay in the microphone signal. Therefore, in the frequency domain, the complex spectrum of each microphone signal needs to be phase-delayed before being input into the area pickup model to achieve adjustment or deflection of the area pickup direction.
[0048] Using this approach, users can flexibly adjust the position of the target pickup area through the interface, so that the target pickup area covers the speaker's location as much as possible, achieving directional adjustable area pickup, that is, achieving area pickup offset in a specified direction.
[0049] It should be noted that although Figures 2A-2C above illustrate the use of a linear microphone array as an example, the embodiments of this application are not limited thereto. In other feasible implementations, the microphone on the electronic device can also be a planar array or a three-dimensional array, etc.
[0050] The following is a preliminary description to illustrate the approach of this application in estimating the target signal in the target pickup area.
[0051] During a meeting or teaching session, each microphone picks up a microphone signal. At time t, the microphone signal of the i-th microphone is... As shown in the following formula (1):
[0052]
[0053] in, It is the microphone signal of the i-th microphone at time t. It is the target signal within the target pickup area. It is the room impulse response, which depends on the location of the sound source of the target signal, the location of the microphone, and the acoustic environment of the target space. This represents the interference signal captured by the i-th microphone, including interfering human voices and noise. The noise includes noise within the target pickup area and noise outside the target pickup area. The symbol * indicates convolution. It is understandable that both the interfering human voices and noise increase the reverberation effect.
[0054] Obviously, in formula (1), It is a known quantity, the target signal. It is an unknown quantity. In order for remote participants to hear the accurate target signal, it is necessary to rely on the microphone signal. Estimate the target signal .
[0055] Theoretically, for formula (1) , , , Performing short-time Fourier transforms on each yields the following formula (2):
[0056] Formula (2)
[0057] Where n represents the frequency point index and f represents the time frame index. , , , They are , , , The Fourier transform of, i.e., the Fourier transform of , , , The complex spectrum is obtained by performing short-time Fourier transforms on each of the following: , , , . Also known as the acoustic transfer function (ATF), it is the room impulse response. The STFT is represented in the frequency domain.
[0058] The process of estimating the target signal in the target pickup area is to determine the complex spectrum in formula (2). The process involves determining the complex spectrum through filtering and summation. Complex number spectrum Also known as the complex spectrum of the target pickup area. As shown in the following formula (3):
[0059] Formula (3)
[0060] Where I indicates that the microphone array contains I microphones. This represents the weight of the complex spectrum of the i-th microphone signal when the frequency index is n and the time frame index is f.
[0061] Determine the complex spectrum Then, the complex spectrum Perform an inverse short-time fourier transform (ISTFT) to estimate the target signal within the target pickup area.
[0062] According to formula (3), in order to determine the complex spectrum... It is necessary to determine the weights of the complex spectrum of the i-th microphone signal. Therefore, how to determine the weights of the complex spectrum of the i-th microphone signal? This is a key point. In the embodiments of this application, the weights of the complex spectrum of each microphone signal are determined using a regional pickup model.
[0063] Below, based on the above description of the pickup area and preamble, the regional pickup method of this application embodiment will be described in detail. For example, please refer to FIG3.
[0064] Figure 3 is a flowchart of the area sound pickup method provided in an embodiment of this application. The execution subject of this embodiment is the aforementioned electronic device with a microphone array, and this embodiment includes:
[0065] 301. Use each microphone in the microphone array to pick up sound and obtain the microphone signal of each microphone.
[0066] Among them, the microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.
[0067] In this embodiment, the electronic device with a microphone array is located in a target space, such as a conference room or classroom. The default target pickup area is a fan-shaped area directly in front of the microphone array, with the center of the microphone array as the reference. The angle of this fan-shaped area is, for example, 30 degrees, 60 degrees, or 90 degrees. Users can select the angle of the pickup area and / or change its position before the meeting or class begins.
[0068] During a meeting or lecture, the speaker located in the target pickup area generates the target signal. Simultaneously, interference arises from people chatting, talking, or playing audio / video outside the target pickup area; there is also noise inside or outside the target pickup area, such as fan noise, computer noise, and footsteps; furthermore, the wall material and size of the target space, as well as the different room impulse responses at different locations within the target space, all contribute to increased reverberation of the target signal, interference sounds, and noise. Therefore, each microphone signal is a noisy and reverberant signal.
[0069] Taking a linear microphone array as an example, assuming the microphone array contains I microphones, the microphone signal of the i-th microphone is as shown in formula (1):
[0070]
[0071] For a description of formula (1), please refer to the preceding description above, which will not be repeated here.
[0072] Obviously, in formula (1), It is a known quantity, the target signal. It is an unknown quantity, a quantity to be estimated. To ensure that remote participants hear the accurate target signal, it is necessary to estimate the microphone signal. Estimate the target signal .
[0073] 302. Determine the complex spectrum of each microphone signal.
[0074] In this embodiment, the complex spectrum is also referred to as the spectrum, speech spectrum, or the time-frequency (TF) representation of microphone signals. The electronic device processes each microphone signal... Perform a short-time Fourier transform to obtain the complex spectrum of each microphone signal. .
[0075] 303. Input the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum.
[0076] The weights are used to preserve the target signal within the target pickup area, suppress reverberation, and shield interfering human voices and noise.
[0077] As described above, the electronic device in this application embodiment deploys at least one, two, or all of the following regional sound pickup models: a 30-degree regional sound pickup model, a 90-degree regional sound pickup model, or a 60-degree regional sound pickup model. The electronic device defaults to a target sound pickup area angle of 60 degrees, meaning it uses a 60-degree regional sound pickup model to determine the weights of the complex spectrum. When the user selects a different angle, the electronic device uses the corresponding regional sound pickup model to determine the weights of the complex spectrum. For example, if the user selects 90 degrees, the electronic device uses a 90-degree regional sound pickup model to determine the weights of the complex spectrum.
[0078] In this embodiment, the function of the regional sound pickup model is to distinguish between the target sound pickup area and the shielded area outside the target sound pickup area. By using the weights of each complex spectrum, it suppresses interfering human voices and noise, and suppresses the reverberation effect of the target signal, thereby enhancing the target signal and achieving the purpose of regional sound pickup and noise reduction.
[0079] 304. Determine the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
[0080] After determining the weights of each complex spectrum using a regional pickup model, the electronic device then uses the complex spectrum of each microphone signal... and the weights of each complex spectrum The complex spectrum of the speech signal in the target pickup area is estimated by filtering and summing the complex spectra. Then, the complex spectrum of the speech signal was analyzed. An inverse Fourier transform is performed to estimate the target signal within the target pickup area, and then the estimated target signal is sent to the remote end so that the remote participants can hear the accurate speech signal.
[0081] The area pickup method provided in this application involves an electronic device with a microphone array and a pre-trained area pickup model deployed on the device. The electronic device uses each microphone in the microphone array to pick up the microphone signal, which is a noisy reverberant signal. Then, the electronic device determines the complex spectrum of each microphone signal and inputs it into the area pickup model to obtain the weights of each complex spectrum. Based on the complex spectrum of each microphone and its weights, the target signal within the target pickup area is estimated. Using this method, the electronic device uses the area pickup model to determine the weights of the complex spectrum of each microphone signal. These weights are used to retain the target signal within the target pickup area, suppress reverberation, shield noise in the target space, and block interfering human voices outside the target pickup area. This makes the estimated target signal closer to the actual target signal, achieving the goal of accurately distinguishing noise, the target signal within the target area, and interfering human voices outside the target area. This allows remote participants to hear accurate speech signals, thereby improving the quality of remote meetings.
[0082] As described above, this application embodiment requires the use of a regional sound pickup model to determine the weights of the complex spectrum of each microphone signal. Therefore, this application embodiment includes two stages: a training stage and an inference stage. The training stage is used to train the regional sound pickup model; in the inference stage, the weights of each complex spectrum are determined using the regional sound pickup model. The two stages will be described in detail below.
[0083] First, the training phase.
[0084] Figure 4 is a schematic diagram of the regional sound pickup method provided in this application embodiment. Referring to Figure 4, during the training phase, the electronic device constructs a reverberation and noise environment through simulation. A training sample set is constructed based on clean speech samples, noise samples, interfering human voice samples, and a sample room. The initial model is trained using the training sample set to obtain the regional sound pickup model. A pair of training samples in the training sample set includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of reverberation, interfering human voice, and noise to the sample speech in the sample pickup area of the sample room. The source of the interfering human voice is located outside the sample pickup area within the sample room, while the source of the noise is located at any position within the sample room. For example, if the model is trained for a 30-degree pickup area, the angle of the sample pickup area is 30 degrees.
[0085] During training, a short-time Fourier transform is performed on the noisy reverberation samples to obtain complex spectra. These complex spectra are then input into the initial model, which is used to estimate the weights of the complex spectra of the noisy reverberation samples. Based on these weights, the complex spectra of the noisy reverberation samples are filtered and summed to determine the complex spectra of the clean speech samples. An inverse Fourier transform is then performed on the complex spectra of the clean speech samples to estimate the clean samples. Finally, the loss between the estimated clean speech samples and the clean speech samples is calculated, and the model parameters are updated based on the loss value until the model converges. Here, the clean speech samples are the samples corresponding to the noisy reverberation samples.
[0086] Using this approach, since the noisy reverberation samples in the training sample set are obtained by simulating the distinction between human voices, reverberation, and noise inside and outside the pickup area, the regional pickup model trained based on this training sample set can accurately output weights for preserving the target signal, suppressing reverberation effects, and shielding interfering human voices and noise. Thus, in the inference stage, it achieves the purpose of enhancing the target signal within the pickup area, suppressing noise, and eliminating interfering human voices and reverberation outside the pickup area.
[0087] In this embodiment of the application, the model training stage includes two sub-stages: data preparation and simulation, and model building and training.
[0088] A. Data preparation and simulation.
[0089] The purpose of data preparation and simulation is to construct a training sample set. An electronic device simulates a preset number of sample rooms. Within each sample room's pickup area, there is a first number of sound source locations; outside the pickup area, there are a second number of interference locations; and within the room, there are a third number of noise locations. Then, for each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interference voice samples, noise samples, sound source locations, interference locations, and noise locations. Finally, the fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.
[0090] For example, to simulate the reverberation of a real room, electronic devices use acoustic simulation tools to set different room sizes, reverberation times, microphone array positions, sound source positions, interference positions, and noise positions. Each sample room has a sample pickup area, and the area outside the sample pickup area is a shielded area. The sample pickup area has a first number of sound source positions, the area outside the sample pickup area has a second number of interference positions, and the sample room has a third number of noise positions. The noise positions can be located either inside or outside the sample pickup area. The first, second, and third numbers can be equal or unequal, for example, all four.
[0091] Taking a sample room with 4 sound source locations, 4 interference locations, and 4 noise locations as an example, the electronic device selects a clean speech sample from the clean dataset and chooses one sound source location from the 4 sound source locations. This clean speech sample represents the speech emitted by a person at the selected sound source location. Similarly, the electronic device selects another clean speech sample from the clean speech dataset as interference speech, and chooses one interference location from the 4 interference locations to simulate interference speech outside the pickup area during a meeting. Likewise, the electronic device selects a noise sample from the noise dataset and randomly selects one noise location from the 4 noise locations to simulate noise from fans or other sources during a meeting. Then, the electronic device generates a noisy reverberation sample based on the clean speech sample, interference speech sample, noise sample, selected sound source location, interference location, noise location, and the room impulse response at each location. This noisy reverberation sample and the clean speech sample form a training pair.
[0092] Understandably, when constructing the training sample set, for each sample room, different sound source locations, different interference locations, different noise locations, different numbers of selected interference locations, different numbers of selected noise locations, different clean speech samples, and different selected noise samples will generate different noisy reverberation samples. For example, for the same clean speech sample, in one approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position d, and the noise sample is sample x, the electronic device generates one noisy reverberation sample; in another approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position e, and the noise sample is sample x, the electronic device generates another noisy reverberation sample; in yet another approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position d, and the noise sample is sample y, the electronic device generates yet another noisy reverberation sample.
[0093] Suppose the electronic device simulates 10,000 sample rooms, and generates a fourth set of training samples for each sample room. For example, if the fourth set is 5, then 5 pairs of training samples are generated for each sample room, resulting in a total of 50,000 pairs of training samples. The sampling rate for each sample is, for example, 16 kHz, and the duration is 12 seconds.
[0094] In addition, when electronic devices generate noisy reverberation samples, different signal-to-interference ratios and signal-to-noise ratios can be set. By adding sample noise to the reverberant signal, noisy reverberation samples can be generated.
[0095] In the above embodiments, the clean dataset is, for example, an open-source dataset, such as some Chinese speech databases, English speech databases, etc., and this application embodiment is not limited to this. The noise samples in the noise dataset are, for example, noise data collected in real-world meeting scenarios.
[0096] Using this approach, electronic devices simulate reverberation and noise environments, select clean speech samples, interfering human voice samples, and noise samples to construct a training sample set, making the training sample set more similar to the real meeting scenario. This enables the trained regional sound pickup model to accurately distinguish noise, target signals within the target area, and interfering human voices outside the target area, thereby improving the accuracy of the regional sound pickup model.
[0097] B. Model building and training.
[0098] During the process of training an initial model using a training sample set to obtain a regional sound pickup model, the electronic device determines the average loss value for each training epoch. If the decrease in the average loss value for this epoch is greater than or equal to a preset range, it indicates that the parameter adjustment step size is reasonable, and the electronic device continues training for the next epoch.
[0099] A training epoch refers to completing one training run on all samples in the training sample set. Since the training sample set contains a large number of samples (50,000 pairs, etc.), and the memory and computing power of electronic devices are limited, it's impossible to input all the training samples into the initial model at once. Therefore, only one batch of training samples is input at a time, such as 16 pairs. That is, there are a total of 50,000 training samples, and 16 pairs are processed each time. After training 16 pairs, a loss value is determined. After completing one epoch, 3125 loss values are obtained. The average loss value is calculated by averaging these 3125 loss values. Each epoch has an average loss value.
[0100] In this embodiment, the initial model is, for example, a classic convolutional recurrent network (CRN). The input of the model is the complex spectrum of the noisy reverberation sample, and the output of the model is the weight of the complex spectrum. After outputting the weight, the complex spectrum of each noisy reverberation sample is filtered and summed according to the weight to estimate the complex spectrum of the clean speech sample corresponding to the noisy reverberation sample. Then, the estimated complex spectrum is subjected to inverse Fourier transform to estimate the clean speech. The loss value is calculated based on the estimated clean speech and the actual clean speech sample. The loss function is a weighted combination loss function, as shown in the following formula (4):
[0101] Formula (4)
[0102] in, and These are real clean speech samples and estimated clean speech samples, respectively. yes The complex spectrum, yes The complex spectrum. The Signal-to-Interference and Noise Ratio (SISNR), also known as the time-domain scale-invariant signal-to-noise ratio, has a mean absolute error (MAE) that is the absolute error of the complex spectrum. This is the weighting coefficient, with an empirical value of 100.
[0103] During training, each noisy reverberation sample was sampled at 16 kHz. A 32 ms Hanning window was used for STFT, with an overlap rate of 50%. The Adam optimizer was used for training, with an initial learning rate of 0.001. The adjustment strategy was as follows: if the average loss value did not decrease or decreased less than a preset amount over 5 consecutive epochs, the learning rate was reduced to half its original value. Training lasted for a total of 100 epochs.
[0104] When the decrease in the average loss value in a given round is less than a preset threshold, the electronic device determines if there are consecutive rounds with a preset number of rounds in which the decrease in the average loss value in each round is less than the preset threshold (e.g., 5 rounds). If there are fewer than 5 consecutive rounds with a small decrease in the average loss value (e.g., 3 or 2 rounds), the electronic device continues training for the next round. If the decrease in the average loss value is less than the preset threshold for 5 consecutive rounds, it indicates that the parameter adjustment step size is unreasonable. The electronic device reduces the learning rate and then continues training the initial model based on the reduced learning rate until convergence. The converged initial model is used as the region pickup model. The parameters of the converged region pickup model are fixed.
[0105] This approach reduces the learning rate when the average loss value decreases less than a preset range over a set number of consecutive rounds, thereby accelerating the model's convergence and improving training speed.
[0106] Secondly, the reasoning stage.
[0107] During the training phase, when the initial model converges, it is used as the region pickup model, and its parameters are fixed. During the inference phase, as shown in Figure 3, the electronic device performs a short-time Fourier transform on each microphone signal to obtain a complex spectrum. This complex spectrum is used as the input to the region pickup model, and its output is the weights of each complex spectrum. Subsequently, the electronic device performs filtering and summation operations on each complex spectrum to estimate the complex spectrum of the target signal in the target pickup area. The complex spectrum of the target signal An inverse Fourier transform is performed to estimate the target signal within the target pickup area. This target signal is a single-channel speech signal, which is the result of area pickup and noise reduction.
[0108] In the above embodiments, the target pickup area is assumed to be directly in front of the microphone array, as shown by the solid lines in Figures 2A-2C. That is, the user has not slid the slider on the adjustment interface to change the target pickup area from the preset pickup area. In this case, the pickup direction is directly in front of the microphone array, the sound source is in the far field, and the sound wave can be considered a plane wave. The sound wave arriving at each microphone in the direct front direction arrives at the same time.
[0109] However, in order to adjust or deflect the pickup direction, it is inevitable in practice that the position of the target pickup area will be adjusted. The centerline of the adjusted target pickup area will have a certain angle with the centerline of the preset pickup area, as shown in Figure 2A. In this case, the microphone signals of each microphone will be delayed in the time domain. Therefore, the complex spectrum of each microphone signal needs to be phase-delayed in the frequency domain.
[0110] To achieve phase delay, the electronic device determines the offset angle of the target pickup area relative to the preset pickup area, and then performs phase delay on the complex spectrum of each microphone signal among multiple microphone signals based on the offset angle. The target pickup area is obtained by offsetting the entire preset pickup area by the offset angle.
[0111] As shown in Figure 2A, since the offset angle is 15 degrees, the electronic device performs phase delay on the complex spectrum of each microphone signal based on this offset angle. For example, the electronic device pre-stores a mapping table that stores the relationship between the offset angle and the phase delay. By querying the mapping table, the electronic device determines the magnitude of the phase delay and performs phase delay on the complex spectrum of each microphone signal based on the magnitude of the phase delay.
[0112] Using this method, when the pickup direction is adjusted, the target pickup area is offset relative to the preset pickup area. The electronic device performs phase delay on the complex spectrum of each microphone signal according to the offset angle, so as to achieve pickup of the area offset in the specified direction, without the need for more sample data when the pickup direction is directly in front.
[0113] Optionally, during the phase delay of the complex spectrum of each microphone signal among multiple microphone signals based on the offset angle, the electronic device determines the time delay of each microphone in the microphone array based on the distance from the microphone to the center point of the microphone array and the offset angle. Then, the electronic device performs phase delay on the complex spectrum of the microphone signals based on the time delay.
[0114] Figure 5 is a schematic diagram of the adjustable direction in the area pickup method provided in this application embodiment. Referring to Figure 5, the gray-filled circle represents the position of the microphone before the pickup direction is adjusted, and the black-filled circle represents the position of the microphone after the pickup direction is adjusted and the offset angle is α.
[0115] Please refer to Figure 5. Assuming the distance between the i-th microphone and the center point of the microphone array is d, when the offset angle is a, the propagation distance of the i-th microphone signal increases by b. Therefore, a time delay needs to be added to the i-th microphone signal, as shown in the following formula (5):
[0116] Formula (5)
[0117] in, The time delay added to the i-th microphone signal is represented by c, which is the speed of sound and can be a fixed value of 340 m / s. According to formula (5), the time-domain delay is converted to the frequency domain as the spectral phase difference, i.e., the phase delay of the complex spectrum. Therefore, the complex spectrum phase delay of the i-th microphone signal is shown in formula (6) below:
[0118] Formula (6)
[0119] To The complex spectrum after phase delay, where j is the imaginary unit.
[0120] Please refer to formulas (5) and (6). For each microphone, the complex spectrum of the microphone signal is phase-delayed according to the distance of the microphone from the center point and the offset angle, thereby achieving the overall directional deflection.
[0121] It should be noted that, for ease of understanding, Figure 5A illustrates the offset angle 'a' by rotating the entire microphone array by 'a'. In practice, to adjust or deflect the pickup direction, electronic devices such as smart interactive panels are not rotated to drive the microphone array to rotate. Instead, software algorithms are used to adjust or deflect the pickup direction, thereby changing the position of the target pickup area.
[0122] Figure 6 is another flowchart of the area pickup method provided in this application embodiment. This embodiment includes the following steps:
[0123] 601. Perform a short-time Fourier transform on the signals picked up by each microphone to obtain the complex spectrum of each microphone signal.
[0124] 602. Based on the offset angle, perform phase delay on the complex spectrum of each microphone signal.
[0125] When the pickup direction is adjusted or deflected, the signals from each microphone are delayed in the time domain. The delay time is determined based on the distance between the microphone and the center point of the microphone array, as well as the offset angle. Correspondingly, the complex spectrum of each microphone signal needs to be phase-delayed in the frequency domain.
[0126] 603. Input the phase-delayed complex spectrum into the pre-trained regional pickup model so that the regional pickup model outputs the weights of the complex spectrum of each microphone signal.
[0127] 604. Filter and sum the complex spectra of each microphone signal to estimate the complex spectrum of the target signal in the target pickup area.
[0128] 605. Perform an inverse Fourier transform on the complex spectrum of the estimated target signal to estimate the target signal within the target pickup area.
[0129] 606. Send the estimated target signal to the remote end so that the remote participants hear the accurate target signal.
[0130] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.
[0131] Figure 7 is a schematic diagram of a regional sound pickup device provided in an embodiment of this application. The regional sound pickup device 700 includes: a sound pickup module 71, a determination module 72, a processing module 73, and an estimation module 74.
[0132] The pickup module 71 is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noise-band reverberant signal. The noise-band reverberant signal is formed by adding reverberation effect to the target signal in the target pickup area of the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.
[0133] The determination module 72 is used to determine the complex spectrum of each microphone signal;
[0134] The processing module 73 is used to input the complex spectrum of each microphone signal into the pre-trained area pickup model so that the area pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target pickup area, suppress reverberation effects, and shield interfering human voices and noise.
[0135] The estimation module 74 is used to estimate the target signal in the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
[0136] In one feasible implementation, the processing module 73 inputs the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. Before that, it is also used to construct a training sample set. The training sample set contains multiple pairs of training samples. Each pair of training samples includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of reverberation, interfering human voice, and noise to the sample speech of the sample pickup area in the sample room. The source of the interfering human voice is located outside the sample pickup area in the sample room, and the source of the noise is located at any position in the sample room. The initial model is trained using the training sample set to obtain the regional sound pickup model.
[0137] In one feasible implementation, when the processing module 73 constructs the training sample set, it is used to simulate a preset number of sample rooms. Each sample room has a first number of sound source locations within its sample pickup area, a second number of interference locations outside its sample pickup area, and a third number of noise locations within its sample room. For each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interference voice samples, noise samples, sound source locations, interference locations, and noise locations. The fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.
[0138] In one feasible implementation, when the processing module 73 uses the training sample set to train the initial model to obtain the regional sound pickup model, it is used to determine the average loss value of the current round after each training round. Training one round means: completing one training for all samples in the training sample set.
[0139] When the decrease in the average loss value is less than the preset range, determine whether there are consecutive rounds of a preset number of rounds in which the decrease in the average loss value of each round is less than the preset range; when there are consecutive rounds of a preset number of rounds, reduce the learning rate; continue training the initial model according to the reduced learning rate until convergence, and use the converged initial model as the region pickup model.
[0140] In one feasible implementation, the processing module 73 inputs the complex spectrum of each microphone signal into the pre-trained regional pickup model so that before the regional pickup model outputs the weights of each complex spectrum, it is also used to determine the offset angle of the target pickup area relative to the preset pickup area. The target pickup area is obtained by offsetting the preset pickup area by the offset angle. The complex spectrum of each microphone signal in the multiple microphone signals is phase-delayed according to the offset angle.
[0141] In one feasible implementation, when the processing module 73 performs phase delay on the complex spectrum of each microphone signal among multiple microphone signals according to the offset angle, it is used to determine the time delay of the corresponding microphone for each microphone in the microphone array according to the distance from the microphone to the center point of the microphone array and the offset angle; and to perform phase delay on the complex spectrum of the microphone signal according to the time delay.
[0142] In one feasible implementation, before the pickup module 71 uses each microphone in the microphone array to pick up sound from the target area, the processing module 73 is also used to control the display screen of the electronic device to display an adjustment interface for adjusting the position of the target pickup area; in response to the sliding operation of the slider on the adjustment interface, the position of the target pickup area is adjusted.
[0143] The area pickup device provided in this application embodiment can perform the actions of the electronic device in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.
[0144] Figure 8 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device is, for example, the aforementioned smart interactive flat panel. Referring to Figure 8, the electronic device 800 in this embodiment of the application includes: at least one processor 81, at least one communication bus 82, a user interface 83, at least one network interface 84, a memory 85, and a microphone array 87.
[0145] The communication bus 82 is used to enable communication between these components.
[0146] The user interface 83 may include a display screen and a camera. Optionally, the user interface 83 may also include a standard wired interface and a wireless interface.
[0147] Among them, the network interface 84 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).
[0148] The processor 81 may include one or more processing cores. The processor 81 connects to various parts of the electronic device 800 using various interfaces and lines, and performs various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 85, and by calling data stored in the memory 85. Optionally, the processor 81 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 81 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the panoramic sphere required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 81 and may be implemented as a separate chip.
[0149] The memory 85 may include random access memory (RAM) or read-only memory. Optionally, the memory 85 may include a non-transitory computer-readable storage medium. The memory 85 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 85 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 85 may also be at least one storage device located remotely from the aforementioned processor 81. As shown in FIG8, the memory 85, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and operating applications for electronic devices.
[0150] The microphone array 87 can be a linear array microphone, a ring array microphone, etc., and the embodiments of this application are not limited thereto.
[0151] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the above-described area pickup method.
[0152] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described area pickup method.
[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams.
[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.
[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.
[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0158] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0159] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0160] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0161] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. A method for regional sound pickup, characterized in that, Applied to an electronic device having a microphone array, the method includes: The microphone signals of each microphone are obtained by using each microphone in the microphone array to pick up sound. The microphone signals are noisy reverberation signals. The noisy reverberation signals are formed by adding reverberation effects to the target signal in the target pickup area of the target space, interfering with human voices and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voices is located outside the target pickup area, and the sound source of noise is located at any position in the target space. Determine the complex spectrum of the signals from each microphone; The complex spectrum of each microphone signal is input into a pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise. The target signal within the target pickup area is estimated based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
2. The method according to claim 1, characterized in that, Before inputting the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum, the method further includes: A training sample set is constructed, which contains multiple pairs of training samples. Each pair of training samples includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of reverberation, interfering human voice, and noise to the sample speech in the sample pickup area of the sample room. The source of the interfering human voice is located outside the sample pickup area in the sample room, and the source of the noise is located at any position in the sample room. The initial model is trained using the training sample set to obtain the regional sound pickup model.
3. The method according to claim 2, characterized in that, The construction of the training sample set includes: The simulation uses a preset number of sample rooms. Each sample room has a first number of sound source locations within its sample pickup area, a second number of interference locations outside its sample pickup area, and a third number of noise locations within its sample room. For each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interfering human voice samples, noise samples, the location of the sound source, the location of the interference, and the location of the noise. The fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.
4. The method according to claim 2, characterized in that, The process of training an initial model using the training sample set to obtain the region sound pickup model includes: After each training round, the average loss value for that round is determined. One training round refers to completing one training run on all samples in the training sample set. When the decrease in the average loss value is less than a preset range, it is determined whether there are consecutive rounds of a preset number of rounds in which the decrease in the average loss value of each round of the preset number of rounds is less than the preset range. When there are a preset number of consecutive rounds, reduce the learning rate; The initial model is trained again using the reduced learning rate until convergence, and the converged initial model is used as the regional sound pickup model.
5. The method according to any one of claims 1 to 4, characterized in that, Before inputting the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum, the method further includes: The offset angle of the target pickup area relative to the preset pickup area is determined, and the target pickup area is obtained by offsetting the preset pickup area as a whole by the offset angle; The complex spectrum of each microphone signal is phase-delayed according to the offset angle.
6. The method according to claim 5, characterized in that, The step of performing phase delay on the complex spectrum of each microphone signal among the plurality of microphone signals according to the offset angle includes: For each microphone in the microphone array, the time delay of the corresponding microphone is determined based on the distance of the microphone from the center point of the microphone array and the offset angle; The complex spectrum of the microphone signal is phase-delayed according to the time delay.
7. The method according to any one of claims 1 to 4, characterized in that, Before using each microphone in the microphone array to pick up sound from the target area, the method further includes: The interface displays an adjustment feature for adjusting the position of the target pickup area; In response to a sliding operation of the slider on the adjustment interface, the position of the target pickup area is adjusted.
8. A regional sound pickup device, characterized in that, Integrated into an electronic device having a microphone array, the device includes: The sound pickup module is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space. The determination module is used to determine the complex spectrum of each microphone signal; The processing module is used to input the complex spectrum of each microphone signal into a pre-trained regional sound pickup model, so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise. An estimation module is used to estimate the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.
9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.