Area pickup method, device, equipment and readable storage medium

Abstract

Embodiments of the present application provide a region pickup method, device, equipment and readable storage medium. An electronic device uses each microphone in a microphone array to pick up sound, and obtains a microphone signal of each microphone. The microphone signal is a noisy and reverberant signal. Then, the electronic device determines a complex spectrum of each microphone signal, inputs the complex spectrum into a region pickup model to obtain a weight of each complex spectrum, and estimates a target signal in a target pickup region according to the complex spectrum of each microphone and the weight of each complex spectrum. With this scheme, the electronic device uses the region pickup model to determine the weight of the complex spectrum of each microphone signal. The weight is used to retain the target signal in the target pickup region, suppress the reverberation effect, shield the noise of the target space and the interfering human voice outside the target pickup region, so that the estimated target signal is closer to the actual target signal, and then the remote participant can hear the accurate voice signal, thereby achieving the purpose of improving the quality of the remote conference.

Description

Technical Field

[0001] This application relates to the field of voice acquisition technology, and in particular to a method, apparatus, device, and readable storage medium for regional voice pickup. Background Technology

[0002] With the rapid development of internet technology and the limitations of physical venues, remote conferencing is being used more and more widely.

[0003] During the meeting, the local audio pickup device uses microphone array beamforming technology to enhance the picked-up audio signal and send the enhanced audio signal to the remote end, enabling remote participants to better listen to and understand the speaker's speech.

[0004] However, traditional beamforming-based area pickup cannot accurately distinguish between the speaker's voice signal and interfering voices, resulting in remote participants hearing noisy sounds and seriously affecting the quality of remote meetings. Summary of the Invention

[0005] This application provides a method, apparatus, device, and readable storage medium for regional sound pickup. The electronic device uses deep learning to shield noise and interfering human voices outside the pickup area, thereby estimating the speech signal of the target pickup area, so that remote participants can hear accurate speech signals and achieve the goal of improving the quality of remote meetings.

[0006] In a first aspect, embodiments of this application provide a regional sound pickup method, applied to an electronic device having a microphone array, the method comprising:

[0007] The microphone signals of each microphone are obtained by using each microphone in the microphone array to pick up sound. The microphone signals are noisy reverberation signals. The noisy reverberation signals are formed by adding reverberation effects to the target signal in the target pickup area of ​​the target space, interfering with human voices and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voices is located outside the target pickup area, and the sound source of noise is located at any position in the target space.

[0008] Determine the complex spectrum of the signals from each microphone;

[0009] The complex spectrum of each microphone signal is input into a pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise.

[0010] The target signal within the target pickup area is estimated based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

[0011] Secondly, embodiments of this application provide a regional sound pickup device, which is integrated on an electronic device having a microphone array, and the device includes:

[0012] The sound pickup module is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of ​​the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.

[0013] The determination module is used to determine the complex spectrum of each microphone signal;

[0014] The processing module is used to input the complex spectrum of each microphone signal into a pre-trained regional sound pickup model, so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise.

[0015] An estimation module is used to estimate the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

[0016] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it causes the electronic device to implement the method described in the first aspect or various possible implementations of the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the method described in the first aspect or various possible implementations of the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product comprising a computing program, wherein when the computer program is executed by a processor, it implements the method described in the first aspect or various possible implementations of the first aspect.

[0019] The regional sound pickup method, apparatus, device, and readable storage medium provided in this application embodiment include an electronic device with a microphone array and a pre-trained regional sound pickup model deployed on the electronic device. The electronic device uses each microphone in the microphone array to pick up the microphone signal, which is a noisy reverberation signal. Then, the electronic device determines the complex spectrum of each microphone signal, inputs the complex spectrum into the regional sound pickup model to obtain the weights of each complex spectrum, and estimates the target signal within the target pickup area based on the complex spectrum of each microphone and its weights. Using this scheme, the electronic device uses the regional sound pickup model to determine the weights of the complex spectrum of each microphone signal. These weights are used to retain the target signal within the target pickup area, suppress reverberation, shield noise in the target space, and block interfering human voices outside the target pickup area. This makes the estimated target signal closer to the actual target signal, achieving the goal of accurately distinguishing noise, the target signal within the target area, and interfering human voices outside the target area. This allows remote participants to hear accurate speech signals, thereby improving the quality of remote meetings. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a diagram illustrating the area for sound pickup pointing directly forward.

[0022] Figure 2A This is a schematic diagram of a 30-degree pickup zone in the regional pickup method provided in the embodiments of this application;

[0023] Figure 2B This is a schematic diagram of a 60-degree pickup zone in the area pickup method provided in the embodiments of this application;

[0024] Figure 2C This is a schematic diagram of a 90-degree pickup zone in the area pickup method provided in the embodiments of this application;

[0025] Figure 3 This is a flowchart of the area pickup method provided in the embodiments of this application;

[0026] Figure 4 This is a schematic diagram of the regional sound pickup method provided in the embodiments of this application;

[0027] Figure 5 This is a schematic diagram of the directional adjustable method for regional sound pickup provided in the embodiments of this application;

[0028] Figure 6 This is another flowchart of the area pickup method provided in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of a regional sound pickup device provided in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0031] In remote audio and video calls in conference rooms or open spaces, the traditional approach is to enhance the picked-up audio signal using beamforming technology and then send the enhanced audio signal to the remote end so that remote participants can better hear and understand the required audio information.

[0032] Zoned voice pickup refers to preserving the speaker's voice within a designated area while blocking various noises and interfering voices from outside the pickup area. For an example, please refer to... Figure 1 . Figure 1 This is a diagram showing the area for sound pickup pointing directly forward. Please refer to it. Figure 1 The microphone array 11 of the electronic device is used for sound pickup, and the participants 13 are located within the pickup area 12. The purpose of area pickup is to preserve the sound within the pickup area 12 while shielding noise and interfering human voices outside the pickup area 12. The noise includes both noise within and outside the pickup area 12. The sound within the pickup area 12 includes the human voices of the participants or the sound played from the conference tablet, etc.

[0033] However, traditional beamforming technology does not distinguish between areas within and outside the pickup zone. Instead, it enhances the picked-up voice signal, that is, it enhances both the voice signal within the pickup zone 12 and the interfering voices outside the pickup zone 12. This results in the inability to achieve a precise spatial shielding effect, which in turn causes remote participants to hear noisy sounds, seriously affecting the quality of remote meetings.

[0034] In addition, some regional voice pickup algorithms can only suppress interfering human voices and noise outside the pickup area, but have no obvious suppression effect on noise within the pickup area. This leads to the need to add speech denoising algorithms to suppress noise and improve speech quality. The process is cumbersome, consumes a lot of computing resources of electronic devices, and affects the performance of electronic devices.

[0035] Based on this, embodiments of this application provide a method, apparatus, device, and readable storage medium for regional sound pickup. The electronic device uses deep learning to shield noise and interfering human voices outside the pickup area. While shielding interfering human voices, it suppresses noise inside and outside the target pickup area, thereby accurately estimating the speech signal of the target pickup area, enabling remote participants to hear accurate speech signals and achieving the goal of improving the quality of remote meetings.

[0036] The area sound pickup method provided in this application is applicable to various electronic devices with microphone arrays, including but not limited to smart interactive flat panels, mobile phones, tablets, computers, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, personal digital assistants (PDAs), in-vehicle devices, wearable devices, etc. Below, taking a smart interactive flat panel as an example, a detailed description will be provided of the electronic device used to execute the area sound pickup method of this application.

[0037] In meeting or teaching settings, electronic devices are large interactive whiteboards or electronic blackboards, such as 55-inch, 65-inch, 86-inch or even larger.

[0038] The embodiments of this application do not limit the use of microphone arrays, such as linear array microphones, circular array microphones, stereo array microphones, or planar array microphones, on electronic devices.

[0039] Pre-trained regional sound pickup models are deployed on electronic devices. The number of regional sound pickup models can be one, two, or three, etc. For example, an electronic device may deploy a 30-degree regional sound pickup model, a 60-degree regional sound pickup model, and a 90-degree regional sound pickup model, that is, a total of three regional sound pickup models are deployed. Here, 30 degrees, 60 degrees, or 90 degrees refer to the angle of the sound pickup area. The larger the angle of the sound pickup area, the larger the area of ​​the sound pickup area.

[0040] Alternatively, the area pickup model can be deployed on a server. Electronic devices send the microphone signals from each microphone to the server, which processes the microphone signals to accurately estimate the speech signal of the target pickup area and sends it to the remote end, so that remote participants can hear the accurate speech signal.

[0041] Figure 2A This is a schematic diagram of a 30-degree pickup zone in the area pickup method provided in this application embodiment. Figure 2BThis is a schematic diagram of a 60-degree pickup zone in the area pickup method provided in this application embodiment. Figure 2C This is a schematic diagram of a 90-degree pickup zone in the area pickup method provided in the embodiments of this application.

[0042] Please refer to Figures 2A-2C The default pickup area is a fan-shaped area directly in front of the microphone array, with the center of the microphone array as the reference point. This area is also known as the preset pickup area. The purpose of zone pickup is to preserve the speaker's voice within the pickup area while blocking various noises and interfering voices outside the pickup area.

[0043] When three pickup patterns are deployed on an electronic device, the user can select the size of the pickup area according to their needs. For example, the electronic device displays an interface for the user to select the angle of the pickup area; through this interface, the user can select a pickup area of ​​30 degrees, 60 degrees, or 90 degrees. The default angle of the pickup area is 60 degrees, but this embodiment of the application is not limited to this.

[0044] Taking a user selecting a 60-degree pickup area as an example, please refer to... Figure 2B During the meeting, the electronic equipment uses each microphone in the microphone array to pick up the microphone signal. Then, the electronic equipment performs a short-time Fourier transform (STFT) on each microphone signal to obtain the complex spectrum of each microphone signal, and uses a regional pickup model corresponding to a 60-degree pickup area to determine the weight of each complex spectrum. Based on this weight, the purpose of preserving the speaker's voice within the pickup area and blocking various noises and interfering voices outside the pickup area is achieved.

[0045] In addition to selecting the angle of the pickup area, users can also adjust its position. Taking a 30-degree pickup area as an example, the electronic device displays an adjustment interface for setting the target pickup area's position. This interface has a sliding slider; when the user slides the slider, the electronic device responds to the sliding operation and adjusts the position of the target pickup area. After adjustment, the position of the target pickup area is as follows: Figure 2A As shown in the dashed area. Obviously, relative to the preset pickup area located directly in front of the microphone array, the target pickup area is obtained by offsetting the preset pickup area by a certain offset angle. The target pickup area and the preset pickup area are exactly the same in area and shape. Figure 2A The dotted line represents the centerline of the pickup area, and the angle between the two centerlines is 15 degrees. The preset pickup area is rotated 15 degrees to the right to obtain the target pickup area, with an offset angle of 15 degrees. It can be understood that when the offset angle is 0 degrees, the target pickup area and the preset pickup area completely overlap.

[0046] When the offset angle is not 0, the propagation distance of the microphone signal changes compared to the preset pickup area, resulting in a time delay in the microphone signal. Therefore, in the frequency domain, the complex spectrum of each microphone signal needs to be phase-delayed before being input into the area pickup model to achieve adjustment or deflection of the area pickup direction.

[0047] Using this approach, users can flexibly adjust the position of the target pickup area through the interface, so that the target pickup area covers the speaker's location as much as possible, achieving directional adjustable area pickup, that is, achieving area pickup offset in a specified direction.

[0048] It should be noted that, although the above Figures 2A-2C The example described uses a linear microphone array. However, this embodiment is not limited to this; in other feasible implementations, the microphone on the electronic device can also be a planar array or a three-dimensional array, etc.

[0049] The following is a preliminary description to illustrate the approach of this application in estimating the target signal in the target pickup area.

[0050] During a meeting or teaching session, each microphone picks up a microphone signal. At time t, the microphone signal of the i-th microphone is x. i (t), as shown in formula (1) below:

[0051] x i (t)=a i (t)*s(t)+n i (t) Formula (1)

[0052] Where, x i s(t) is the microphone signal of the i-th microphone at time t, s(t) is the target signal within the target pickup area, and a i (t) is the room impulse response, which depends on the location of the sound source of the target signal, the location of the microphone, and the acoustic environment of the target space, etc. i (t) represents the interference signal collected by the i-th microphone, including interfering human voice and noise. The noise includes noise within the target pickup area and noise outside the target pickup area. The symbol * denotes convolution. It is understandable that both the interfering human voice and the noise increase the reverberation effect.

[0053] Obviously, in formula (1), x i (t) is a known quantity, and the target signal s(t) is an unknown quantity. In order for remote participants to hear the accurate target signal, it is necessary to determine the microphone signal x. i (t) Estimate the target signal s(t).

[0054] Theoretically, for s(t) and x in formula (1) i(t), a i (t), n i (t) are subjected to short-time Fourier transforms to obtain the following formula (2):

[0055] X i (n, f) = A i (f)S(n,f)+N i Formula (2) (n, f)

[0056] Where n represents the frequency index, f represents the time frame index, and S(n, f), X i (n, f), A i (f), N i (n, f) represent s(t) and x respectively. i (t), a i (t), n i The Fourier transform of s(t), i.e., the transformation of s(t) and x i (t), a i (t), n i (t) Perform short-time Fourier transforms to obtain the complex spectra: S(n, f), X i (n, f), A i (f), N i (n, f). A i (f) Also known as the acoustic transfer function (ATF), it is the room impulse response. i (t) Representation in the frequency domain after STFT.

[0057] The process of estimating the target signal in the target pickup area is the process of determining the complex spectrum S(n, f) in formula (2). The complex spectrum S(n, f) can be determined by filtering and summing. The complex spectrum S(n, f) is also called the complex spectrum of the target pickup area. As shown in the following formula (3):

[0058]

[0059] Where I represents that the microphone array contains I microphones, and W i (n, f) represents the weight of the complex spectrum of the i-th microphone signal when the frequency index is n and the time frame index is f.

[0060] After determining the complex spectrum S(n,f), an inverse short-time fourier transform (ISTFT) is performed on the complex spectrum S(n,f) to estimate the target signal within the target pickup area.

[0061] According to formula (3), in order to determine the complex spectrum S(n, f), it is necessary to determine the weight W of the complex spectrum of the i-th microphone signal. i (n, f). Therefore, how to determine the weight W of the complex spectrum of the i-th microphone signal? i (n, f) is a key point. In this embodiment, the weights of the complex spectrum of each microphone signal are determined using a regional pickup model.

[0062] Below, based on the above description of the pickup area and preamble, the regional pickup method of this application embodiment will be described in detail. For example, please refer to... Figure 3 .

[0063] Figure 3 This is a flowchart of the area sound pickup method provided in this application embodiment. The execution subject of this embodiment is the aforementioned electronic device with a microphone array, and this embodiment includes:

[0064] 301. Use each microphone in the microphone array to pick up sound and obtain the microphone signal of each microphone.

[0065] Among them, the microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of ​​the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.

[0066] In this embodiment, the electronic device with a microphone array is located in a target space, such as a conference room or classroom. The default target pickup area is a fan-shaped area directly in front of the microphone array, with the center of the microphone array as the reference. The angle of this fan-shaped area is, for example, 30 degrees, 60 degrees, or 90 degrees. Users can select the angle of the pickup area and / or change its position before the meeting or class begins.

[0067] During a meeting or lecture, the speaker located in the target pickup area generates the target signal. Simultaneously, interference arises from people chatting, talking, or playing audio / video outside the target pickup area; there is also noise inside or outside the target pickup area, such as fan noise, computer noise, and footsteps; furthermore, the wall material and size of the target space, as well as the different room impulse responses at different locations within the target space, all contribute to increased reverberation of the target signal, interference sounds, and noise. Therefore, each microphone signal is a noisy and reverberant signal.

[0068] Taking a linear microphone array as an example, assuming the microphone array contains I microphones, the microphone signal of the i-th microphone is as shown in formula (1):

[0069] xi (t)=a i (t)*s(t)+n i (t) Formula (1)

[0070] For a description of formula (1), please refer to the preceding description above, which will not be repeated here.

[0071] Obviously, in formula (1), x i (t) is a known quantity, and the target signal s(t) is an unknown quantity, which is the quantity to be estimated. To ensure that remote participants hear the accurate target signal, it is necessary to estimate the microphone signal x. i (t) Estimate the target signal s(t).

[0072] 302. Determine the complex spectrum of each microphone signal.

[0073] In this embodiment, the complex spectrum is also referred to as the spectrum, speech spectrum, or the time-frequency (TF) representation of microphone signals. The electronic device represents each microphone signal x... i (t) Perform a short-time Fourier transform to obtain the complex spectrum X of each microphone signal. i (n, f).

[0074] 303. Input the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum.

[0075] The weights are used to preserve the target signal within the target pickup area, suppress reverberation, and shield interfering human voices and noise.

[0076] As described above, the electronic device in this application embodiment deploys at least one, two, or all of the following regional sound pickup models: a 30-degree regional sound pickup model, a 90-degree regional sound pickup model, or a 60-degree regional sound pickup model. The electronic device defaults to a target sound pickup area angle of 60 degrees, meaning it uses a 60-degree regional sound pickup model to determine the weights of the complex spectrum. When the user selects a different angle, the electronic device uses the corresponding regional sound pickup model to determine the weights of the complex spectrum. For example, if the user selects 90 degrees, the electronic device uses a 90-degree regional sound pickup model to determine the weights of the complex spectrum.

[0077] In this embodiment, the function of the regional sound pickup model is to distinguish between the target sound pickup area and the shielded area outside the target sound pickup area. By using the weights of each complex spectrum, it suppresses interfering human voices and noise, and suppresses the reverberation effect of the target signal, thereby enhancing the target signal and achieving the purpose of regional sound pickup and noise reduction.

[0078] 304. Determine the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

[0079] After determining the weights of each complex spectrum using a regional pickup model, the electronic device then calculates the weights based on the complex spectrum X of each microphone signal. i (n, f) and the weights W of each complex spectrum i The complex spectrum S(n,f) of the speech signal in the target pickup area is estimated by filtering and summing the complex spectra. Then, an inverse Fourier transform is performed on the complex spectrum S(n,f) of the speech signal to estimate the target signal in the target pickup area. The estimated target signal is then sent to the remote end so that the remote participants can hear the accurate speech signal.

[0080] The area pickup method provided in this application involves an electronic device with a microphone array and a pre-trained area pickup model deployed on the device. The electronic device uses each microphone in the microphone array to pick up the microphone signal, which is a noisy reverberant signal. Then, the electronic device determines the complex spectrum of each microphone signal and inputs it into the area pickup model to obtain the weights of each complex spectrum. Based on the complex spectrum of each microphone and its weights, the target signal within the target pickup area is estimated. Using this method, the electronic device uses the area pickup model to determine the weights of the complex spectrum of each microphone signal. These weights are used to retain the target signal within the target pickup area, suppress reverberation, shield noise in the target space, and block interfering human voices outside the target pickup area. This makes the estimated target signal closer to the actual target signal, achieving the goal of accurately distinguishing noise, the target signal within the target area, and interfering human voices outside the target area. This allows remote participants to hear accurate speech signals, thereby improving the quality of remote meetings.

[0081] As described above, this application embodiment requires the use of a regional sound pickup model to determine the weights of the complex spectrum of each microphone signal. Therefore, this application embodiment includes two stages: a training stage and an inference stage. The training stage is used to train the regional sound pickup model; in the inference stage, the weights of each complex spectrum are determined using the regional sound pickup model. The two stages will be described in detail below.

[0082] First, the training phase.

[0083] Figure 4 This is a schematic diagram illustrating the process of the area pickup method provided in the embodiments of this application. Please refer to... Figure 4During the training phase, the electronic device simulates a reverberation and noise environment. A training sample set is constructed based on clean speech samples, noise samples, interfering human voice samples, and a sample room. This training sample set is used to train the initial model to obtain the regional sound pickup model. Each pair of training samples in the training sample set includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of the following to the sample speech in the sample pickup area of ​​the sample room: reverberation, interfering human voice, or noise. The source of the interfering human voice is located outside the sample pickup area within the sample room, while the source of the noise is located at any position within the sample room. For example, if the model is trained for a 30-degree pickup area, then the angle of the sample pickup area is 30 degrees.

[0084] During training, a short-time Fourier transform is performed on the noisy reverberation samples to obtain complex spectra. These complex spectra are then input into the initial model, which is used to estimate the weights of the complex spectra of the noisy reverberation samples. Based on these weights, the complex spectra of the noisy reverberation samples are filtered and summed to determine the complex spectra of the clean speech samples. An inverse Fourier transform is then performed on the complex spectra of the clean speech samples to estimate the clean samples. Finally, the loss between the estimated clean speech samples and the clean speech samples is calculated, and the model parameters are updated based on the loss value until the model converges. Here, the clean speech samples are the samples corresponding to the noisy reverberation samples.

[0085] Using this approach, since the noisy reverberation samples in the training sample set are obtained by simulating the distinction between human voices, reverberation, and noise inside and outside the pickup area, the regional pickup model trained based on this training sample set can accurately output weights for preserving the target signal, suppressing reverberation effects, and shielding interfering human voices and noise. Thus, in the inference stage, it achieves the purpose of enhancing the target signal within the pickup area, suppressing noise, and eliminating interfering human voices and reverberation outside the pickup area.

[0086] In this embodiment of the application, the model training stage includes two sub-stages: data preparation and simulation, and model building and training.

[0087] A. Data preparation and simulation.

[0088] The purpose of data preparation and simulation is to construct a training sample set. An electronic device simulates a preset number of sample rooms. Within each sample room's pickup area, there is a first number of sound source locations; outside the pickup area, there are a second number of interference locations; and within the room, there are a third number of noise locations. Then, for each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interference voice samples, noise samples, sound source locations, interference locations, and noise locations. Finally, the fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.

[0089] For example, to simulate the reverberation of a real room, electronic devices use acoustic simulation tools to set different room sizes, reverberation times, microphone array positions, sound source positions, interference positions, and noise positions. Each sample room has a sample pickup area, and the area outside the sample pickup area is a shielded area. The sample pickup area has a first number of sound source positions, the area outside the sample pickup area has a second number of interference positions, and the sample room has a third number of noise positions. The noise positions can be located either inside or outside the sample pickup area. The first, second, and third numbers can be equal or unequal, for example, all four.

[0090] Taking a sample room with 4 sound source locations, 4 interference locations, and 4 noise locations as an example, the electronic device selects a clean speech sample from the clean dataset and chooses one sound source location from the 4 sound source locations. This clean speech sample represents the speech emitted by a person at the selected sound source location. Similarly, the electronic device selects another clean speech sample from the clean speech dataset as interference speech, and chooses one interference location from the 4 interference locations to simulate interference speech outside the pickup area during a meeting. Likewise, the electronic device selects a noise sample from the noise dataset and randomly selects one noise location from the 4 noise locations to simulate noise from fans or other sources during a meeting. Then, the electronic device generates a noisy reverberation sample based on the clean speech sample, interference speech sample, noise sample, selected sound source location, interference location, noise location, and the room impulse response at each location. This noisy reverberation sample and the clean speech sample form a training pair.

[0091] Understandably, when constructing the training sample set, for each sample room, different sound source locations, different interference locations, different noise locations, different numbers of selected interference locations, different numbers of selected noise locations, different clean speech samples, and different selected noise samples will generate different noisy reverberation samples. For example, for the same clean speech sample, in one approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position d, and the noise sample is sample x, the electronic device generates one noisy reverberation sample; in another approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position e, and the noise sample is sample x, the electronic device generates another noisy reverberation sample; in yet another approach, the sound source location is position a, the interference locations are positions b and c, the noise location is position d, and the noise sample is sample y, the electronic device generates yet another noisy reverberation sample.

[0092] Suppose the electronic device simulates 10,000 sample rooms, and generates a fourth set of training samples for each sample room. For example, if the fourth set is 5, then 5 pairs of training samples are generated for each sample room, resulting in a total of 50,000 pairs of training samples. The sampling rate for each sample is, for example, 16 kHz, and the duration is 12 seconds.

[0093] In addition, when electronic devices generate noisy reverberation samples, different signal-to-interference ratios and signal-to-noise ratios can be set. By adding sample noise to the reverberant signal, noisy reverberation samples can be generated.

[0094] In the above embodiments, the clean dataset is, for example, an open-source dataset, such as some Chinese speech databases, English speech databases, etc., and this application embodiment is not limited to this. The noise samples in the noise dataset are, for example, noise data collected in real-world meeting scenarios.

[0095] Using this approach, electronic devices simulate reverberation and noise environments, select clean speech samples, interfering human voice samples, and noise samples to construct a training sample set, making the training sample set more similar to the real meeting scenario. This enables the trained regional sound pickup model to accurately distinguish noise, target signals within the target area, and interfering human voices outside the target area, thereby improving the accuracy of the regional sound pickup model.

[0096] B. Model building and training.

[0097] During the process of training an initial model using a training sample set to obtain a regional sound pickup model, the electronic device determines the average loss value after each training epoch. If the decrease in the average loss value of this epoch is greater than or equal to a preset range, it indicates that the step size of the parameter adjustment is reasonable, and the electronic device continues training for the next epoch.

[0098] A training epoch refers to completing one training run on all samples in the training sample set. Since the training sample set contains a large number of samples (50,000 pairs), and the memory and computing power of electronic devices are limited, it's impossible to input all the training samples into the initial model at once. Therefore, only one batch of training samples is input at a time, such as 16 pairs. That is, there are a total of 50,000 training samples, and 16 pairs are processed each time. After training 16 pairs, a loss value is determined. After completing one epoch, 3125 loss values ​​are obtained. The average loss value is calculated by averaging these 3125 loss values. Each epoch has an average loss value.

[0099] In this embodiment, the initial model is, for example, a classic convolutional recurrent network (CRN). The input of the model is the complex spectrum of the noisy reverberation sample, and the output of the model is the weight of the complex spectrum. After outputting the weight, the complex spectrum of each noisy reverberation sample is filtered and summed according to the weight to estimate the complex spectrum of the clean speech sample corresponding to the noisy reverberation sample. Then, the estimated complex spectrum is subjected to inverse Fourier transform to estimate the clean speech. The loss value is calculated based on the estimated clean speech and the actual clean speech sample. The loss function is a weighted combination loss function, as shown in the following formula (4):

[0100]

[0101] Where, s(t) and Let S(n,f) be the real clean speech sample and the estimated clean speech, respectively, and let S(n,f) be the complex spectrum of s(t). yes The complex spectrum. The signal-to-interference and noise ratio (SISNR), also known as the time-domain scale-invariant signal-to-noise ratio, uses the mean absolute error (MAE) of the complex spectrum as the absolute error, and γ is the weighting coefficient, with an empirical value of 100.

[0102] During training, the sampling rate for each noisy reverberation sample was 16 kHz. A 32 ms Hanning window was used for STFT, with an overlap rate of 50%. The Adam optimizer was used for training, with an initial learning rate of 0.001. The adjustment strategy was as follows: if the average loss value did not decrease or decreased by a preset amount over five consecutive epochs, the learning rate was reduced to half its original value. Training lasted for a total of 100 epochs.

[0103] When the decrease in the average loss value in a given round is less than a preset threshold, the electronic device determines if there are consecutive rounds with a preset number of rounds in which the decrease in the average loss value in each round is less than the preset threshold (e.g., 5 rounds). If there are fewer than 5 consecutive rounds with a small decrease in the average loss value (e.g., 3 or 2 rounds), the electronic device continues training for the next round. If the decrease in the average loss value is less than the preset threshold for 5 consecutive rounds, it indicates that the parameter adjustment step size is unreasonable. The electronic device reduces the learning rate and then continues training the initial model based on the reduced learning rate until convergence. The converged initial model is used as the region pickup model. The parameters of the converged region pickup model are fixed.

[0104] This approach reduces the learning rate when the average loss value decreases less than a preset range over a set number of consecutive rounds, thereby accelerating the model's convergence and improving training speed.

[0105] Secondly, the reasoning stage.

[0106] During the training phase, when the initial model converges, the converged initial model is used as the region-based sound pickup model, and the parameters of the region-based sound pickup model are fixed. During the inference phase, such as... Figure 3 As shown, the electronic device performs a short-time Fourier transform on each microphone signal to obtain a complex spectrum. This complex spectrum is used as the input to the area pickup model, and the output of the model is the weight of each complex spectrum. Next, the electronic device performs filtering and summation on each complex spectrum to estimate the complex spectrum S(n,f) of the target signal in the target pickup area. Finally, it performs an inverse Fourier transform on the complex spectrum S(n,f) to estimate the target signal within the target pickup area. This target signal is a single-channel speech signal, representing the result of area pickup and noise reduction.

[0107] The above embodiments assume the target pickup area is directly in front of the microphone array, such as... Figures 2A-2C The solid line area is shown. This means the user did not slide the slider on the adjustment interface to change the target pickup area from the preset pickup area. In this case, the pickup direction is directly in front of the microphone array, the sound source is in the far field, and the sound wave can be considered a plane wave. The sound wave arriving at each microphone in the direct front direction arrives at the same time.

[0108] However, in order to adjust or deflect the pickup direction, it is inevitable in practice that the position of the target pickup area will be adjusted. The centerline of the adjusted target pickup area will then have a certain angle with the centerline of the preset pickup area, such as... Figure 2A As shown. In this case, the microphone signals of each microphone are delayed in the time domain, therefore, the complex spectrum of each microphone signal needs to be phase-delayed in the frequency domain.

[0109] To achieve phase delay, the electronic device determines the offset angle of the target pickup area relative to the preset pickup area, and then performs phase delay on the complex spectrum of each microphone signal among multiple microphone signals based on the offset angle. The target pickup area is obtained by offsetting the entire preset pickup area by the offset angle.

[0110] like Figure 2A As shown, since the offset angle is 15 degrees, the electronic device performs phase delay on the complex spectrum of each microphone signal based on this offset angle. For example, the electronic device pre-stores a mapping table that stores the relationship between the offset angle and the phase delay. By querying the mapping table, the electronic device determines the magnitude of the phase delay and performs phase delay on the complex spectrum of each microphone signal based on the magnitude of the phase delay.

[0111] Using this method, when the pickup direction is adjusted, the target pickup area is offset relative to the preset pickup area. The electronic device performs phase delay on the complex spectrum of each microphone signal according to the offset angle, so as to achieve pickup of the area offset in the specified direction, without the need for more sample data when the pickup direction is directly in front.

[0112] Optionally, during the phase delay of the complex spectrum of each microphone signal among multiple microphone signals based on the offset angle, the electronic device determines the time delay of each microphone in the microphone array based on the distance from the microphone to the center point of the microphone array and the offset angle. Then, the electronic device performs phase delay on the complex spectrum of the microphone signals based on the time delay.

[0113] Figure 5 This is a schematic diagram illustrating the directional adjustability of the area pickup method provided in this application embodiment. Please refer to... Figure 5 The gray-filled circle represents the microphone position before the pickup direction is adjusted, and the black-filled circle represents the microphone position after the pickup direction is adjusted and the offset angle is 'a'.

[0114] Please refer to Figure 5 Assuming the distance between the i-th microphone and the center point of the microphone array is d, when the offset angle is a, the propagation distance of the i-th microphone signal increases by b. Therefore, a time delay needs to be added to the i-th microphone signal, as shown in the following formula (5):

[0115]

[0116] Where, τ i The time delay added to the i-th microphone signal is represented by c, which is the speed of sound and can be a fixed value of 340 m / s. According to formula (5), the time-domain delay is converted to the frequency domain as the spectral phase difference, i.e., the phase delay of the complex spectrum. Therefore, the complex spectrum phase delay of the i-th microphone signal is shown in formula (6) below:

[0117]

[0118] X' i (n,f) represents the pair of X i (n,f) is the complex spectrum after phase delay, where j is the imaginary unit.

[0119] Please refer to formulas (5) and (6). For each microphone, the complex spectrum of the microphone signal is phase-delayed according to the distance of the microphone from the center point and the offset angle, thereby achieving the overall directional deflection.

[0120] It should be noted that, for ease of understanding, Figure 5 The offset angle 'a' is illustrated by rotating the entire microphone array by 'a'. In practice, to adjust or deflect the pickup direction, electronic devices such as smart interactive panels are not rotated to drive the microphone array to rotate. Instead, software algorithms are used to adjust or deflect the pickup direction, thereby changing the position of the target pickup area.

[0121] Figure 6 This is another flowchart of the area pickup method provided in this application embodiment. This embodiment includes the following steps:

[0122] 601. Perform a short-time Fourier transform on the signals picked up by each microphone to obtain the complex spectrum of each microphone signal.

[0123] 602. Based on the offset angle, perform phase delay on the complex spectrum of each microphone signal.

[0124] When the pickup direction is adjusted or deflected, the signals from each microphone are delayed in the time domain. The delay time is determined based on the distance between the microphone and the center point of the microphone array, as well as the offset angle. Correspondingly, the complex spectrum of each microphone signal needs to be phase-delayed in the frequency domain.

[0125] 603. Input the phase-delayed complex spectrum into the pre-trained regional pickup model so that the regional pickup model outputs the weights of the complex spectrum of each microphone signal.

[0126] 604. Filter and sum the complex spectra of each microphone signal to estimate the complex spectrum of the target signal in the target pickup area.

[0127] 605. Perform an inverse Fourier transform on the complex spectrum of the estimated target signal to estimate the target signal within the target pickup area.

[0128] 606. Send the estimated target signal to the remote end so that the remote participants hear the accurate target signal.

[0129] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0130] Figure 7 This is a schematic diagram of a local sound pickup device provided in an embodiment of this application. The local sound pickup device 700 includes: a sound pickup module 71, a determination module 72, a processing module 73, and an estimation module 74.

[0131] The pickup module 71 is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noise-band reverberant signal. The noise-band reverberant signal is formed by adding reverberation effect to the target signal in the target pickup area of ​​the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space.

[0132] The determination module 72 is used to determine the complex spectrum of each microphone signal;

[0133] The processing module 73 is used to input the complex spectrum of each microphone signal into the pre-trained area pickup model so that the area pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target pickup area, suppress reverberation effects, and shield interfering human voices and noise.

[0134] The estimation module 74 is used to estimate the target signal in the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

[0135] In one feasible implementation, the processing module 73 inputs the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. Before that, it is also used to construct a training sample set. The training sample set contains multiple pairs of training samples. Each pair of training samples includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of reverberation, interfering human voice, and noise to the sample speech of the sample pickup area in the sample room. The source of the interfering human voice is located outside the sample pickup area in the sample room, and the source of the noise is located at any position in the sample room. The initial model is trained using the training sample set to obtain the regional sound pickup model.

[0136] In one feasible implementation, when the processing module 73 constructs the training sample set, it is used to simulate a preset number of sample rooms. Each sample room has a first number of sound source locations within its sample pickup area, a second number of interference locations outside its sample pickup area, and a third number of noise locations within its sample room. For each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interference voice samples, noise samples, sound source locations, interference locations, and noise locations. The fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.

[0137] In one feasible implementation, when the processing module 73 uses the training sample set to train the initial model to obtain the regional sound pickup model, it is used to determine the average loss value of the current round after each training round. Training one round means: completing one training for all samples in the training sample set.

[0138] When the decrease in the average loss value is less than the preset range, determine whether there are consecutive rounds of a preset number of rounds in which the decrease in the average loss value of each round is less than the preset range; when there are consecutive rounds of a preset number of rounds, reduce the learning rate; continue training the initial model according to the reduced learning rate until convergence, and use the converged initial model as the region pickup model.

[0139] In one feasible implementation, the processing module 73 inputs the complex spectrum of each microphone signal into the pre-trained regional pickup model so that before the regional pickup model outputs the weights of each complex spectrum, it is also used to determine the offset angle of the target pickup area relative to the preset pickup area. The target pickup area is obtained by offsetting the preset pickup area by the offset angle. The complex spectrum of each microphone signal in the multiple microphone signals is phase-delayed according to the offset angle.

[0140] In one feasible implementation, when the processing module 73 performs phase delay on the complex spectrum of each microphone signal among multiple microphone signals according to the offset angle, it is used to determine the time delay of the corresponding microphone for each microphone in the microphone array according to the distance from the microphone to the center point of the microphone array and the offset angle; and to perform phase delay on the complex spectrum of the microphone signal according to the time delay.

[0141] In one feasible implementation, before the pickup module 71 uses each microphone in the microphone array to pick up sound from the target area, the processing module 73 is also used to control the display screen of the electronic device to display an adjustment interface for adjusting the position of the target pickup area; in response to the sliding operation of the slider on the adjustment interface, the position of the target pickup area is adjusted.

[0142] The area pickup device provided in this application embodiment can perform the actions of the electronic device in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0143] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device is, for example, the aforementioned smart interactive flat panel. Please refer to... Figure 8 The electronic device 800 of this application embodiment includes: at least one processor 81, at least one communication bus 82, user interface 83, at least one network interface 84, memory 85, and microphone array 87.

[0144] The communication bus 82 is used to enable communication between these components.

[0145] The user interface 83 may include a display screen and a camera. Optionally, the user interface 83 may also include a standard wired interface and a wireless interface.

[0146] Among them, the network interface 84 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0147] The processor 81 may include one or more processing cores. The processor 81 connects to various parts within the electronic device 800 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 85, and by calling data stored in the memory 85. Optionally, the processor 81 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 81 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the panoramic sphere required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip without being integrated into the processor 81.

[0148] The memory 85 may include random access memory (RAM) or read-only memory. Optionally, the memory 85 may include a non-transitory computer-readable storage medium. The memory 85 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 85 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 85 may also be at least one storage device located remotely from the aforementioned processor 81. Figure 8 As shown, the memory 85, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and operating applications for electronic devices.

[0149] The microphone array 87 can be a linear array microphone, a ring array microphone, etc., and the embodiments of this application are not limited thereto.

[0150] This application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, are used to implement the above-described area pickup method.

[0151] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described area pickup method.

[0152] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0154] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0155] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0156] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0157] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0158] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0159] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0160] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for regional sound pickup, characterized in that, Applied to an electronic device having a microphone array, the method includes: The microphone signals of each microphone are obtained by using each microphone in the microphone array to pick up sound. The microphone signals are noisy reverberation signals. The noisy reverberation signals are formed by adding reverberation effects to the target signal in the target pickup area of ​​the target space, interfering with human voices and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voices is located outside the target pickup area, and the sound source of noise is located at any position in the target space. Determine the complex spectrum of the signals from each microphone; The complex spectrum of each microphone signal is input into a pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise. The target signal within the target pickup area is estimated based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

2. The method according to claim 1, characterized in that, Before inputting the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum, the method further includes: A training sample set is constructed, which contains multiple pairs of training samples. Each pair of training samples includes a clean speech sample and a noisy reverberation sample. The noisy reverberation sample is obtained by adding at least one of reverberation, interfering human voice, and noise to the sample speech in the sample pickup area of ​​the sample room. The source of the interfering human voice is located outside the sample pickup area in the sample room, and the source of the noise is located at any position in the sample room. The initial model is trained using the training sample set to obtain the regional sound pickup model.

3. The method according to claim 2, characterized in that, The construction of the training sample set includes: The simulation uses a preset number of sample rooms. Each sample room has a first number of sound source locations within its sample pickup area, a second number of interference locations outside its sample pickup area, and a third number of noise locations within its sample room. For each sample room, a fourth number of noisy reverberation samples are constructed based on clean speech samples, interfering human voice samples, noise samples, the location of the sound source, the location of the interference, and the location of the noise. The fourth number of training samples in the training sample set is generated based on the clean speech samples and the fourth number of noisy reverberation samples.

4. The method according to claim 2, characterized in that, The process of training an initial model using the training sample set to obtain the region sound pickup model includes: After each training round, the average loss value for that round is determined. One training round refers to completing one training run on all samples in the training sample set. When the decrease in the average loss value is less than a preset range, it is determined whether there are consecutive rounds of a preset number of rounds in which the decrease in the average loss value of each round of the preset number of rounds is less than the preset range. When there are a preset number of consecutive rounds, reduce the learning rate; The initial model is trained again using the reduced learning rate until convergence, and the converged initial model is used as the regional sound pickup model.

5. The method according to any one of claims 1 to 4, characterized in that, Before inputting the complex spectrum of each microphone signal into the pre-trained regional sound pickup model so that the regional sound pickup model outputs the weights of each complex spectrum, the method further includes: The offset angle of the target pickup area relative to the preset pickup area is determined, and the target pickup area is obtained by offsetting the preset pickup area as a whole by the offset angle; The complex spectrum of each microphone signal is phase-delayed according to the offset angle.

6. The method according to claim 5, characterized in that, The step of performing phase delay on the complex spectrum of each microphone signal among the plurality of microphone signals according to the offset angle includes: For each microphone in the microphone array, the time delay of the corresponding microphone is determined based on the distance of the microphone from the center point of the microphone array and the offset angle; The complex spectrum of the microphone signal is phase-delayed according to the time delay.

7. The method according to any one of claims 1 to 4, characterized in that, Before using each microphone in the microphone array to pick up sound from the target area, the method further includes: The interface displays an adjustment feature for adjusting the position of the target pickup area; In response to a sliding operation of the slider on the adjustment interface, the position of the target pickup area is adjusted.

8. A regional sound pickup device, characterized in that, Integrated into an electronic device having a microphone array, the device includes: The sound pickup module is used to pick up sound using each microphone in the microphone array to obtain the microphone signal of each microphone. The microphone signal is a noisy reverberation signal. The noisy reverberation signal is formed by adding reverberation effect to the target signal in the target pickup area of ​​the target space, interfering with human voice and noise. The sound source of the target signal is located in the target pickup area, the sound source interfering with human voice is located outside the target pickup area, and the sound source of noise is located at any position in the target space. The determination module is used to determine the complex spectrum of each microphone signal; The processing module is used to input the complex spectrum of each microphone signal into a pre-trained regional sound pickup model, so that the regional sound pickup model outputs the weights of each complex spectrum. The weights are used to retain the target signal in the target sound pickup area, suppress the reverberation effect, and shield the interfering human voice and the noise. An estimation module is used to estimate the target signal within the target pickup area based on the complex spectrum of each microphone signal and the weight of each complex spectrum.

9. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it causes the electronic device to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images