Near-field and far-field voice separation method and apparatus, and wearable device
Patent Information
- Application Number
- PCT/CN2025/082501
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-03-14
- Publication Date
- 2026-09-17
Smart Images
Figure CN2025082501_17092026_PF_FP_ABST
Abstract
Description
Far and near field speech separation method, device and wearable device
TECHNICAL FIELD
[0001] The present application relates to the technical field of speech processing, in particular to a far and near field speech separation method, device and wearable device.
BACKGROUND
[0002] With the gradual popularity of wearable devices such as True Wireless Stereo (TWS), Augmented Reality (AR) glasses, and the like, speech applications based on wearable devices are also becoming more and more widespread; in speech call, voice assistant, and far field translation application scenarios, the technology of distinguishing between the wearer's speech and other people's speech is crucial.
[0003] In terms of speech call, the wearable device needs to be able to eliminate the background speaker sound outside the wearer to provide a clear call experience; for voice assistant functions, the wearable device needs to be able to recognize the wearer's voice instructions and provide a response; for far field translation scenarios, the wearable device needs to be able to distinguish between the wearer's voice and the far field speaker's voice, and then be able to only translate the far field speech.
[0004] A common solution in the prior art is to use directional beamforming technology to pick up far field speech or near field speech input to related applications based on the multi-mic audio signal picked up by the microphone array. However, this solution can improve the signal-to-noise ratio of near field or far field speech to some extent, but still cannot completely separate near field speech and far field speech.
[0005] Therefore, it is necessary to provide a far and near field speech separation method, device and wearable device.
SUMMARY
[0006] The purpose of the embodiments of the present application is to provide a far and near field speech separation method, device and wearable device.
[0007] The technical solution of the present application is as follows:
[0008] In a first aspect, the embodiments of the present application provide a far and near field speech separation method, comprising:
[0009] According to the microphone array signal collected by the first microphone under the target scene and the beamforming algorithm, a near field beamforming signal and a far field beamforming signal are obtained;
[0010] According to the near field beamforming signal, the far field beamforming signal and the bone conduction microphone signal, a near field speech masking matrix and a far field speech masking matrix are calculated;
[0011] outputting near-field speech according to the near-field beamforming signal and the near-field speech mask matrix;
[0012] outputting far-field speech according to the far-field beamforming signal and the far-field speech mask matrix.
[0013] Further, the bone conduction microphone signal is obtained by the following steps:
[0014] obtaining a wearer speech signal collected by a second microphone in the target scene;
[0015] taking the wearer speech signal as the bone conduction microphone signal.
[0016] Further, the calculation of the near-field speech mask matrix and the far-field speech mask matrix according to the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal comprises:
[0017] inputting the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal into a separation neural network to obtain the near-field speech mask matrix and the far-field speech mask matrix.
[0018] Further, the separation neural network is a convolutional recurrent neural network, and the convolutional recurrent neural network comprises a convolutional neural network, a recurrent neural network and a deep neural network, and the inputting of the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal into the separation neural network to obtain the near-field speech mask matrix and the far-field speech mask matrix comprises:
[0019] inputting the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal into the convolutional neural network to extract near-field audio features and far-field audio features;
[0020] inputting the near-field audio features and the far-field audio features into the recurrent neural network to extract near-field speech information and far-field speech information;
[0021] inputting the near-field speech information and the far-field speech information into the deep neural network to obtain the near-field speech mask matrix and the far-field speech mask matrix.
[0022] Further, the inputting of the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal into the convolutional neural network to extract near-field audio features and far-field audio features comprises:
[0023] If the effective frequency bandwidth of the bone conduction microphone signal is within the preset frequency range, the number of convolution kernels of the convolutional neural network is increased and / or the stride of the convolutional neural network is reduced.
[0024] If the effective frequency bandwidth of the bone conduction microphone signal is outside the preset frequency range, the number of convolution kernels of the convolutional neural network is reduced and / or the stride of the convolutional neural network is increased.
[0025] Further, the near-field beamforming signal and the far-field beamforming signal are obtained according to the microphone array signals collected by the first microphone in the target scene and the beamforming algorithm, and the calculation formula is as follows:
[0026] wherein N t,f represents the near-field beamforming signal, F t,f represents the far-field beamforming signal, S i,t,f represents the time-domain signal S i of the i th microphone array signal, and represents the first filter weight, represents the second filter weight, i, t and f are positive integers.
[0027] Further, the near-field speech is output according to the near-field beamforming signal and the near-field speech masking matrix, and the calculation formula is as follows:
[0028] wherein N' t,f represents the near-field speech, N t,f represents the near-field beamforming signal, represents the near-field speech masking matrix, t and f are positive integers.
[0029] Further, the far-field speech is output according to the far-field beamforming signal and the far-field speech masking matrix, and the calculation formula is as follows:
[0030] wherein F' t,f represents the far-field speech, F t,f represents the far-field beamforming signal, represents the far-field speech masking matrix, t and f are positive integers.
[0031] In a second aspect, an embodiment of the present application provides a near-far field speech separation device, comprising:
[0032] The acquisition module is configured to acquire a near-field beamforming signal and a far-field beamforming signal according to a microphone array signal collected by a first microphone in a target scene and a beamforming algorithm;
[0033] The calculation module is configured to calculate a near-field voice masking matrix and a far-field voice masking matrix according to the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal.
[0034] The first output module is configured to output near-field voice according to the near-field beamforming signal and the near-field voice masking matrix.
[0035] The second output module is configured to output far-field voice according to the far-field beamforming signal and the far-field voice masking matrix.
[0036] In a third aspect, an embodiment of the present application provides a wearable device including the near-field and far-field voice separation device provided in the first aspect.
[0037] The present application has the beneficial effects that the microphone array signal collected by the first microphone in the target scene is acquired, the near-field beamforming signal and the far-field beamforming signal are obtained by using the beamforming algorithm, the near-field voice masking matrix and the far-field voice masking matrix are calculated based on the near-field beamforming signal and the far-field beamforming signal and based on the bone conduction microphone signal, the near-field voice and the far-field voice are effectively separated by fusing the bone conduction microphone signal and the traditional microphone array signal, and thus the wearer's voice and the far-field voice are obtained. BRIEF DESCRIPTION OF DRAWINGS
[0038] FIG. 1 is a schematic diagram of an application scenario of a near-field and far-field voice separation method according to an embodiment of the present application;
[0039] FIG. 2 is a flowchart of a near-field and far-field voice separation method according to an embodiment of the present application;
[0040] FIG. 3 is a flowchart of a near-field and far-field voice separation method according to an embodiment of the present application;
[0041] FIG. 4 is a schematic diagram of a structure of a near-field and far-field voice separation device according to an embodiment of the present application;
[0042] FIG. 5 is a schematic diagram of a structure of a wearable device according to an embodiment of the present application. DETAILED DESCRIPTION
[0043] The present application will be further described below in combination with the drawings and embodiments.
[0044] Fig. 1 is a schematic diagram of an application scenario of a near-far field voice separation method provided by an embodiment of the present application. As shown in Fig. 1, the method can be applied to TWS, AR glasses and other wearable devices. The AR glasses are taken as an example for illustration. Generally, the AR glasses include a plurality of first microphones. A wearer wears the AR glasses. When a controller inside the AR glasses receives a separation instruction, near-field beamforming signals and far-field beamforming signals are obtained according to microphone array signals collected by the first microphones in a target scene and a beamforming algorithm. Near-field voice masking matrices and far-field voice masking matrices are calculated according to the near-field beamforming signals, the far-field beamforming signals and bone conduction microphone signals. Near-field voice is output according to the near-field beamforming signals and the near-field voice masking matrices. Far-field voice is output according to the far-field beamforming signals and the far-field voice masking matrices. Thus, the AR glasses can well separate the near-field voice and the far-field voice, eliminate the far-field voice outside the wearer and provide the wearer with clear call experience, thereby improving the user experience of the wearer for the AR glasses.
[0045] Fig. 2 is a flowchart of a near-far field voice separation method provided by an embodiment of the present application. As shown in Fig. 2, microphone array signals collected by the first microphones are subjected to a near-far field beamforming algorithm to obtain near-field beam signals and far-field beam signals. The near-field beam signals, the far-field beam signals and bone conduction microphone signals are subjected to a near-far field separation algorithm to obtain masking matrices corresponding to near-far field voice. The near-field beam signals and the far-field beam signals are respectively fused with the corresponding masking matrices. The near-far field beam signals are multiplied by the corresponding masking matrices to obtain separated near-field voice and far-field voice.
[0046] Fig. 3 is a flowchart of a near-far field voice separation method provided by an embodiment of the present application. As shown in Fig. 3, the method includes:
[0047] A near-far field voice separation method provided by an embodiment of the present application is taken as an example for illustration. The AR glasses include five first microphones, which are all conventional microphones. Four of the first microphones are respectively located on two temple legs of the AR glasses, as shown in Fig. 1 at positions marked by reference numerals 1 to 5. The other first microphone is located on a nose pad of the AR glasses. The AR glasses further include one second microphone, which is also located on the nose pad of the AR glasses, as shown in Fig. 1 at a position marked by reference numeral 6. The layout can well utilize the space on the glasses and provide more flexibility for the beamforming algorithm with the existence of microphones with small and large distances.
[0048] The target scene can be that the wearer wears AR glasses and talks with others, in which the wearer's voice is near-field speech and the voice of the other person is far-field speech. By performing the near-field and far-field speech separation method, the voices of the wearer and the other person can be effectively separated, thereby improving the user experience of the wearer using the AR glasses.
[0049] S110, obtaining a near-field beamforming signal and a far-field beamforming signal according to a microphone array signal collected by the first microphone in the target scene and a beamforming algorithm;
[0050] The microphone array signal collected by the first microphone in the target scene is obtained, which includes the audio signals collected by the above-mentioned five first microphones and contains both the audio of the wearer speaking and the audio of the other person speaking beside. After obtaining the microphone array signal, the near-field beamforming signal and the far-field beamforming signal can be obtained through the beamforming algorithm.
[0051] The beamforming algorithm is a signal processing technology, and the core concept of the beamforming algorithm is to perform weighted summation on the microphone array signals received by each first microphone in the AR glasses, thereby forming a beam with a specific directivity. This beam can enhance the signal strength in the desired direction while suppressing the interference signals in other directions.
[0052] For example, when the near-field beamforming signal and the far-field beamforming signal are obtained by using the beamforming algorithm, a signal model needs to be established to distinguish the near field and the far field first. Then the steering vector is constructed, and the near-field steering vector and the far-field steering vector can be obtained. Based on the near-field steering vector and the far-field steering vector, the covariance matrix estimation is performed, and finally the separation is performed based on the minimum variance distortionless response algorithm or based on the subspace method, thereby obtaining the near-field beamforming signal and the far-field beamforming signal.
[0053] S120, calculating a near-field speech masking matrix and a far-field speech masking matrix according to the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal;
[0054] After obtaining the near-field beamforming signal and the far-field beamforming signal in the above-mentioned steps, the near-field speech masking matrix is calculated according to the relationship between the near-field beamforming signal and the bone conduction microphone signal, and the far-field speech masking matrix is calculated according to the relationship between the far-field beamforming signal and the bone conduction microphone signal.
[0055] Wherein, in the near-field speech processing, the near-field speech refers to the speech signal of a speaker close to the microphone (usually within tens of centimeters). Due to the relatively complex near-field environment, various background noises and interference signals may exist; the near-field speech masking matrix is used to describe the mutual relationship between the speech signal and these interference signals, and plays a key role especially in the process of separating or enhancing the speech signal.
[0056] In the far-field speech environment, the speech signal source is relatively far away from the microphone array. At this time, the signal propagation process will be disturbed by more environmental factors, such as reverberation, long-distance background noise, etc. The far-field speech masking matrix is mainly used to process this complex far-field speech situation, the purpose of which is to highlight the target speech signal and suppress the interference signal.
[0057] S130, outputting near-field speech according to the near-field beamforming signal and the near-field speech masking matrix;
[0058] S140, outputting far-field speech according to the far-field beamforming signal and the far-field speech masking matrix.
[0059] After calculating the near-field speech masking matrix according to the above method, the near-field beamforming signal is further refined and denoised according to the near-field beamforming signal and the near-field speech masking matrix, and the near-field speech is obtained; the far-field beamforming signal is further refined and denoised according to the far-field beamforming signal and the far-field speech masking matrix, and the far-field speech is obtained.
[0060] For example, the near-field beamforming signal and the near-field speech masking matrix are multiplied to obtain the near-field speech; the far-field beamforming signal and the far-field speech masking matrix are multiplied to obtain the far-field speech.
[0061] The embodiment of the present application provides a near-far field speech separation method, which first acquires a microphone array signal collected by a first microphone in a target scene, and obtains a near-field beamforming signal and a far-field beamforming signal by using a beamforming algorithm; based on the near-field beamforming signal and the far-field beamforming signal, a near-field speech masking matrix and a far-field speech masking matrix are calculated based on a bone conduction microphone signal, and by fusing the bone conduction microphone signal with a traditional microphone array signal, the near-field speech and the far-field speech can be effectively separated, so as to obtain a wearer's voice and a far-field voice.
[0062] As an implementation manner, the bone conduction microphone signal is obtained by the following steps:
[0063] Acquiring a wearer's speech signal collected by a second microphone in the target scene;
[0064] Taking the wearer's speech signal as the bone conduction microphone signal.
[0065] In the embodiments of the present application, the voice signal of the wearer in the target scene is collected through a second microphone on the AR glasses. Generally, the second microphone can be located on the nose pad of the AR glasses to collect the speaking voice of the wearer at a close distance.
[0066] In the actual implementation process, after the voice signal of the wearer is collected, the voice signal of the wearer is generally transformed into a frequency domain. The voice signal of the wearer after being transformed into the frequency domain is used as the bone conduction microphone signal.
[0067] The second microphone can be a voice accelerometer (VACC) or a voice pickup sensor (VPU). The voice signals collected by the voice accelerometer and the voice pickup sensor can capture the voice signal of the wearer through the bone, muscle and other tissues, effectively isolate the environmental noise and interference voice from the outside world, and thus improve the accuracy and clarity of voice pickup.
[0068] In some embodiments, the calculation of the near-field voice masking matrix and the far-field voice masking matrix according to the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal comprises:
[0069] The near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal are input into a separation neural network to obtain the near-field voice masking matrix and the far-field voice masking matrix.
[0070] In the embodiments of the present application, the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal are input into a separation neural network. The separation neural network can output the near-field voice masking matrix and the far-field voice masking matrix.
[0071] As an implementation manner, the separation neural network is a convolutional recurrent neural network, which includes a convolutional neural network, a recurrent neural network and a deep neural network. The input of the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal into the separation neural network to obtain the near-field voice masking matrix and the far-field voice masking matrix comprises:
[0072] The near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal are input into the convolutional neural network to extract near-field audio features and far-field audio features.
[0073] The near-field audio features and the far-field audio features are input into the recurrent neural network to extract near-field voice information and far-field voice information.
[0074] input the near-field voice information and the far-field voice information into the deep neural network to obtain the near-field voice masking matrix and the far-field voice masking matrix.
[0075] In the embodiment of the present application, the separation neural network is a convolutional recurrent neural network, which is composed of a convolutional neural network (CNN), a recurrent neural network (RNN) and a deep neural network (DNN). The steps of obtaining the near-field voice masking matrix and the far-field voice masking matrix are as follows:
[0076] First, the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal are combined into an input feature, which can be represented as Fea∈C T×F×C where T represents the number of signal frames, F represents the number of signal frequency bands, and C represents the number of input channels. In the embodiment of the present application, C=3, indicating that the wearable device includes three channels of near field, far field and bone conduction.
[0077] The input feature is then input into the convolutional neural network for feature extraction to extract near-field audio features and far-field audio features. Then, the near-field audio features and the far-field audio features are input into the recurrent neural network, which can model the time series of the voice signal to extract the near-field voice information and the far-field voice information. Finally, the near-field voice information and the far-field voice information are input into the deep neural network to obtain the near-field voice masking matrix and the far-field voice masking
[0078] As an implementation manner, before the step of inputting the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone into the convolutional neural network to extract near-field audio features and far-field audio features, the step includes:
[0079] If the effective frequency width of the bone conduction microphone signal is within the preset frequency range, the number of convolution kernels of the convolutional neural network is increased and / or the stride of the convolutional neural network is reduced.
[0080] If the effective frequency width of the bone conduction microphone signal is outside the preset frequency range, the number of convolution kernels of the convolutional neural network is reduced and / or the stride of the convolutional neural network is increased.
[0081] In the embodiment of the application, before the bone conduction microphone signal is input into the convolutional neural network, a relationship between an effective frequency bandwidth of the bone conduction microphone signal and a preset frequency range is compared. If the effective frequency bandwidth of the bone conduction microphone signal is within the preset frequency range, the number of convolution kernels of the convolutional neural network is increased, and / or the stride of the convolutional neural network is reduced. If the effective frequency bandwidth of the bone conduction microphone signal is outside the preset frequency range, the number of convolution kernels of the convolutional neural network is reduced, and / or the stride of the convolutional neural network is increased.
[0082] For example, the preset frequency range is 0-2KHz. In combination with the frequency response characteristics of the VACC / VPU signal, different network structures can be used for different frequency bands, so as to effectively use the VACC / VPU signal and reduce the overall complexity. For example, the effective frequency bandwidth of the VACC is only about 2KHz. For the frequency band below 2KHz, the convolutional neural network can use more convolution kernels and smaller stride to better use the VACC to separate the near-field and far-field speech through fine modeling. For the frequency band above 2KHz, fewer convolution kernels and larger stride can be used to reduce the overall complexity of the convolutional neural network.
[0083] As an example, the near-field beamforming signal and the far-field beamforming signal are obtained according to the microphone array signal collected by the first microphone in the target scene and the beamforming algorithm, and the calculation formula is as follows:
[0084] wherein, N t,f represents the near-field beamforming signal, F t,f represents the far-field beamforming signal, S i,t,f represents the time-domain signal S i of the i-th microphone array signal, and represents the first filter weight, represents the second filter weight, i, t, and f are positive integers.
[0085] It should be noted that the first filter weight and the second filter weight are determined according to the specific layout information of the microphone.
[0086] As an example, the near-field speech is output according to the near-field beamforming signal and the near-field speech masking matrix, and the calculation formula is as follows:
[0087] wherein, N' t,f represents the near-field speech, N t,f represents the near-field beamforming signal, denotes the near-field speech masking matrix, t and f are positive integers.
[0088] As an example, the far-field speech is output according to the far-field speech masking matrix and the far-field beamforming signal, and the calculation formula is as follows:
[0089] wherein F' is the far-field speech masking matrix, t and f are positive integers. t,f denotes the far-field speech, F t,f denotes the far-field beamforming signal, denotes the near-field speech masking matrix, t and f are positive integers.
[0090] Fig. 4 is a structural schematic diagram of a near-far field speech separation device provided by an embodiment of the present application, as shown in Fig. 4, the device comprises:
[0091] The acquisition module 210 is configured to acquire the near-field beamforming signal and the far-field beamforming signal according to the microphone array signal collected by the first microphone in the target scene and the beamforming algorithm.
[0092] The calculation module 220 is configured to calculate the near-field speech masking matrix and the far-field speech masking matrix according to the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal.
[0093] The first output module 230 is configured to output the near-field speech according to the near-field beamforming signal and the near-field speech masking matrix.
[0094] The second output module 240 is configured to output the far-field speech according to the far-field beamforming signal and the far-field speech masking matrix.
[0095] The embodiment of the present application is a device corresponding to the above method, and the implementation process is the same as the above method embodiment. For details, refer to the above method embodiment, and the device embodiment will not be described here.
[0096] Fig. 5 is a structural schematic diagram of a wearable device provided by an embodiment of the present application, as shown in Fig. 5, the wearable device 300 comprises the above near-far field speech separation device 200. The wearable device comprises TWS, AR glasses and the like.
[0097] The above is only an embodiment of the present application, and it should be noted that for those skilled in the art, without departing from the creative concept of the present application, improvements can be made, but these all belong to the protection scope of the present application.
Claims
1. A method for separating near and far-field speech, characterized in that, include: Based on the microphone array signal collected by the first microphone in the target scene and the beamforming algorithm, near-field beamforming signal and far-field beamforming signal are obtained. Based on the near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal, the near-field speech masking matrix and the far-field speech masking matrix are calculated. Based on the near-field beamforming signal and the near-field speech masking matrix, output near-field speech; Based on the far-field beamforming signal and the far-field speech masking matrix, far-field speech is output.
2. The near-field and far-field speech separation method according to claim 1, characterized in that, The bone conduction microphone signal is obtained through the following steps: Acquire the wearer's voice signal collected by the second microphone in the target scenario; The wearer's voice signal is used as the bone conduction microphone signal.
3. The near-field and far-field speech separation method according to claim 1, characterized in that, The calculation of the near-field speech masking matrix and the far-field speech masking matrix based on the near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal includes: The near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal are input into a separation neural network to obtain the near-field speech masking matrix and the far-field speech masking matrix.
4. The near-field and far-field speech separation method according to claim 3, characterized in that, The separating neural network is a convolutional recurrent neural network, which includes convolutional neural networks, recurrent neural networks, and deep neural networks. The step of inputting the near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal into the separating neural network to obtain the near-field speech masking matrix and the far-field speech masking matrix includes: The near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal are input into the convolutional neural network to extract near-field audio features and far-field audio features; The near-field audio features and the far-field audio features are input into the recurrent neural network to extract near-field speech information and far-field speech information; The near-field speech information and the far-field speech information are input into the deep neural network to obtain the near-field speech masking matrix and the far-field speech masking matrix.
5. The near-field and far-field speech separation method according to claim 4, characterized in that, The step of inputting the near-field beamforming signal, the far-field beamforming signal, and the bone conduction microphone signal into the convolutional neural network to extract near-field and far-field audio features includes: If the effective bandwidth of the bone conduction microphone signal is within a preset frequency range, the number of convolutional kernels in the convolutional neural network will be increased and / or the stride of the convolutional neural network will be decreased. If the effective bandwidth of the bone conduction microphone signal is outside the preset frequency range, then the number of convolutional kernels of the convolutional neural network is reduced and / or the stride of the convolutional neural network is increased.
6. The near-field and far-field speech separation method according to claim 1, characterized in that, The near-field beamforming signal and far-field beamforming signal are obtained based on the microphone array signal collected by the first microphone in the target scene and the beamforming algorithm. The calculation formula is as follows: Where, N t,f F represents the near-field beamforming signal. t,f S represents the far-field beamforming signal. i,t,f S represents the time-domain signal of the i-th microphone array. i The frequency domain signal of the f-th frequency point in the t-th speech frame after SIFT transformation Indicates the weights of the first filter. The values represent the weights of the second filter, where i, t, and f are all positive integers.
7. The near-field and far-field speech separation method according to claim 1, characterized in that, The near-field speech is output based on the near-field beamforming signal and the near-field speech masking matrix, and the calculation formula is as follows: Where, N′ t,f N represents the near-field speech. t,f This indicates the near-field beamforming signal. Let f represent the near-field speech masking matrix, where t and f are both positive integers.
8. The near-field and far-field speech separation method according to claim 1, characterized in that, The far-field speech is output based on the far-field beamforming signal and the far-field speech masking matrix, and the calculation formula is as follows: Among them, F′ t,f F represents the far-field speech. t,f This represents the far-field beamforming signal. Let f represent the far-field speech masking matrix, where t and f are both positive integers.
9. A near-field and far-field speech separation device, characterized in that, include: The acquisition module is used to acquire near-field beamforming signals and far-field beamforming signals based on the microphone array signals acquired by the first microphone in the target scene and the beamforming algorithm. The calculation module is used to calculate the near-field speech masking matrix and the far-field speech masking matrix based on the near-field beamforming signal, the far-field beamforming signal and the bone conduction microphone signal; The first output module is used to output near-field speech based on the near-field beamforming signal and the near-field speech masking matrix; The second output module is used to output far-field speech based on the far-field beamforming signal and the far-field speech masking matrix.
10. A wearable device, characterized in that, Includes the near-field and far-field speech separation device as described in claim 9.