Voice extraction method, system, medium and robot in robot interaction process

Abstract

The application discloses a speech extraction method and system in a robot interaction process, a medium and a robot, designs an artificial intelligence technology field, and comprises the following steps: obtaining registered audio and mixed speech of a target speaker; extracting frame-level voiceprint features from the registered audio and pooling the frame-level voiceprint features into sentence-level voiceprint features, and extracting sub-band embedding sequences of multiple different sub-bands from the mixed speech; inputting fused features obtained by fusing the sub-band embedding sequences and the sentence-level voiceprint features into a speaker extraction module; the speaker extraction module divides the frequency spectrum into multiple sub-frequency spectrum graphs according to predefined frequency bands; each sub-frequency spectrum graph is mapped into a sub-band embedding sequence, and all the sub-band embedding sequences are input into a mixed expert model; target speaker features after separation are obtained; and target speech of the target speaker is obtained after decoding the target speaker features; and the method realizes robust, stable and high-fidelity extraction of the target speaker's speech.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to methods, systems, media, and robots for voice extraction during robot interaction. Background Technology

[0002] As service robots and companion robots become increasingly common in homes, nursing homes, and hospitals, voice has become one of the most natural and frequently used human-computer interaction methods. Unlike traditional voice control in quiet, single-person environments, companion robots operate in open spaces such as living rooms, hospital wards, and public activity areas, presenting complex situations including multiple family members speaking simultaneously, continuous television or radio noise, and the target user's constantly shifting location. If a robot cannot understand what the designated person is saying in a noisy environment, it will struggle to provide safe and reliable companionship, care, and emotional communication.

[0003] Most existing speaker extraction methods are based at the intersection of end-to-end speech separation and speaker recognition technologies. The basic idea is to provide a segment of the target speaker's registration speech or other prior information along with the input multi-speaker mixed speech. A fixed-dimensional speaker embedding vector is extracted by a voiceprint encoder, and this vector is injected as a conditional signal into the separation network so that the model retains only the components that match the embedding during time-frequency masking or time-domain reconstruction.

[0004] Early representative works, such as SpeakerBeam, employ a frequency domain masking framework and use an adaptive network to fuse the target speaker features with mixed speech features to enhance the target speech and suppress interference. Google's VoiceFilter series uses pre-trained d-vectors / x-vectors as conditions to inject speaker embeddings into the Conv / RNN structure through concatenation, gating, or feature recalibration, which significantly reduces the error rate of ASR in multi-speaker scenarios. In terms of network backbone, many methods in recent years have adopted temporal separation structures such as Conv-TasNet (temporal speech separation network), DPRNN (dual-path recurrent neural network), and SepFormer (speech separator) to fuse speaker embeddings and encoded temporal features in the channel dimension or attention module. This avoids the phase problem caused by frequency domain processing and takes into account the modeling capabilities of low latency and wideband signals. Furthermore, specific speaker separation in multi-channel scenarios generally introduces the idea of ​​neural beamforming: using deep networks to estimate the target mask and spatial covariance matrix, constructing beamformers such as MVDR (minimum variance distortionless response) to achieve joint optimization of speaker conditions and array gain.

[0005] For companion robots that operate in open environments such as homes, nursing homes, and hospital wards, the aforementioned speaker extraction methods have the following drawbacks: First, the scene simultaneously features multi-speaker dialogues and complex background noise, with dynamic changes in sound source location and distance. Second, the robot needs to maintain stable recognition even with users whose acoustic characteristics fluctuate significantly, such as the elderly and children. Existing speaker extraction methods that rely solely on frame-level overall features and single speaker embedding vectors have limited adaptability to scene switching, noise spectrum changes, and multi-person conversations, making it difficult to consistently provide high signal-to-noise ratio and low-distortion target speech for speech recognition and semantic understanding modules. Summary of the Invention

[0006] Based on the technical problems existing in the background technology, this invention proposes a speech extraction method, system, medium and robot in the process of robot interaction, which realizes robust, stable and high-fidelity extraction of the speech of the target speaker.

[0007] The speech extraction method for robot interaction proposed in this invention includes: Obtain the registered audio and mixed speech of the target speaker; Frame-level voiceprint features are extracted from the registered audio and pooled into sentence-level voiceprint features. Subband embedding sequences of multiple different subbands are extracted from the mixed speech. The fused feature obtained by fusing the sub-band embedding sequence with the sentence-level voiceprint feature is input into the speaker extraction module; In the speaker extraction module, the fused features of each sub-band are input into the hybrid expert model, which includes a globally shared expert and multiple independent experts. The outputs of each independent expert are adaptively fused through a gating network to obtain the separated target speaker features. The target speech of the target speaker is obtained by decoding the characteristics of the target speaker.

[0008] Furthermore, the extraction of subband embedding sequences from multiple different subbands of mixed speech specifically involves: The spectrogram of the mixed speech is divided into sub-spectrums of different frequency sub-bands according to the bandwidth of the predefined frequency band; Each sub-spectrum is mapped to a sub-band embedding sequence with a unified feature dimension.

[0009] Furthermore, in the global shared expert group, all subbands share the same set of parameters.

[0010] Furthermore, the forward computation of the hybrid expert model is represented as follows: ; in, For the first The output features corresponding to each sub-band For the first The fusion features corresponding to each sub-band For global sharing of experts, Trainable parameters from experts are shared globally across all subbands. For the first An independent expert, For the first Trainable parameters for an independent expert For gating networks The output of the first The activation weight of an individual expert To activate the top K independent experts, sorted in descending order of weight.

[0011] Furthermore, the training process of the speaker extraction module is as follows: The system acquires the registered audio, the mixed speech, and the original speech paired with the mixed speech. It then performs pre-denoising processing on the original speech to obtain the real speech. After feature extraction from the registered audio and mixed speech, the two are fused and input into the speaker extraction module to obtain the target speech of the target speaker. The target speech of the target speaker and the real speech are used to construct a scale-invariant signal-to-noise ratio loss, and the trainable parameters of the speaker extraction module are updated.

[0012] Furthermore, the pre-denoising process employs a pre-trained ResNet34 denoising subnetwork.

[0013] Furthermore, the scale-invariant signal-to-noise ratio loss The calculation formula is as follows: ; in, The target speech of the target speaker. This is real voice recording.

[0014] A computer system includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the method described above.

[0015] A computer-readable storage medium storing a plurality of computer programs for being invoked by a processor and executing the method described above.

[0016] A companion robot includes a microphone array, a memory, and a processor. The memory stores a computer program, and the processor executes the program to implement the method described above.

[0017] The advantages of the speech extraction method, system, medium, and robot provided by this invention in robot interaction are as follows: Firstly, by introducing a fine-grained frequency band modeling mechanism based on hybrid experts, target features and interference features in different frequency bands can be learned differentially, enhancing the ability to distinguish complex spectral structures. Secondly, in the loss design, noise reduction preprocessing is performed on the original speech to weaken the adverse effects of noise components on parameter updates. The synergistic effect of these two aspects enables the speech extraction method to achieve robust, stable, and high-fidelity extraction of a specific speaker's speech even in complex environments with multiple noises, multiple speakers, and strong interference, providing reliable acoustic front-end support for the speech interaction of companion robots in real-world scenarios. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0019] The technical solution of the present invention will now be described in detail through specific embodiments. Many specific details are set forth in the following description to provide a thorough understanding of the invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0020] like Figure 1 As shown, the speech extraction method in the robot interaction process proposed in this invention includes the following steps: Step 1: Obtain the registered audio and mixed speech of the target speaker; Step 2: Extract frame-level voiceprint features from the registered audio and pool them into sentence-level voiceprint features; extract sub-band embedding sequences of multiple different sub-bands from the mixed speech. Step 3: Fuse the sub-band embedding sequence with the sentence-level voiceprint features, and input the resulting fused features into the speaker extraction module; Step 4: In the speaker extraction module, the fused features of each sub-band are input into the hybrid expert model. The hybrid expert model includes a globally shared expert and multiple independent experts. The outputs of each independent expert are adaptively fused through a gating network to obtain the separated target speaker features. Step 5: After decoding the target speaker's features, the target speech of the target speaker is obtained.

[0021] In the complex and variable acoustic environment of companion robots, this embodiment breaks through the limitations of traditional whole-frame modeling, distinguishes and suppresses noise and interference in different frequency bands at a finer-grained time-frequency scale, and achieves adaptive enhancement of target speech by combining prior information of the target speaker, thereby significantly improving the robustness and stability of specific speaker extraction.

[0022] This embodiment sets up a feature extraction model, including a speech encoder, a speaker encoder, a speaker extraction module, and a speech decoder. The registered audio of the target speaker and the mixed speech are used as inputs to the feature extraction model, and the target speech of the target speaker is output accordingly.

[0023] The speech encoder can be a ResNet34 or Transformer encoder, and the speaker encoder can be a learnable convolutional encoder.

[0024] In one embodiment, step one, obtaining the registered audio and mixed speech of the target speaker, specifically (a1) to (a7): Registration audio is a pre-collected voice sample used to identify the target speaker. As one embodiment of registration audio acquisition in a companion robot application scenario, the acquisition of registration audio typically includes the following sub-steps: (a1) Registration guidance: The robot prompts the target user (such as the elderly or children) to read a fixed text (e.g., I am so-and-so, please remember my voice) or to make any self-spoken statement (e.g., hello, robot). The collection time is generally 3 to 10 seconds.

[0025] (a2) Environmental control: The registration process should be carried out in a relatively quiet environment, or the robot should actively issue guidance instructions to ask other personnel to be quiet temporarily in order to reduce interference.

[0026] (a3) Audio acquisition: The target speaker’s voice is acquired through the robot’s microphone array. The sampling rate is usually 16kHz and the quantization accuracy is 16bit.

[0027] (a4) Preprocessing: Endpoint detection (VAD) is performed on the acquired registered audio to remove the first and last silent segments; optional mild background noise suppression (such as spectral subtraction) is performed, but the original timbre characteristics are preserved.

[0028] (a5) Quality check: Calculate the signal-to-noise ratio (SNR) or the percentage of voice activity. If the quality does not meet the preset threshold (e.g., SNR < 15dB), then guide the user to record again.

[0029] (a7) Storage: Store the processed registered audio in the robot's local or cloud storage and bind it to the user ID.

[0030] Mixed speech is a composite audio recording acquired in real time by the robot during actual interaction, containing the target speaker's voice, other interfering human voices, and environmental noise. Its acquisition process includes (b1) to (b5): (b1) Continuous listening: When the robot is in a state of waiting to interact, the microphone array continuously collects the surrounding acoustic signals.

[0031] (b2) Multi-channel acquisition: If the robot is equipped with a multi-microphone array (such as a 4-microphone ring array or a 6-microphone linear array), the time domain signals of multiple channels can be acquired simultaneously, which is convenient for subsequent beamforming or spatial filtering.

[0032] (b3) Activity detection: Use voice activity detection (VAD) or wake word detection (such as Xiaohu Tongxue) to determine whether there is human voice input and trigger subsequent processing.

[0033] (b4) Cache and segmentation: Cache mixed speech segments for a fixed duration (e.g., 1 second) or in a continuous streaming manner as input to the speaker extraction module.

[0034] The obtained registration audio and mixed speech have the following relationships (c1) to (c6): (c1) Identity Identity: The registered audio and the target speaker's voice in the mixed speech belong to the same natural person. The registered audio is used to extract the speaker's voiceprint embedding as a target identity label; the target component in the mixed speech is the object to be extracted.

[0035] (c2) Temporal independence: Registration audio is usually collected once when the robot is used for the first time or when the user changes, while mixed speech is collected in real time during each interaction. The collection time of the two can differ by several days or even months, but it is required that the acoustic characteristics of the target speaker remain relatively stable.

[0036] (c3) Content independence: The text content of the registered audio (e.g., I am Zhang San) can be completely different from the text content of the target speaker in the mixed speech (e.g., pour me a glass of water). The extraction method relies on voiceprint features rather than speech content, and therefore possesses text independence.

[0037] (c4) Signal-to-noise ratio difference: Registration audio usually requires a high signal-to-noise ratio (≥20dB) to ensure the accuracy of voiceprint embedding; mixed speech is in a complex acoustic environment, and the signal-to-noise ratio may be as low as 0dB or even negative, including multi-person conversations, TV sound, air conditioner noise, etc.

[0038] (c5) The pairing relationship between registered audio and mixed speech during the training phase of the feature extraction model; During feature extraction model training, a training sample contains: Registration audio: A clean audio clip from the target speaker; Mixed speech: artificially synthesized by superimposing interfering human voices and environmental noise into another speech of the target speaker (different from the registered audio content); Original speech (label): Speech paired with mixed speech.

[0039] Therefore, during the training phase, the target speaker components in the registered audio and mixed speech come from different speech segments of the same speaker, but their identities are consistent.

[0040] (c6) Mapping relationships in the reasoning stage; In actual deployment of the feature extraction model, the robot uses pre-stored registered audio to generate sentence-level voiceprint features through a speech encoder. It then uses a speaker encoder to extract features from the real-time collected mixed speech to obtain a sub-band embedding sequence. The sentence-level voiceprint features and the sub-band embedding sequence are sent to the speaker extraction module. The speaker extraction module filters out the features belonging to the target speaker from the sub-band embedding sequence. After decoding by the speech decoder, the target speech of the target speaker is obtained.

[0041] In this embodiment, after obtaining the registered audio, the registered audio is processed by a speech encoder to extract features to obtain frame-level voiceprint features, and then a pooling layer is used to obtain sentence-level voiceprint features.

[0042] Specifically, the original waveform of the registered audio is subjected to a short-time Fourier transform (STFT) to obtain the original spectrogram, or acoustic features such as Mel-frequency cepstral coefficients (MFCC) and Mel-spectrogram are directly extracted and used as input to the neural network. The frame length of the input features is typically 25ms (configurable), and the frame shift is 10ms (configurable).

[0043] Neural networks can employ structures such as Time-Delayed Neural Networks (TDNNs), Convolutional Neural Networks (CNNs), or Transformers to perform nonlinear transformations on the acoustic features of each frame, outputting frame-level voiceprint features. These frame-level voiceprint features have variable lengths (depending on the duration of the registered audio). Pooling layers aggregate these variable-length frame-level voiceprint features into fixed-dimensional sentence-level voiceprint features.

[0044] Speaker encoders can employ ECAPA-TDNN (a deep learning-based voiceprint recognition model) and ResNet34 (a deep residual neural network), among others.

[0045] In this embodiment, after acquiring the mixed speech, a speech encoder extracts sub-band embedding sequences of multiple different sub-bands from the mixed speech. That is, the spectrogram of the mixed speech is divided into sub-spectrums of different frequency sub-bands according to the bandwidth of a predefined frequency band; and each sub-spectrum is mapped to a sub-band embedding sequence with a unified feature dimension.

[0046] Specifically, addressing the shortcomings of traditional methods that only model at the whole-frame level and cannot distinguish information from different frequency bands, this paper first performs frequency band division and fine-scale modeling on the spectrogram of mixed speech. Specifically, the complete spectrogram... Sub-spectrum diagram showing the division of the bandwidth of a predefined frequency band into different frequency sub-bands. , For the first Each sub-spectrum corresponds to a relatively independent frequency sub-band. This predefined frequency band is not fixed, but can be adaptively adjusted according to different scenarios where the companion robot is located (such as a family living room, hospital ward, corridor, etc.) as well as noise type and speaker characteristics. In order to dynamically optimize the frequency band division strategy in cases where high-frequency noise is obvious, low-frequency ambient sound dominates, or speech energy distribution is abnormal, the frequency resolution capability and scene adaptability of specific speakers can be improved from the source.

[0047] After completing the spectrum partitioning, each sub-spectrum map is mapped to a series of sub-band embedding sequences with a unified feature dimension. In one specific embodiment, the speech encoder outputs sub-band embedding sequences of 8 sub-bands, each sub-band embedding sequence being 64-dimensional and having a time frame count of T, thus the shape of the sub-band embedding sequence is [8, 64, T]. The sentence-level voiceprint features are 256-dimensional. The sentence-level voiceprint features are copied T times and then concatenated with the sub-band embedding sequences along the feature dimensions to obtain the fused features, with a shape of [8, 64+256, T]=[8, 320, T].

[0048] In one embodiment, step four involves inputting the fused features obtained by fusing the sub-band embedding sequence with the sentence-level voiceprint features into the speaker extraction module; specifically: The fused features, obtained by fusing sub-band embedding sequences and sentence-level speaker characteristics, are fed into the Mixture of Experts (MOE) model for speaker extraction, where they are used for modeling and inference. The MOE model includes both a globally shared expert modeling global patterns and independent experts providing fine-grained representations of characteristics across different frequency bands. The shared expert captures the common features and overall speech patterns of the target speaker across frequency bands, while the independent experts focus on local differences and subtle variations within each sub-band. Through gating or weighting mechanisms, the MOE adaptively selects and fuses the outputs of different experts based on the frequency sub-band attributes of the input features and scene information. This allows for feature extraction and enhancement of a specific speaker at a finer-grained time-frequency scale, significantly improving the accuracy and robustness of speaker extraction in complex and variable acoustic environments.

[0049] The forward computation formula for the hybrid expert model is expressed as: ; in, For the first The output features corresponding to each sub-band For the first The fusion features corresponding to each sub-band For global sharing of experts, Trainable parameters from experts are shared globally across all subbands and are responsible for learning the common timbre benchmarks of the target speaker and the speech envelope trends across frequency bands. For the first Each expert j is an independent expert, and each expert j has independent parameters. Each corresponds to a different frequency band response characteristic; For the first Trainable parameters for an independent expert For gating networks The output of the first The activation weight of an individual expert To activate the top K independent experts, sorted in descending order of weight.

[0050] (d1) The Global Shared Expert is a neural network module that shares parameters across all subbands to extract frequency-independent common features. Since all subbands share the same set of parameters, the number of parameters in the speaker extraction module does not increase linearly with the number of subbands. The Global Shared Expert can employ a two-layer fully connected network (128→256→128).

[0051] (d2) For independent experts, each independent expert is a neural network module with independent parameters, which is specifically responsible for processing a certain type of frequency band characteristics. The structure of independent experts is usually shallower and narrower than that of shared experts to keep the model lightweight. Independent experts can use a single-layer linear network (128→128).

[0052] The number of independent experts is usually greater than the number of subbands, which allows the fusion features corresponding to each subband to select different expert combinations to achieve overcomplete representation.

[0053] Each independent expert is good at handling specific frequency band response patterns, for example: (1) Low frequency expert: enhances the fundamental harmonics of speech and suppresses low-frequency noise such as air conditioners; (2) Mid frequency expert: preserves the clarity of tone and suppresses mid-frequency vowels of other speakers; (3) High frequency expert: captures details such as fricatives and sibilants and suppresses high-frequency noise in television.

[0054] (d3) Gated networks can adopt a linear layer structure. For example, a gated network is a single-layer fully connected network (without optional biases), whose input is the fusion features corresponding to the sub-bands and whose output is an M-dimensional activation weight.

[0055] The specific implementation process is as follows: The gated network calculates the original score of the fused features through a linear layer (128-dimensional input, M-dimensional output), normalizes it using Softmax, selects the Top K largest weights, and then re-normalizes the retained weights as the activation coefficients of each independent expert. This setting ensures the lightweight nature of the feature extraction model while adaptively selecting the most matching independent experts for fine-grained modeling of different frequency bands, significantly improving the performance of speaker-specific extraction.

[0056] The target speaker features obtained in step four are decoded by a speech decoder to obtain the target speech of the target speaker. The speech decoder can be a waveform decoder. The core operation is usually a 1-D linear transpose convolution, with the kernel size consistent with the encoder (typically 16) and the stride the same as the encoder (e.g., the speech encoder stride is 8, and the speech decoder stride is also 8), thereby transforming the feature sequence back into a time-domain waveform.

[0057] The training process of the feature extraction model in this embodiment is as follows (e1) to (e3): (e1) Construct the training set, specifically: construct a training dataset including registration audio, mixed speech and original speech in the manner described in (c5) above, where the original speech is paired with the mixed speech and used as a label.

[0058] Raw speech inevitably contains a certain proportion of environmental noise and non-target components. If the loss function is directly applied to the target speech output by the feature extraction model and the raw speech, it's equivalent to forcing the feature extraction model to fit these noise and interference components along with the target speaker's features. This can cause some network parameters to be misled by noise, leading to a shift in the optimization direction. This not only reduces the feature extraction model's ability to characterize the target speaker's features but also slows down or even interferes with the overall convergence process, limiting the generalization performance of the final feature extraction model in complex environments.

[0059] To address the aforementioned issues, this embodiment introduces pre-denoising processing of the raw data during the training phase. Specifically, the raw speech is first input into a pre-trained denoising subnetwork based on the ResNet34 structure to suppress background noise and stabilizing interference, resulting in relatively clean, realistic speech.

[0060] (e2) The registered audio and mixed speech are feature extracted separately and then fused and input into the speaker extraction module to obtain the target speech of the target speaker; (e3) Utilizing the target speech of the target speaker With real voice Constructing scale-invariant signal-to-noise ratio loss Update the trainable parameters of the speaker extraction module: .

[0061] By doing so, the interference of the original noise on the loss term can be largely eliminated, so that the gradient update mainly revolves around the effective speech components of the target speaker, thereby accelerating the convergence speed of the feature extraction model and improving the stability of training.

[0062] In summary, this embodiment, on the one hand, introduces a fine-grained frequency band modeling mechanism based on hybrid experts, enabling differentiated learning of target and interference features across different frequency bands, thus enhancing the ability to distinguish complex spectral structures; on the other hand, it performs noise reduction preprocessing on the original speech in the loss design, mitigating the adverse effects of noise components on parameter updates. The synergistic effect of these two approaches allows the method in this embodiment to achieve robust, stable, and high-fidelity extraction of a specific speaker's speech even in complex environments with multiple noise levels, multiple speakers, and strong interference, providing reliable acoustic front-end support for the voice interaction of companion robots in real-world scenarios.

[0063] Based on the above description of the embodiments, those skilled in the art will understand that the voice extraction method, system, medium, and robot in the robot interaction process described in this embodiment can be implemented in pure software or deployed and run on a general-purpose or dedicated computing hardware platform. Based on this essence, the technical solution of this embodiment can be specifically implemented in the form of a software product containing program instructions. This software product can be stored on various non-volatile storage media or directly deployed as a local or cloud service. The program instructions are used to cause computer devices with processing capabilities—including but not limited to personal computers, server clusters, mobile terminals, or other network devices—to execute the steps described in this embodiment.

[0064] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for speech extraction during robot interaction, characterized in that, include: Obtain the registered audio and mixed speech of the target speaker; Frame-level voiceprint features are extracted from the registered audio and pooled into sentence-level voiceprint features. Subband embedding sequences of multiple different subbands are extracted from the mixed speech. The fused feature obtained by fusing the sub-band embedding sequence with the sentence-level voiceprint feature is input into the speaker extraction module; In the speaker extraction module, the fused features of each sub-band are input into the hybrid expert model, which includes a globally shared expert and multiple independent experts. The outputs of each independent expert are adaptively fused through a gating network to obtain the separated target speaker features. The target speaker's target speech is obtained by decoding the target speaker's features. The forward computation of the hybrid expert model is represented as follows: ； in, For the first The output features corresponding to each sub-band For the first The fusion features corresponding to each sub-band For global sharing of experts, Trainable parameters from experts are shared globally and across all subbands. For the first An independent expert, For the first Trainable parameters for an independent expert For gating networks The output of the first The activation weight of an individual expert To activate the top K independent experts, sorted in descending order of weight.

2. The method according to claim 1, characterized in that, The extraction of subband embedding sequences from multiple different subbands of mixed speech specifically involves: The spectrogram of the mixed speech is divided into sub-spectrums of different frequency sub-bands according to the bandwidth of the predefined frequency band; Each sub-spectrum is mapped to a sub-band embedding sequence with a unified feature dimension.

3. The method according to claim 1, characterized in that, In a globally shared expert, all subbands share the same set of parameters.

4. The method according to claim 1, characterized in that, The training process for the speaker extraction module is as follows: The system acquires the registered audio, the mixed speech, and the original speech paired with the mixed speech. It then performs pre-denoising processing on the original speech to obtain the real speech. After feature extraction from the registered audio and mixed speech, the two are fused and input into the speaker extraction module to obtain the target speech of the target speaker. The target speech of the target speaker and the real speech are used to construct a scale-invariant signal-to-noise ratio loss, and the trainable parameters of the speaker extraction module are updated.

5. The method according to claim 4, characterized in that, The pre-denoising process uses a pre-trained ResNet34 denoising subnetwork.

6. The method according to claim 4, characterized in that, The scale-invariant signal-to-noise ratio loss The calculation formula is as follows: ； in, The target speech of the target speaker. This is real voice recording.

7. A computer system comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of computer programs, which are used to be invoked by a processor and to execute the method as described in any one of claims 1-6.

9. A companion robot, characterized in that, The device includes a microphone array, a memory, and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method according to any one of claims 1 to 6.

Images