Speech determination model training method, object speech extraction method, device, electronic equipment and storage medium

Abstract

The present disclosure relates to a speech determination model training method, an object speech extraction method, a device, an electronic device and a storage medium. The method comprises: obtaining a second speech set, the second speech set comprising a plurality of second sample speeches and a second object identifier corresponding to each second sample speech, each second sample speech comprising a fifth speech and a sixth speech, the fifth speech comprising at least speech corresponding to a fifth speech non-associated object identifier, and the sixth speech being silent audio; performing speech extraction on the fifth speech in each second sample speech based on a speech extraction model to obtain a seventh speech corresponding to each second sample speech; and training a candidate speech determination model based on the seventh speech and the sixth speech to obtain a target speech determination model. The present application increases the training samples by adding the speech corresponding to the fifth speech non-associated object identifier and the silence, trains the candidate speech determination model, obtains a model suitable for richer scenarios, and reduces the error rate of audio recognition.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of Internet, and in particular, to a speech determination model training method, an object speech extraction method, device, electronic equipment and storage medium. BACKGROUND

[0002] We can hear noise or other people's voices everywhere in our daily life. With the substantial growth of real-time voice communication needs, if the user's client can remove noise while also removing interfering human voices, the far-end listener in conversation with the user can clearly hear the user's voice, so as to no longer be plagued by noise and interfering human voices.

[0003] In the related art, the user's voice can be separated from the audio to be processed based on the user's voice feature information. Then, when there is no user's voice in the audio to be processed, i.e., the user does not make a sound, the voices of other people in the audio to be processed can be mistakenly considered as the user's voice, thereby causing a high error rate of recognition. SUMMARY

[0004] The present disclosure provides a speech determination model training method, an object speech extraction method, device, electronic equipment and storage medium, and the technical solutions of the present disclosure are as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a speech determination model training method is provided, comprising:

[0006] obtaining a candidate speech determination model; the candidate speech determination model is obtained based on training of a first speech set; the first speech set comprises a plurality of first sample speeches and a first object identifier corresponding to each first sample speech; each first sample speech comprises a first speech, a second speech and a third speech; the first speech at least comprises a speech corresponding to the first object identifier associated with the first speech; the second speech at least comprises a speech corresponding to the first object identifier associated with the second speech and a speech corresponding to an object identifier not associated with the second speech; the third speech is a speech corresponding to the first object identifier associated with the third speech;

[0007] obtaining a second speech set; the second speech set comprises a plurality of second sample speeches and a second object identifier corresponding to each second sample speech; each second sample speech comprises a fourth speech, a fifth speech and a sixth speech; the fourth speech at least comprises a speech corresponding to the second object identifier associated with the fourth speech; the fifth speech at least comprises a speech corresponding to an object identifier not associated with the fifth speech; the sixth speech is a mute audio;

[0008] extracting a voiceprint feature of the fourth speech in each second sample speech based on a voiceprint feature extraction model in the candidate speech determination model, to obtain voiceprint feature information corresponding to each second sample speech;

[0009] extract a fifth voice in each second sample voice based on a voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each second sample voice, to obtain a seventh voice corresponding to each second sample voice; each seventh voice represents a voice corresponding to each second object identifier extracted from the fifth voice in each second sample voice;

[0010] train the candidate voice determination model based on the seventh voice and the sixth voice corresponding to each second sample voice, to obtain a target voice determination model.

[0011] In some possible embodiments, the method further includes a process of training the initial voice determination model based on the first voice set to obtain the candidate voice determination model, including:

[0012] extract voiceprint features of a first voice in each first sample voice based on a voiceprint feature extraction model in the initial voice determination model, to obtain voiceprint feature information corresponding to each first sample voice;

[0013] determine a predicted object identifier corresponding to each first sample voice based on the voiceprint feature information corresponding to each first sample voice;

[0014] extract a second voice in each first sample voice based on a voice extraction model in the initial voice determination model and the voiceprint feature information corresponding to each first sample voice, to obtain an eighth voice corresponding to each first sample voice; each eighth voice represents a voice corresponding to each first object identifier extracted from the second voice in each first sample voice by the voice extraction model;

[0015] train the initial voice determination model based on the first object identifier, the predicted object identifier, the third voice and the eighth voice corresponding to each first sample voice, to obtain the candidate voice determination model.

[0016] In some possible embodiments, the voiceprint features of the first voice in each first sample voice are extracted based on the voiceprint feature extraction model in the initial voice determination model, to obtain the voiceprint feature information corresponding to each first sample voice, including:

[0017] extract frequency domain features of the first voice in each first sample voice, to obtain first voice frequency domain information corresponding to each first sample voice;

[0018] perform log-Mel feature extraction or Mel cepstrum coefficient feature extraction on the first voice frequency domain information corresponding to each first sample voice, to obtain voice feature information corresponding to each first sample voice;

[0019] The voiceprint feature extraction model in the initial voice determination model based on the attention mechanism is used to perform voiceprint feature extraction on voice feature information corresponding to each first sample voice to obtain voiceprint feature information corresponding to each first sample voice.

[0020] The voiceprint feature information represents the timbre feature of the object indicated by the first object identifier.

[0021] In some possible embodiments, the voice extraction model includes a voice extraction sub-model and a mask filtering model.

[0022] The voice extraction model in the initial voice determination model is used to perform voice extraction on the second voice in each first sample voice based on the voiceprint feature information corresponding to each first sample voice to obtain an eighth voice corresponding to each first sample voice, including:

[0023] The frequency domain feature extraction is performed on the second voice in each first sample voice to obtain second voice frequency domain information corresponding to each first sample voice.

[0024] The voice extraction sub-model is used to perform voice extraction on the second voice frequency domain information corresponding to each first sample voice based on the voiceprint feature information corresponding to each first sample voice to obtain a voice mask corresponding to each first sample voice.

[0025] The mask filtering model is used to filter the voice mask corresponding to each first sample voice and the second voice frequency domain information corresponding to each first sample voice to obtain the eighth voice corresponding to each first sample voice.

[0026] In some possible embodiments, the eighth voice corresponding to each first sample voice is voice information in a frequency domain space.

[0027] The initial voice determination model is trained based on the first object identifier, the predicted object identifier, the third voice, and the eighth voice corresponding to each first sample voice to obtain a candidate voice determination model, including:

[0028] The first loss data corresponding to each first sample voice is determined based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice.

[0029] The voiceprint feature extraction model in the initial voice determination model is trained based on the first loss data corresponding to each first sample voice to obtain an updated voiceprint feature extraction model.

[0030] The eighth voice corresponding to each first sample voice is converted into a time domain feature to obtain a ninth voice in a time domain space corresponding to each first sample voice.

[0031] Perform frequency domain feature transformation on the third speech corresponding to each first sample speech to obtain the tenth speech corresponding to the first sample speech in the frequency domain space.

[0032] The second loss data corresponding to each first sample speech is determined based on the third speech corresponding to each first sample speech and the ninth speech corresponding to each first sample speech.

[0033] The third loss data corresponding to each first sample speech is determined based on the eighth speech and the tenth speech corresponding to each first sample speech.

[0034] The speech extraction model in the initial speech determination model is trained based on the second loss data and the third loss data corresponding to each first sample speech to obtain the updated speech extraction model.

[0035] The candidate speech determination model is determined based on the updated voiceprint feature extraction model and the updated speech extraction model.

[0036] In some possible embodiments, the method further includes, before acquiring the second speech set:

[0037] Based on the voiceprint feature extraction model in the candidate speech determination model, the voiceprint feature is extracted from the first speech in each first sample speech to obtain the voiceprint feature information corresponding to each first sample speech in the current loop.

[0038] The prediction object identifier corresponding to each first sample speech in the current loop is determined based on the voiceprint feature information corresponding to each first sample speech in the current loop.

[0039] Based on the speech extraction model in the candidate speech determination model and the voiceprint feature information corresponding to each first sample speech in the current loop, the second speech in each first sample speech is extracted to obtain the eighth speech corresponding to each first sample speech in the current loop.

[0040] The first loss data for each first sample speech in the current loop is determined based on the first object identifier corresponding to each first sample speech and the predicted object identifier corresponding to each first sample speech in the current loop.

[0041] Based on the eighth speech corresponding to each first sample speech in the current loop and the third speech corresponding to each first sample speech in the current loop, determine the second loss data and the third loss data corresponding to each first sample speech in the current loop.

[0042] The voiceprint feature extraction model and the speech extraction model in the candidate speech determination model are trained based on the first loss data corresponding to each first sample voice of the current cycle, the second loss data corresponding to each first sample voice of the current cycle, and the third loss data corresponding to each first sample voice of the current cycle, to obtain an updated candidate speech determination model.

[0043] In some possible embodiments, the first speech includes a first object identifier corresponding speech associated with the first speech and first background noise; and a signal-to-noise ratio of the first speech is greater than a preset signal-to-noise ratio.

[0044] The second speech includes a second object identifier corresponding speech associated with the second speech, a second object identifier corresponding speech not associated with the second speech, and second background noise.

[0045] The fourth speech includes a second object identifier corresponding speech associated with the fourth speech and third background noise; and a signal-to-noise ratio of the fourth speech is greater than a preset signal-to-noise ratio; and the fifth speech includes a fifth object identifier corresponding speech not associated with the fifth speech and fourth background noise.

[0046] In some possible embodiments, the speech extraction sub-model includes an encoder, a gated recurrent unit, and a decoder; the encoder includes a first preset number of convolutional layers, the gated recurrent unit includes a second preset number of gated recurrent networks, and the decoder includes a third preset number of transposed convolutional layers.

[0047] The second speech frequency domain information corresponding to each first sample voice is extracted based on the speech extraction sub-model and the voiceprint feature information corresponding to each first sample voice, to obtain a speech mask corresponding to each first sample voice, including:

[0048] The information extraction is performed on each second speech frequency domain information based on the first preset number of convolutional layers, to obtain a convolutional output information set corresponding to each first sample voice; and a number of convolutional output information in the convolutional output information set is the same as the first preset number.

[0049] The information extraction is performed on the convolutional output information in the convolutional output information set corresponding to each first sample voice based on the second preset number of gated recurrent networks and the voiceprint feature information corresponding to each first sample voice, to obtain a gated output information set corresponding to each first sample voice; and a number of gated output information in the gated output information set is the same as the second preset number.

[0050] The information extraction is performed on the output information set and the gated information set corresponding to each first sample voice based on the third preset number of transposed convolutional layers, to obtain the speech mask corresponding to each first sample voice.

[0051] According to a second aspect of the embodiments of the present disclosure, an object speech extraction method is provided, including:

[0052] obtaining to-be-recognized audio; the to-be-recognized audio includes speech of a target object;

[0053] performing speech extraction on the to-be-recognized audio based on a speech extraction model in the target speech determination model and voiceprint feature information of the target object to obtain speech of the target object; the voiceprint feature information of the target object is obtained by performing voiceprint feature extraction on reference speech of the target object based on a voiceprint feature extraction model in the target speech determination model; and the target speech determination model is obtained by training based on a speech determination model training method.

[0054] According to a third aspect of the embodiments of the present disclosure, a speech determination model training apparatus is provided, comprising:

[0055] The model obtaining module is configured to perform obtaining of a candidate speech determination model; the candidate speech determination model is obtained based on training of a first speech set; the first speech set includes a plurality of first sample speeches and a first object identifier corresponding to each first sample speech; each first sample speech includes a first speech, a second speech, and a third speech; the first speech at least includes speech corresponding to the first object identifier associated with the first speech; the second speech at least includes speech corresponding to the first object identifier associated with the second speech and speech corresponding to an object identifier not associated with the second speech; and the third speech is speech corresponding to the first object identifier associated with the third speech;

[0056] The speech set obtaining module is configured to perform obtaining of a second speech set; the second speech set includes a plurality of second sample speeches and a second object identifier corresponding to each second sample speech; each second sample speech includes a fourth speech, a fifth speech, and a sixth speech; the fourth speech at least includes speech corresponding to the second object identifier associated with the fourth speech; the fifth speech at least includes speech corresponding to an object identifier not associated with the fifth speech; and the sixth speech is silent audio;

[0057] The feature information extracting module is configured to perform, based on a voiceprint feature extraction model in the candidate speech determination model, voiceprint feature extraction on the fourth speech in each second sample speech to obtain voiceprint feature information corresponding to each second sample speech;

[0058] The speech extracting module is configured to perform, based on a speech extraction model in the candidate speech determination model and the voiceprint feature information corresponding to each second sample speech, speech extraction on the fifth speech in each second sample speech to obtain a seventh speech corresponding to each second sample speech; each seventh speech represents speech corresponding to each second object identifier extracted from the fifth speech in each second sample speech;

[0059] The training module is configured to perform training of the candidate speech determination model based on the seventh speech corresponding to each second sample speech and the sixth speech to obtain the target speech determination model.

[0060] In some possible embodiments, the training module is configured to perform:

[0061] performing voiceprint feature extraction on the first speech in each first sample speech based on a voiceprint feature extraction model in the initial speech determination model to obtain voiceprint feature information corresponding to each first sample speech;

[0062] determining a predicted object identifier corresponding to each first sample speech based on the voiceprint feature information corresponding to each first sample speech;

[0063] performing speech extraction on a second speech in each first sample speech based on a speech extraction model in the initial speech determination model and the voiceprint feature information corresponding to each first sample speech to obtain an eighth speech corresponding to each first sample speech; each eighth speech represents a speech corresponding to each first object identifier extracted from the second speech in each first sample speech by the speech extraction model;

[0064] training the initial speech determination model based on the first object identifier, the predicted object identifier, the third speech, and the eighth speech corresponding to each first sample speech to obtain the candidate speech determination model.

[0065] In some possible embodiments, the training module is configured to perform:

[0066] performing frequency domain feature extraction on the first speech in each first sample speech to obtain first speech frequency domain information corresponding to each first sample speech;

[0067] performing logarithmic mel feature extraction or mel cepstrum coefficient feature extraction on the first speech frequency domain information corresponding to each first sample speech to obtain speech feature information corresponding to each first sample speech;

[0068] performing voiceprint feature extraction on the speech feature information corresponding to each first sample speech based on a voiceprint feature extraction model in the initial speech determination model carrying an attention mechanism to obtain voiceprint feature information corresponding to each first sample speech;

[0069] The voiceprint feature information represents a timbre feature of an object indicated by the first object identifier corresponding to the voiceprint feature information.

[0070] In some possible embodiments, the speech extraction model includes a speech extraction sub-model and a mask filtering model.

[0071] The training module is configured to perform:

[0072] perform frequency domain feature extraction on the second speech in each first sample speech to obtain second speech frequency domain information corresponding to each first sample speech;

[0073] perform speech extraction on the second speech frequency domain information corresponding to each first sample speech based on the speech extraction sub-model and the voiceprint feature information corresponding to each first sample speech to obtain a speech mask corresponding to each first sample speech;

[0074] perform filtering on the speech mask corresponding to each first sample speech and the second speech frequency domain information corresponding to each first sample speech based on the mask filtering model to obtain an eighth speech corresponding to each first sample speech.

[0075] In some possible embodiments, the eighth speech corresponding to each first sample speech is speech information in a frequency domain space;

[0076] The training module is configured to perform:

[0077] determine first loss data corresponding to each first sample speech based on the first object identifier corresponding to each first sample speech and the predicted object identifier corresponding to each first sample speech;

[0078] train the voiceprint feature extraction model in the initial speech determination model based on the first loss data corresponding to each first sample speech to obtain an updated voiceprint feature extraction model;

[0079] perform time domain feature conversion on the eighth speech corresponding to each first sample speech to obtain a ninth speech corresponding to each first sample speech in a time domain space;

[0080] perform frequency domain feature conversion on the third speech corresponding to each first sample speech to obtain a tenth speech corresponding to each first sample speech in a frequency domain space;

[0081] determine second loss data corresponding to each first sample speech based on the third speech corresponding to each first sample speech and the ninth speech corresponding to each first sample speech;

[0082] determine third loss data corresponding to each first sample speech based on the eighth speech corresponding to each first sample speech and the tenth speech corresponding to each first sample speech;

[0083] train the speech extraction model in the initial speech determination model based on the second loss data corresponding to each first sample speech and the third loss data corresponding to each first sample speech to obtain an updated speech extraction model;

[0084] determine the candidate speech determination model based on the updated voiceprint feature extraction model and the updated speech extraction model.

[0085] In some possible embodiments, the training module is configured to perform:

[0086] performing voiceprint feature extraction on the first voice in each first sample voice based on a voiceprint feature extraction model in the candidate voice determination model, to obtain voiceprint feature information corresponding to each first sample voice in the current cycle;

[0087] determining a predicted object identifier corresponding to each first sample voice in the current cycle based on the voiceprint feature information corresponding to each first sample voice in the current cycle;

[0088] performing voice extraction on the second voice in each first sample voice based on a voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each first sample voice in the current cycle, to obtain an eighth voice corresponding to each first sample voice in the current cycle;

[0089] determining first loss data corresponding to each first sample voice in the current cycle based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice in the current cycle;

[0090] determining second loss data corresponding to each first sample voice in the current cycle and third loss data corresponding to each first sample voice in the current cycle based on the eighth voice corresponding to each first sample voice in the current cycle and the third voice corresponding to each first sample voice;

[0091] training the voiceprint feature extraction model and the voice extraction model in the candidate voice determination model based on the first loss data corresponding to each first sample voice in the current cycle, the second loss data corresponding to each first sample voice in the current cycle, and the third loss data corresponding to each first sample voice in the current cycle, to obtain an updated candidate voice determination model.

[0092] In some possible embodiments, the first voice includes a voice corresponding to the first object identifier associated with the first voice and first background noise; and a signal-to-noise ratio of the first voice is greater than a preset signal-to-noise ratio;

[0093] The second voice includes a voice corresponding to the first object identifier associated with the second voice, a voice corresponding to an object identifier not associated with the second voice, and second background noise;

[0094] The fourth voice includes a voice corresponding to the second object identifier associated with the fourth voice and third background noise; and a signal-to-noise ratio of the fourth voice is greater than a preset signal-to-noise ratio;

[0095] The fifth voice includes a voice corresponding to an object identifier not associated with the fifth voice and fourth background noise.

[0096] In some possible embodiments, the speech extraction sub-model comprises an encoder, a gated recurrent unit, and a decoder; the encoder comprises a first preset number of convolutional layers, the gated recurrent unit comprises a second preset number of gated recurrent networks, and the decoder comprises a third preset number of transposed convolutional layers.

[0097] a training module configured to perform:

[0098] perform information extraction on each second speech frequency domain information based on the first preset number of convolutional layers, to obtain a set of convolutional output information corresponding to each first sample speech; the number of convolutional output information in the set of convolutional output information is the same as the first preset number;

[0099] perform information extraction on the convolutional output information in the set of convolutional output information corresponding to each first sample speech based on the second preset number of gated recurrent networks and the voiceprint feature information corresponding to each first sample speech, to obtain a set of gated output information corresponding to each first sample speech; the number of gated output information in the set of gated output information is the same as the second preset number;

[0100] perform information extraction on the set of output information and the set of gated information corresponding to each first sample speech based on the third preset number of transposed convolutional layers, to obtain a speech mask corresponding to each first sample speech.

[0101] According to a fourth aspect of the embodiments of the present disclosure, an object speech extraction apparatus is provided, comprising:

[0102] an audio acquisition module configured to perform acquisition of to-be-recognized audio; the to-be-recognized audio comprises speech of a target object;

[0103] a speech extraction module configured to perform speech extraction on the to-be-recognized audio based on a speech extraction model in a target speech determination model and voiceprint feature information of the target object, to obtain speech of the target object; the voiceprint feature information of the target object is obtained by performing voiceprint feature extraction on reference speech of the target object based on a voiceprint feature extraction model in the target speech determination model; and the target speech determination model is obtained by training based on a speech determination model training method.

[0104] According to a fifth aspect of the embodiments of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method of any one of the first aspect or the second aspect.

[0105] According to a sixth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the method of any one of the first aspect or the second aspect of the embodiments of the present disclosure.

[0106] According to a seventh aspect of the embodiments of the present disclosure, a computer program product is provided, which comprises a computer program stored in a readable storage medium, and at least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the computer device executes the method according to any one of the first aspect or the second aspect of the embodiments of the present disclosure.

[0107] The embodiments of the present disclosure have at least the following beneficial effects:

[0108] The method comprises the following steps: obtaining a candidate speech determination model, the candidate speech determination model being obtained based on training of a first speech set, the first speech set comprising a plurality of first sample speeches and a first object identifier corresponding to each first sample speech, each first sample speech comprising a first speech, a second speech, and a third speech, the first speech comprising at least a speech corresponding to a first object identifier associated with the first speech, the second speech comprising at least a speech corresponding to the first object identifier associated with the second speech and a speech corresponding to an object identifier not associated with the second speech, and the third speech being a speech corresponding to the first object identifier associated with the third speech; obtaining a second speech set, the second speech set comprising a plurality of second sample speeches and a second object identifier corresponding to each second sample speech, each second sample speech comprising a fourth speech, a fifth speech, and a sixth speech, the fourth speech comprising at least a speech corresponding to a second object identifier associated with the fourth speech, the fifth speech comprising at least a speech corresponding to an object identifier not associated with the fifth speech, and the sixth speech being a mute audio; performing voiceprint feature extraction on the fourth speech in each second sample speech based on a voiceprint feature extraction model in the candidate speech determination model to obtain voiceprint feature information corresponding to each second sample speech; performing speech extraction on the fifth speech in each second sample speech based on a speech extraction model in the candidate speech determination model and the voiceprint feature information corresponding to each second sample speech to obtain a seventh speech corresponding to each second sample speech, each seventh speech representing a speech corresponding to each second object identifier extracted from the fifth speech in each second sample speech; and training the candidate speech determination model based on the seventh speech and the sixth speech corresponding to each second sample speech to obtain a target speech determination model. The present application increases the training samples by the speech corresponding to the object identifier not associated with the fifth speech and the mute, trains the candidate speech determination model, obtains a model suitable for richer scenarios (including a user mute scenario), and reduces the error rate of audio recognition.

[0109] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0110] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort based on these drawings.

[0111] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0112] Figure 2 is a flowchart of a voice determination model training method according to an exemplary embodiment;

[0113] Figure 3 is a flowchart of an alternative voice determination model training method according to an exemplary embodiment;

[0114] Figure 4 is a schematic diagram of an initial voice determination model according to an exemplary embodiment;

[0115] Figure 5 is a flowchart of a voiceprint feature information determination method according to an exemplary embodiment;

[0116] Figure 6 is a schematic diagram of a voiceprint feature extraction model according to an exemplary embodiment;

[0117] Figure 7 is a schematic diagram of a voice extraction model according to an exemplary embodiment;

[0118] Figure 8 is a flowchart of a determination of an eighth voice according to an exemplary embodiment;

[0119] Figure 9 is a flowchart of a determination of an alternative voice determination model according to an exemplary embodiment;

[0120] Figure 10 is a flowchart of a joint training of a model according to an exemplary embodiment;

[0121] Figure 11 is a flowchart of an object voice extraction method according to an exemplary embodiment;

[0122] Figure 12 is a block diagram of a voice determination model training apparatus according to an exemplary embodiment;

[0123] Figure 13 is a block diagram of an object voice extraction apparatus according to an exemplary embodiment;

[0124] Figure 14 is a block diagram of an electronic device for voice determination model training or audio recognition according to an example embodiment. DETAILED DESCRIPTION

[0125] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0126] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation described in the following exemplary embodiments does not represent all implementations consistent with the present disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0127] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, analyzed data, etc.) involved in the present disclosure are all information and data authorized by the user or authorized by all parties.

[0128] Please refer to Figure 1 , Figure 1 is a schematic diagram of an application environment of a voice determination model training method according to an example embodiment, as Figure 1 shown, the application environment can include a server 01 and a client 02.

[0129] In some possible embodiments, the server 01 can include a stand-alone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud voice determination model training, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. Basic cloud computing services. The operating system running on the server can include but is not limited to Android system, IOS system, linux, windows, Unix, etc.

[0130] In some possible embodiments, the client 02 described above can include, but is not limited to, a smart phone, a desktop computer, a tablet computer, a notebook computer, a smart speaker, a digital assistant, an augmented reality (AR) / virtual reality (VR) device, a smart wearable device, and the like. It can also be a software running on the client, such as an application, a widget, and the like. Optionally, the operating system running on the client can include, but is not limited to, an Android system, an IOS system, Linux, Windows, Unix, and the like.

[0131] In some possible embodiments, the server 01 or the client 02 can obtain an alternative speech determination model, the alternative speech determination model being trained based on a first speech set, the first speech set including a plurality of first sample speeches and a first object identifier corresponding to each first sample speech, each first sample speech including a first speech, a second speech, and a third speech, the first speech including at least a speech corresponding to the first object identifier associated with the first speech, the second speech including at least a speech corresponding to the first object identifier associated with the second speech and a speech corresponding to an object identifier not associated with the second speech, and the third speech being a speech corresponding to the first object identifier associated with the third speech, obtain a second speech set, the second speech set including a plurality of second sample speeches and a second object identifier corresponding to each second sample speech, each second sample speech including a fourth speech, a fifth speech, and a sixth speech, the fourth speech including at least a speech corresponding to the second object identifier associated with the fourth speech, the fifth speech including at least a speech corresponding to an object identifier not associated with the fifth speech, and the sixth speech being a mute audio, perform voiceprint feature extraction on the fourth speech in each second sample speech based on a voiceprint feature extraction model in the alternative speech determination model to obtain voiceprint feature information corresponding to each second sample speech, perform speech extraction on the fifth speech in each second sample speech based on a speech extraction model in the alternative speech determination model and the voiceprint feature information corresponding to each second sample speech to obtain a seventh speech corresponding to each second sample speech, each seventh speech representing a speech corresponding to each second object identifier extracted from the fifth speech in each second sample speech, and train the alternative speech determination model based on the seventh speech and the sixth speech corresponding to each second sample speech to obtain a target speech determination model.

[0132] In some possible embodiments, the client 02 and the server 01 can be connected through a wired link or a wireless link.

[0133] In an example embodiment, the client, the server and the corresponding database of the server can be node devices in a blockchain system, capable of sharing the obtained and generated information to other node devices in the blockchain system, realizing information sharing among multiple node devices. The multiple node devices in the blockchain system can be configured with the same blockchain, which is composed of multiple blocks, and the adjacent blocks have a correlation relationship, so that when the data in any block is tampered with, it can be detected through the next block, thereby avoiding the data in the blockchain being tampered with, and ensuring the security and reliability of the data in the blockchain.

[0134] Figure 2 is a flowchart of a voice determination model training method according to an example embodiment, as shown in Figure 2 The voice determination model training method can be applied to a client, such as a mobile client, or to other node devices, such as a server. The following is an example of a server, which includes the following steps:

[0135] In step S201, an alternative voice determination model is obtained; the alternative voice determination model is obtained based on training of a first voice set; the first voice set includes a plurality of first sample voices and a first object identifier corresponding to each first sample voice; each first sample voice includes a first voice, a second voice and a third voice; the first voice includes at least a voice corresponding to the first object identifier associated with the first voice; the second voice includes at least a voice corresponding to the first object identifier associated with the second voice and a voice corresponding to an object identifier not associated with the second voice; and the third voice is a voice corresponding to the first object identifier associated with the third voice.

[0136] In the embodiments of the present application, the server can obtain a first voice set. The first voice set can include a plurality of first sample voices and a first object identifier corresponding to each first sample voice, each first sample voice including a first voice, a second voice and a third voice, the first voice including at least a voice corresponding to the first object identifier associated with the first voice, the second voice including at least a voice corresponding to the first object identifier associated with the second voice and a voice corresponding to an object identifier not associated with the second voice, and the third voice being a voice corresponding to the first object identifier associated with the third voice.

[0137] Since the embodiments of the present application aim to train a target voice determination model, so that after the target voice determination model is loaded on the client, the client can transmit the voice of a specific object when transmitting voice with other clients, and remove the interfering voice and noise. Based on this, the server can train an initial voice determination model to obtain a target voice determination model. In an alternative embodiment, the server can train the initial voice determination model based on a large number of voice samples.

[0138] Optionally, the server can acquire a first voice set. The first voice set can include a plurality of first sample voices and a first object identifier corresponding to each first sample voice. For example, the plurality of first sample voices can include 100,000 sample voices, each sample voice can have a corresponding first object identifier. Assuming the object is a person, each sample voice can correspond to a person, and 100,000 sample voices can correspond to 100,000 people. Assuming the object is not limited to a person, each sample can correspond to an object, such as a person, a train, an animal, etc., and 100,000 sample voices can correspond to 100,000 objects.

[0139] The following describes the object as a person, each first sample voice can include a first voice, a second voice, and a third voice, and the first object identifier corresponding to each first sample voice can indicate different people. Optionally, the first object identifier is object 1, object 2, object 3, etc. Optionally, the first object identifier is 00001, 00002, 00003, etc.

[0140] In an optional embodiment, the first voice includes a voice corresponding to the first object identifier associated with the first voice, the second voice includes a voice corresponding to the first object identifier associated with the second voice and a voice corresponding to the object identifier not associated with the second voice, and the third voice is a voice corresponding to the first object identifier associated with the third voice. Since the number of first object identifiers corresponding to the first sample voice is one, such as object A. Therefore, the first object identifier associated with the first voice, the first object identifier associated with the second voice, and the first object identifier associated with the third voice in the same first sample voice are all object A. The object identifier not associated with the second voice can be object B or many other objects. As can be seen, the first voice includes a voice corresponding to object A, the second voice includes a voice corresponding to object A and a voice corresponding to object B, and the third voice is a voice corresponding to object A.

[0141] In another optional embodiment, in order to make the speech to be recognized more like the speech acquired in a real environment, and thus make the target speech determination model trained more stable, the environmental noise can be added in the speech to be recognized. Optionally, the first speech includes the first object identification corresponding speech associated with the first speech and the first background noise, and the signal-to-noise ratio of the first speech is greater than a preset signal-to-noise ratio. The second speech includes the first object identification corresponding speech associated with the second speech, the second object identification corresponding speech not associated with the second speech, and the second background noise, and the third speech is the first object identification corresponding speech associated with the third speech. Since the number of the first object identification corresponding to the first sample speech is one, for example, object A. Therefore, the first object identification corresponding speech associated with the first speech, the first object identification corresponding speech associated with the second speech, and the first object identification corresponding speech associated with the third speech in the same first sample speech can all be object A. The second object identification not associated with the second speech can be object B or many other objects. As can be seen, in an optional embodiment, the first speech includes the object A corresponding speech and the first background noise, the second speech includes the object A corresponding speech, the object B corresponding speech, and the second background noise, and the third speech is the object A corresponding speech.

[0142] In the embodiments of the present application, the content of the first object identification corresponding speech included in the first speech, the second speech, and the third speech can all be the first content. Optionally, since the first speech is used to acquire the voiceprint feature information, and the voiceprint feature information generally indicates the timbre characteristics of the object, it does not care what the content of the first object identification corresponding speech is. Therefore, the content of the first object identification corresponding speech included in the first speech can be the first content output by the first object, but the content of the first object identification corresponding speech included in the second speech and the third speech can be the second content output by the first object.

[0143] Optionally, the first background noise can be reverberation, echo, or sound in the environment. The second background noise can be reverberation, echo, or sound in the environment. In the same first sample speech, the first background noise included in the first speech can be the same as the second background noise included in the second speech, or can be different. In this way, by adding the background noise or adding different background noises, the diversity of the first sample speech trained can be increased, so that the target speech determination model trained is more stable and reliable.

[0144] The following introduces an optional embodiment of obtaining one of the first sample voices. First, the server can collect the voice of subject A in a completely quiet environment to obtain a third voice of the first sample voice. Second, the server can synthesize the third voice and the first background noise to obtain the first voice, wherein, in order to make the third voice dominant in the first voice, facilitating the extraction of the subsequent voiceprint feature information, the signal-to-noise ratio of the third voice can be at a relatively high level. Then, the server can collect the voice of subject B, and synthesize the first voice, the voice of subject B and the second background noise to obtain the second voice.

[0145] Alternatively, in another optional embodiment of obtaining one of the first sample voices, first, the server can collect the voice of subject A in a relatively quiet environment to obtain a first voice of the first sample voice, which contains the first background noise. Then, the server can remove the noise from the first voice to obtain a third voice. Next, the server can synthesize the first voice and the voice of subject B to obtain the second voice.

[0146] In the embodiments of the present application, before obtaining the candidate voice determination model, the server can train the initial voice determination model based on the first voice set to obtain the candidate voice determination model. The following introduces how the server trains the initial voice determination model based on the first voice set.

[0147] Figure 3 is a flowchart of a candidate voice determination model training method according to an example embodiment, as shown in Figure 3 , comprising:

[0148] In step S301, the voiceprint feature extraction model in the initial voice determination model is used to extract the voiceprint feature information of the first voice in each first sample voice.

[0149] In the embodiments of the present application, the voiceprint feature information is a sentence-level feature.

[0150] Figure 4 is a schematic diagram of an initial voice determination model according to an example embodiment, as shown in Figure 4 , comprising a voiceprint feature extraction model, a classifier and a voice extraction model.

[0151] In the embodiments of the present application, in combination with Figure 4 , the server can extract the voiceprint feature information of the first voice in each first sample voice based on the voiceprint feature extraction model in the initial voice determination model.

[0152] Figure 5is a flowchart of determining voiceprint feature information according to an exemplary embodiment, as shown, comprising: Figure 5

[0153] In step S501, frequency domain feature extraction is performed on the first voice in each first sample voice to obtain first voice frequency domain information corresponding to each first sample voice.

[0154] In the embodiments of the present application, before using the initial voice determination model, the server can perform frequency domain feature extraction on the first voice in each first sample voice to obtain first voice frequency domain information corresponding to each first sample voice.

[0155] Specifically, the server can perform pre-emphasis processing on the first voice in each first sample voice, that is, pass the first voice in each first sample voice through a high-pass filter. The goal of pre-emphasis is to boost the high frequency part, so that the spectrum of the first voice in each first sample voice becomes flat, maintaining the same signal-to-noise ratio in the entire frequency band from low frequency to high frequency. At the same time, it is also to eliminate the effects of vocal cords and lips during sound production, to compensate for the high frequency part of the voice signal suppressed by the pronunciation system, and to highlight the high frequency resonance peak.

[0156] Next, the server can perform frame processing on the filtered first voice in each first sample voice to obtain the first voice in each first sample voice after frame processing. Specifically, the frame processing includes: first, dividing the filtered first voice in each first sample voice into individual observation units, that is, frames. Each frame can be composed of L (such as 256 or 512) sample points. In order to avoid too large changes between adjacent two frames, there can be an overlapping area between adjacent two frames, so that part of the frame-overlapped filtered first voice in each first sample voice is taken as the first voice in each first sample voice after frame processing.

[0157] The server can perform windowing processing on the first voice in each first sample voice after frame processing to obtain the first voice in each first sample voice after windowing processing. Specifically, the server can multiply each frame of the first voice in each first sample voice after frame processing by a Hamming window to increase the continuity of the left and right ends of the frame. Taking the first voice in each first sample voice as an example, if the first voice with a length of T is represented as x(t) in the time domain, the first voice can be converted to the frequency domain by short-time Fourier transform (STFT) to obtain first frequency domain information in the frequency domain space, and expression (1) is as follows:

[0158] X(n,f) = STFT(x(t)) … Expression (1)

[0159] ​In the expression (1), x(t) represents the first speech after windowing in the time domain space; X(n,f) represents the first frequency domain information in the frequency domain space; n represents a frame sequence, 0 < n < N (N is the total number of frames); f represents a central frequency domain sequence, 0 < f < F, and F is the total number of frequency points.

[0160] Specifically, the server can determine the amplitude value |X(n,f)| of the first frequency domain information X(n,f) as the first speech frequency domain information.

[0161] In step S503, the first speech frequency domain information corresponding to each first sample speech is subjected to logarithmic mel feature extraction or mel cepstrum coefficient feature extraction to obtain speech feature information corresponding to each first sample speech.

[0162] In the embodiments of the present application, in order to make the first speech frequency domain information present more information, the server can perform logarithmic mel feature extraction or mel cepstrum coefficient feature extraction on the first speech frequency domain information corresponding to each first sample speech to obtain speech feature information corresponding to each first sample speech.

[0163] In an optional embodiment, the server can perform logarithmic mel feature extraction on the first speech frequency domain information corresponding to each first sample speech to obtain speech feature information corresponding to each first sample speech.

[0164] Optionally, the process of performing logarithmic mel feature extraction by the server on the first speech frequency domain information corresponding to each first sample speech includes:

[0165] Specifically, the server passes the first speech frequency domain information corresponding to each first sample speech through a Mel scale triangular filter bank. Specifically, the server can define a filter bank with M filters, and the filter used is a triangular filter. The filter bank with M filters can smooth the frequency spectrum and eliminate the effect of harmonics, highlighting the formants of the first speech frequency domain information corresponding to each first sample speech. The server calculates the logarithmic energy at each point of the filter bank with M filters, and determines the logarithmic energy as the first speech frequency domain information feature information corresponding to each first sample speech.

[0166] In another optional embodiment, the server can perform mel cepstrum coefficient feature extraction on the first speech frequency domain information corresponding to each first sample speech to obtain speech feature information corresponding to each first sample speech.

[0167] Specifically, the server passes the first speech frequency domain information corresponding to each first sample speech through a Mel scale triangular filter bank. Specifically, the server can define a filter bank with M filters, and the filter used is a triangular filter. The filter bank with M filters can smooth the spectrum and eliminate the effect of harmonics, highlighting the formants of the first speech frequency domain information corresponding to each first sample speech. The server calculates the logarithmic energy at each point of the filter bank with M filters, and performs discrete cosine transform on the logarithmic energy to obtain L-order mel frequency cepstral coefficients, and takes the mel frequency cepstral coefficients as the first speech frequency domain information feature information corresponding to each first sample speech.

[0168] Discrete cosine transform of the logarithmic energy will remove the correlation of the first speech frequency domain information corresponding to each first sample speech, so that the obtained first speech frequency domain information feature information corresponding to each first sample speech lacks correlation features. Therefore, the server can perform logarithmic mel feature extraction on the first speech frequency domain information corresponding to each first sample speech to obtain first speech frequency domain information feature information corresponding to each first sample speech containing correlation features.

[0169] In this way, the speech feature information of the first speech frequency domain information corresponding to each first sample speech can be determined through the above-mentioned embodiments. The speech feature information of each speech feature information can include time domain information and frequency domain information, and has more feature information than one-dimensional first speech frequency domain information corresponding to each first sample speech.

[0170] Optionally, the server can perform mel frequency cepstral coefficient feature extraction on the first speech frequency domain information corresponding to each first sample speech through expression (2) to obtain speech feature information corresponding to each first sample speech, i.e., frame-based mel frequency cepstral features, where expression (2) is:

[0171] M(n, k) = Mel(|X(n, f)|) … Expression (2)

[0172] In expression (2), |X(n, f)| represents the first speech frequency domain information; M(n, k) represents the speech feature information; n represents the frame sequence, 0 < n < N (N is the total number of frames); k represents the feature dimension.

[0173] In step S505, the voiceprint feature extraction model in the initial speech determination model based on the attention mechanism performs voiceprint feature extraction on the speech feature information corresponding to each first sample speech to obtain voiceprint feature information corresponding to each first sample speech; the voiceprint feature information represents the timbre characteristics of the object represented by the first object identifier corresponding to the voiceprint feature information.

[0174] In an embodiment of the present application, the server can perform voiceprint feature extraction on the speech feature information corresponding to each first sample speech based on a voiceprint feature extraction model carrying an attention mechanism, to obtain voiceprint feature information corresponding to each first sample speech.

[0175] Figure 6 is a schematic diagram of a voiceprint feature extraction model according to an exemplary embodiment, as shown, comprising a multi-layer attention mechanism-based dilated convolution network and an attention-based statistical property pooling layer (attentive statistic pooling). Figure 6

[0176] Optionally, the server can perform feature extraction on the speech feature information corresponding to each first sample speech by using the multi-layer attention mechanism-based dilated convolution network, to obtain frame-level features R(n, l), where l is the number of nodes of the dilated convolution network.

[0177] Optionally, the server can process the frame-level features R(n, l) by using the attention-based statistical property pooling layer, to obtain voiceprint feature information corresponding to each first sample speech. In an optional embodiment, in order to facilitate subsequent processing of the feature information, the voiceprint feature information can be a one-dimensional vector.

[0178] Optionally, the voiceprint feature information can represent the timbre feature, pitch information and amplitude information of the object represented by the first object identifier corresponding to the voiceprint feature information. However, when a person speaks, the pitch and amplitude can change, while the timbre is more difficult to change. Therefore, in order to obtain voiceprint feature information that can better represent information that is difficult to change, the voiceprint feature information can represent the timbre feature of the object represented by the first object identifier corresponding to the voiceprint feature information.

[0179] In step S303, a predicted object identifier corresponding to each first sample speech is determined based on the voiceprint feature information corresponding to each first sample speech.

[0180] In an embodiment of the present application, the server can determine a predicted object identifier corresponding to each first sample speech based on the voiceprint feature information corresponding to each first sample speech by using a classifier.

[0181] In an optional embodiment, the server can use a multi-layer forward network contained in the classifier to convert the voiceprint feature information and predict the probability P(c|e) of the voiceprint feature information belonging to each object in all objects in the training data, where c is an object in the training data.

[0182] ​In step S305, the second speech in each first sample speech is subjected to speech extraction based on the speech extraction model in the initial speech determination model and the voiceprint feature information corresponding to each first sample speech, to obtain an eighth speech corresponding to each first sample speech; each eighth speech represents the speech corresponding to each first object identifier extracted from the second speech in each first sample speech by the speech extraction model.

[0183] In the embodiments of the present application, the server can extract the second speech in each first sample speech based on the speech extraction model in the initial speech determination model and the voiceprint feature information corresponding to each first sample speech, to obtain an eighth speech corresponding to each first sample speech. Wherein each eighth speech represents the speech corresponding to each first object identifier extracted from the second speech in each first sample speech by the speech extraction model.

[0184] Figure 7 is a schematic diagram of a speech extraction model according to an example embodiment, as shown in Figure 7 The speech extraction model includes a speech extraction sub-model and a mask filtering model. The speech extraction sub-model includes an encoder, a gated recurrent unit and a decoder. The encoder includes a first preset number of convolutional layers, such as a first convolutional layer, a second convolutional layer and a third convolutional layer. The gated recurrent unit includes a second preset number of gated recurrent networks, such as a first gated recurrent network, a second gated recurrent network and a third gated recurrent network. The decoder includes a third preset number of transposed convolutional layers, such as a first transposed convolutional layer, a second transposed convolutional layer and a third transposed convolutional layer.

[0185] Figure 8 is a flowchart of determining an eighth speech according to an example embodiment, as shown in Figure 8 includes:

[0186] In step S801, the second speech in each first sample speech is subjected to frequency domain feature extraction to obtain second speech frequency domain information corresponding to each first sample speech.

[0187] In the embodiments of the present application, the server can extract the second speech in each first sample speech based on the speech extraction model in the initial speech determination model and the voiceprint feature information corresponding to each first sample speech, to obtain an eighth speech corresponding to each first sample speech. Wherein each eighth speech represents the speech corresponding to each first object identifier extracted from the second speech in each first sample speech by the speech extraction model.

[0188] Optionally, with reference to the above frequency domain feature extraction of the first speech, for example, the second speech in each first sample speech, the server can convert the second speech with a length of T to the frequency domain through short-time Fourier transform (STFT) to obtain second frequency domain information in the frequency domain. Specifically, the server can determine the amplitude of the second frequency domain information as the second speech frequency domain information |Y *|, the phase value ∠Y of the second frequency domain information is applied to the subsequent inverse Fourier transform as the phase of the predicted voice.

[0189] In step S803, the second speech frequency domain information corresponding to each first sample speech is subjected to speech extraction based on the speech extraction sub-model and the voiceprint feature information corresponding to each first sample speech, to obtain a speech mask corresponding to each first sample speech.

[0190] In an optional embodiment, the server can perform information extraction on each second speech frequency domain information based on a first preset number of convolutional layers to obtain a convolutional output information set corresponding to each first sample speech, and the number of convolutional output information in the convolutional output information set is the same as the first preset number. Subsequently, the server can perform information extraction on the convolutional output information in the convolutional output information set corresponding to each first sample speech based on a second preset number of gated recurrent networks and the voiceprint feature information corresponding to each first sample speech to obtain a gated output information set corresponding to each first sample speech, and the number of gated output information in the gated output information set is the same as the second preset number. The server can perform information extraction on the output information set and the gated information set corresponding to each first sample speech based on a third preset number of transposed convolutional layers to obtain a speech mask corresponding to each first sample speech. In the following, an encoder including a first convolutional layer, a second convolutional layer, and a third convolutional layer, a gated recurrent unit including a first gated recurrent network, a second gated recurrent network, and a third gated recurrent network, and a decoder including a first transposed convolutional layer, a second transposed convolutional layer, and a third transposed convolutional layer are taken as examples for detailed description.

[0191] Specifically, the server can perform information extraction on each second speech frequency domain information based on the first convolutional layer to obtain first output information corresponding to each first sample speech, perform information extraction on each first output information based on the second convolutional layer to obtain second output information corresponding to each first sample speech, and perform information extraction on each second output information based on the third convolutional layer to obtain third output information corresponding to each first sample speech. The first output information, the second output information, and the third output information form a convolutional output information set. Subsequently, the server can perform information extraction on the third output information corresponding to each first sample speech based on the first gated recurrent network and the voiceprint feature information corresponding to each first sample speech to obtain fourth output information corresponding to each first sample speech, perform information extraction on the fourth output information corresponding to each first sample speech based on the second gated recurrent network and the voiceprint feature information corresponding to each first sample speech to obtain fifth output information corresponding to each first sample speech, and perform information extraction on the fifth output information corresponding to each first sample speech based on the third gated recurrent network and the voiceprint feature information corresponding to each first sample speech to obtain sixth output information corresponding to each first sample speech. The fourth output information, the fifth output information, and the sixth output information form a gated output information set. Then, the server can perform information extraction on the third output information and the sixth output information corresponding to each first sample speech based on the first transposed convolutional layer to obtain seventh output information corresponding to each first sample speech, perform information extraction on the second output information and the seventh output information corresponding to each first sample speech based on the second transposed convolutional layer to obtain eighth output information corresponding to each first sample speech, and perform information extraction on the first output information and the eighth output information corresponding to each first sample speech based on the third transposed convolutional layer to obtain a speech mask M corresponding to each first sample speech.

[0192] In step S805, the speech mask corresponding to each first sample speech and the second speech frequency domain information corresponding to each first sample speech are filtered based on the mask filtering model to obtain eighth speech corresponding to each first sample speech.

[0193] In the embodiments of the present application, the server can filter the second speech frequency domain information |Y| corresponding to each first sample speech based on the mask filtering model and the speech mask M corresponding to each first sample speech to obtain eighth speech corresponding to each first sample speech. That is, the speech mask corresponding to the first sample speech of the object A filters the second speech frequency domain information corresponding to the first sample speech of the object A to obtain eighth speech corresponding to the first sample speech of the object A.

[0194] Optionally, the mask filtering module can be a multiplier, and the expression is:

[0195]

[0196] In expression (3), M represents a speech mask corresponding to each first sample speech, |Y| represents second speech frequency domain information corresponding to each first sample speech, represents an eighth speech corresponding to each first sample speech.

[0197] In step S307, the initial speech determination model is trained based on the first object identifier corresponding to each first sample speech, the predicted object identifier, the third speech, and the eighth speech, to obtain an alternative speech determination model.

[0198] In the embodiment of the application, the server can train the initial speech determination model based on the first object identifier corresponding to each first sample speech, the predicted object identifier, the third speech, and the eighth speech, to obtain an alternative speech determination model.

[0199] Optionally, the eighth speech corresponding to each first sample speech is speech information in a frequency domain space. Figure 9 is a flowchart for determining an alternative speech determination model according to an example embodiment, as Figure 9 shown, comprising:

[0200] In step S901, first loss data corresponding to each first sample speech is determined based on the first object identifier corresponding to each first sample speech and the predicted object identifier corresponding to each first sample speech.

[0201] In the embodiment of the application, the server can determine the first loss data corresponding to each first sample speech based on the first object identifier corresponding to each first sample speech and the predicted object identifier corresponding to each first sample speech.

[0202] Specifically, the server can calculate a cross-entropy loss data according to the first object identifier corresponding to each first sample speech and the predicted object identifier corresponding to each first sample speech, and the expression is:

[0203]

[0204] In expression (4), C is the total number of first sample speeches, that is, the total number of first object identifiers, and when the voiceprint feature information e of a certain first speech corresponds to object c, P c equals 1, otherwise equals 0. P(c|e) is the probability of the first speech corresponding to object c.

[0205] In step S902, the voiceprint feature extraction model in the initial speech determination model is trained based on the first loss data corresponding to each first sample speech, to obtain an updated voiceprint feature extraction model.

[0206] In the embodiment of the present application, the server can train the voiceprint feature extraction model in the initial voice determination model based on the first loss data corresponding to each first sample voice to obtain an updated voiceprint feature extraction model.

[0207] Specifically, the server trains the voiceprint feature extraction model in the initial voice determination model based on the first loss data corresponding to each first sample voice, and after updating the model parameters of the voiceprint feature extraction model, the voiceprint feature extraction model after the first training cycle can be obtained.

[0208] Subsequently, the server can input each first voice into the voiceprint feature extraction model after the first training cycle again for training, obtain the first loss data corresponding to the second training cycle under the comparison of each first object identifier, train the voiceprint feature extraction model after the first training cycle using the first loss data corresponding to the second training cycle, and after updating the model parameters of the voiceprint feature extraction model, the voiceprint feature extraction model after the second training cycle can be obtained. Continue the training of the third cycle, the training of the fourth cycle, and so on until the training cutoff condition of the voiceprint feature extraction model is met (such as 200 cycles of training), and then the updated voiceprint feature extraction model is obtained.

[0209] In step S903, the eighth voice corresponding to each first sample voice is converted into a time domain feature to obtain a ninth voice corresponding to each first sample voice in a time domain space.

[0210] Optionally, the eighth voice is determined by the server based on each second voice frequency domain information and the updated voiceprint feature extraction model.

[0211] Since the eighth voice corresponding to each first sample voice is voice information in a frequency domain space, and the third voice is voice information in a time domain space. Therefore, when calculating the loss data using the eighth voice and the third voice, it is necessary to convert the eighth voice and the third voice into the same time-frequency space.

[0212] In the embodiment of the present application, the server can perform invert short time Fourier transform (ISTFT) on the eighth voice corresponding to each first sample voice to obtain a ninth voice corresponding to each first sample voice in a time domain space, and the expression is:

[0213]

[0214] In expression (5), Y represents the eighth voice corresponding to each first sample voice, and ∠Y represents the phase value of the second frequency domain information, represents the ninth voice.

[0215] In step S904, the third voice corresponding to each first sample voice is subjected to frequency domain feature conversion to obtain the tenth voice corresponding to each first sample voice in the frequency domain space.

[0216] Referring to expression (1), in the embodiment of the application, the server can perform short-time Fourier transform on the third voice corresponding to each first sample voice to obtain the tenth voice corresponding to each first sample voice in the frequency domain space.

[0217] In step S905, the second loss data corresponding to each first sample voice is determined based on the third voice corresponding to each first sample voice and the ninth voice corresponding to each first sample voice.

[0218] In the embodiment of the application, the server can determine the second loss data corresponding to each first sample voice based on the third voice corresponding to each first sample voice and the ninth voice corresponding to each first sample voice, and the expression is as follows:

[0219]

[0220] wherein, is the ninth voice, i.e., the voice extracted from the second voice, s is the third voice, i.e., the clean voice of the object, and <,> is the inner product.

[0221] In step S906, the third loss data corresponding to each first sample voice is determined based on the eighth voice corresponding to each first sample voice and the tenth voice corresponding to each first sample voice.

[0222] In the embodiment of the application, the server can determine the third loss data corresponding to each first sample voice based on the eighth voice corresponding to each first sample voice and the tenth voice corresponding to each first sample voice, and the expression is as follows:

[0223]

[0224] In expression (7), is the eighth voice; |s| is the tenth voice, and is an amplitude.

[0225] In step S907, the voice extraction model in the initial voice determination model is trained based on the second loss data corresponding to each first sample voice and the third loss data corresponding to each first sample voice to obtain an updated voice extraction model.

[0226] In the embodiment of the application, the server can train the voice extraction model in the initial voice determination model based on the sum of the second loss data corresponding to each first sample voice and the third loss data corresponding to each first sample voice to obtain an updated voice extraction model.

[0227] Specifically, the server trains the speech extraction model in the initial speech determination model based on the sum of the second loss data corresponding to each first sample speech and the third loss data corresponding to each first sample speech, and after updating the model parameters of the speech extraction model, the speech extraction model after the first training cycle can be obtained.

[0228] Subsequently, the server can input each second speech domain information and each voiceprint feature information into the speech extraction model after the first training cycle again for training, and under the comparison of each third speech, obtain the sum of the second loss data and the third loss data corresponding to the second training cycle, train the speech extraction model after the first training cycle using the sum, and after updating the model parameters of the speech extraction model, the speech extraction model after the second training cycle can be obtained. Continue the training of the third cycle, the training of the fourth cycle, and so on until the training termination condition of the speech extraction model is met (such as 200 cycles of training), and then the updated speech extraction model is obtained.

[0229] In step S908, an alternative speech determination model is determined based on the updated voiceprint feature extraction model and the updated speech extraction model.

[0230] In the embodiments of the present application, the server can determine an alternative speech determination model based on the updated voiceprint feature extraction model and the updated speech extraction model.

[0231] In this way, the server can train the voiceprint feature extraction model through the first speech and the first object identifier, and then can extract the features of the first speech through the trained voiceprint feature extraction model to obtain the voiceprint feature information corresponding to each first speech, that is, the voiceprint feature information of the object to which each first speech belongs.

[0232] Subsequently, the server can train the speech extraction model through the voiceprint feature information corresponding to each first speech and the second speech corresponding to each first speech, and then extract the speech of the object corresponding to each second speech through the trained speech extraction model. In this way, the trained alternative speech determination model can be obtained for subsequent extraction of the speech of the target person from a speech with interfering human voice and noise.

[0233] In the above embodiments, the server can first train the voiceprint feature extraction model, and then train the speech extraction model after the voiceprint feature extraction model is trained, which is not coherent enough. In order to improve the accuracy of the model, the server can jointly train the voiceprint feature extraction model and the speech extraction model trained respectively.

[0234] Figure 10is a flowchart of joint training of a model according to an exemplary embodiment, as Figure 10 as shown, comprising:

[0235] In step S1001, the voiceprint feature extraction model in the alternative voice determination model is used to perform voiceprint feature extraction on the first voice in each first sample voice, to obtain voiceprint feature information corresponding to each first sample voice in the current cycle.

[0236] In the embodiments of the present application, the server can perform voiceprint feature extraction on the first voice in each first sample voice based on the voiceprint feature extraction model in the alternative voice determination model, to obtain voiceprint feature information corresponding to each first sample voice in the current cycle. Specifically, reference can be made to steps S301 and Figure 5 The embodiments related to determining voiceprint feature information are implemented to determine voiceprint feature information corresponding to each first sample voice in the current cycle on the alternative voice determination model, which will not be repeated here.

[0237] In step S1003, the predicted object identifier corresponding to each first sample voice in the current cycle is determined based on the voiceprint feature information corresponding to each first sample voice in the current cycle.

[0238] In the embodiments of the present application, the server can determine the predicted object identifier corresponding to each first sample voice in the current cycle based on the voiceprint feature information corresponding to each first sample voice in the current cycle. Specifically, reference can be made to step S303 and the specific implementation method, which will not be repeated here.

[0239] In step S1005, the second voice in each first sample voice is extracted based on the voice extraction model in the alternative voice determination model and the voiceprint feature information corresponding to each first sample voice in the current cycle, to obtain the eighth voice corresponding to each first sample voice in the current cycle.

[0240] In the embodiments of the present application, the server can extract the second voice in each first sample voice based on the voice extraction model in the alternative voice determination model and the voiceprint feature information corresponding to each first sample voice in the current cycle, to obtain the eighth voice corresponding to each first sample voice in the current cycle. Specifically, reference can be made to step S305 and Figure 8 The embodiments related to determining the eighth voice are implemented to determine the eighth voice corresponding to each first sample voice in the current cycle on the alternative voice determination model, which will not be repeated here.

[0241] In step S1007, the first loss data corresponding to each first sample voice in the current cycle is determined based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice in the current cycle.

[0242] In the embodiments of the present application, the server can determine the first loss data corresponding to each first sample voice in the current cycle based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice in the current cycle. Specifically, reference can be made to step S901 and the embodiments involved, which will not be repeated here.

[0243] In step S1009, the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle are determined based on the eighth voice corresponding to each first sample voice in the current cycle and the third voice corresponding to each first sample voice.

[0244] In the embodiments of the present application, the server can determine the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle based on the eighth voice corresponding to each first sample voice in the current cycle and the third voice corresponding to each first sample voice. Specifically, reference can be made to steps S903-S906 and the embodiments involved, which will not be repeated here.

[0245] In step S1011, the voiceprint feature extraction model and the speech extraction model in the candidate voice determination model are trained based on the first loss data corresponding to each first sample voice in the current cycle, the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle, to obtain an updated candidate voice determination model.

[0246] In the embodiments of the present application, the server can train the voiceprint feature extraction model and the speech extraction model in the candidate voice determination model based on the sum of the first loss data corresponding to each first sample voice in the current cycle, the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle, to obtain the candidate voice determination model after the first round of joint training. Subsequently, the server can train the candidate voice determination model after the first round of joint training to obtain the candidate voice determination model after the second round of joint training, and so on until the joint training stopping condition is reached (such as the number of training times satisfying 200 times), and the training is stopped to obtain the updated candidate voice determination model.

[0247] In this way, the trained candidate speech determination model can be obtained, and the speech of the target object can be extracted from the speech with the interference object and noise in the subsequent process. However, since the second speech in the first sample for training contains the sound of the object to which the first object identifier belongs, assuming that the second speech does not contain the sound of the object to which the first object identifier belongs, but only contains interference human voice and noise, the candidate speech determination model or the updated candidate speech determination model does not consider the corresponding sample of the scene during training. Therefore, it may cause the sound of the interference object, such as the speech of other objects, to be incorrectly identified as the speech of the target person, so that the candidate speech determination model or the updated candidate speech determination model cannot be applied to more scenes. Based on this, the embodiments of the present application can train the candidate speech determination model or the updated candidate speech determination model based on the candidate speech determination model or the updated candidate speech determination model, using the speech containing interference human voice and noise.

[0248] In step S203, a second speech set is obtained; the second speech set includes a plurality of second sample speeches and a second object identifier corresponding to each second sample speech; each second sample speech includes a fourth speech, a fifth speech, and a sixth speech; the fourth speech at least includes a speech corresponding to a second object identifier associated with the fourth speech; the fifth speech at least includes a speech corresponding to an object identifier not associated with the fifth speech; and the sixth speech is a mute audio.

[0249] In the embodiments of the present application, the server can obtain a second speech set. The second speech set includes a plurality of second sample speeches and a second object identifier corresponding to each second sample speech. Each second sample speech includes a fourth speech, a fifth speech, and a sixth speech. The fourth speech at least includes a speech corresponding to a second object identifier associated with the fourth speech. The fifth speech at least includes a speech corresponding to an object identifier not associated with the fifth speech. The sixth speech is a mute audio.

[0250] Optionally, the server can obtain a second speech set. The second speech set can include a plurality of second sample speeches and a second object identifier corresponding to each second sample speech. For example, the plurality of second sample speeches can include 1000 sample speeches, and each sample speech can have a corresponding second object identifier. Assuming that the object is a person, each sample speech can correspond to a person, and then 1000 sample speeches can correspond to 1000 people.

[0251] Each second sample speech can include a fourth speech, a fifth speech, and a sixth speech, and the second object identifier corresponding to each second sample speech can indicate different people. Optionally, the second object identifier is object AAA, object AAB, object AAC, and the like.

[0252] In an optional embodiment, the fourth voice includes a voice corresponding to a second object identifier associated with the fourth voice, the fifth voice includes a voice corresponding to an object identifier not associated with the fifth voice, and the sixth voice is a mute audio. Since the number of the second object identifiers corresponding to the second sample voice is one, such as object AAA. The fourth voice is associated with the second object identifier representing object AAA, and the fifth voice is not associated with the object identifier, which can be object BBA or many other objects. Thus, the fifth voice includes a voice corresponding to object BBA.

[0253] In another optional embodiment, in order to make the voice to be identified more like the voice obtained in a real environment, and thus make the target voice determination model obtained by training more stable, environmental noise can be added to the voice to be identified. The fourth voice can include a voice corresponding to a second object identifier associated with the fourth voice and third background noise, and the signal-to-noise ratio of the fourth voice is greater than a preset signal-to-noise ratio. The fifth voice can include a voice corresponding to an object identifier not associated with the fifth voice and fourth background noise, such as a voice corresponding to object BBA and fourth background noise.

[0254] Optionally, the third background noise and the fourth background noise can be reverberation, echo, or sound in the environment. Optionally, the third background noise and the fourth background noise can be the same noise, or different noise. In this way, the diversity of the second sample voice for training can be increased, and the target voice determination model obtained by training can be more stable and reliable.

[0255] In step S205, a voiceprint feature extraction model in the candidate voice determination model is used to extract voiceprint features from the fourth voice in each second sample voice, to obtain voiceprint feature information corresponding to each second sample voice.

[0256] Optionally, the server can extract voiceprint features from the fourth voice in each second sample voice based on a voiceprint feature extraction model in the candidate voice determination model, to obtain voiceprint feature information corresponding to each second sample voice.

[0257] Subsequently, the server can determine a predicted object identifier corresponding to each second sample voice based on the voiceprint feature information corresponding to each second sample voice. For specific embodiments, reference can be made to step S303, which will not be repeated here.

[0258] In step S207, a voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each second sample voice are used to extract a fifth voice from each second sample voice, to obtain a seventh voice corresponding to each second sample voice; each seventh voice represents a voice corresponding to each second object identifier extracted from the fifth voice in each second sample voice.

[0259] In the embodiments of the present application, the server can perform voice extraction on the fifth voice in each second sample voice based on the voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each second sample voice, to obtain a seventh voice corresponding to each second sample voice; each seventh voice represents the voice corresponding to each second object identifier extracted from the fifth voice in each second sample voice.

[0260] Specifically, the server can refer to steps S305 and Figure 8 The corresponding embodiments determine the seventh voice corresponding to each second sample voice, which will not be repeated here.

[0261] In step S209, the candidate voice determination model is trained based on the seventh voice and the sixth voice corresponding to each second sample voice, to obtain a target voice determination model.

[0262] In the embodiments of the present application, the server can train the candidate voice determination model or the updated candidate voice determination model based on the seventh voice and the sixth voice corresponding to each second sample voice, to obtain a first round trained candidate voice determination model corresponding to the second sample set. The server can train the first round trained candidate voice determination model corresponding to the second sample set based on steps S205-S207, to obtain a second round trained candidate voice determination model corresponding to the second sample set, and so on, until the training is stopped when the training termination condition (such as the training frequency satisfying 100 times) is reached, to obtain the target voice determination model.

[0263] In the above embodiments, the server re-trains the voice extraction model in the candidate voice determination model based on the second voice set, to obtain the target voice determination model.

[0264] In another optional embodiment, the server can train the voiceprint feature extraction model and the voice extraction model in the candidate voice determination model based on the second object identifier, the predicted object identifier, the seventh voice and the sixth voice corresponding to each second sample voice, to obtain the target voice determination model. For specific implementation steps, please refer to steps S307 and Figure 9 The embodiments of determining the candidate voice determination model will not be repeated here.

[0265] In another optional embodiment, the second voice set can include not only a plurality of second sample voices and a second object identifier corresponding to each second sample voice, but also a plurality of first sample voices and a first object identifier corresponding to each first sample voice. Each first sample voice includes a first voice, a second voice, and a third voice; the first voice includes at least a voice corresponding to the first object identifier associated with the first voice; the second voice includes at least a voice corresponding to the first object identifier associated with the second voice and a voice corresponding to an object identifier not associated with the second voice; and the third voice is a voice corresponding to the first object identifier associated with the third voice. In order to align the number of voices in the first sample voice with the number of voices in the second sample voice, the second sample voice can include a fifth voice, a sixth voice, and a fourth voice; the fifth voice includes at least a voice corresponding to an object identifier not associated with the fifth voice; the sixth voice is silence, and the fourth voice includes background noise.

[0266] On the basis of the second voice set, the server can optionally refer to Figure 10 The joint training process of the corresponding model trains the alternative voice determination model or the updated alternative voice determination model to obtain a target voice determination model, and the steps are not repeated here.

[0267] In this way, the server can obtain silence from a voice that includes interfering human voice and noise.

[0268] Figure 11 A flowchart of an object voice extraction method according to an example embodiment is shown in FIG. 1. Figure 11 As shown in FIG. 1, the object voice extraction method can be applied to a server or a client and includes the following steps:

[0269] In step S1101, a to-be-recognized audio is obtained; the to-be-recognized audio includes a voice of a target object.

[0270] In the embodiment of the present application, the client can obtain the to-be-recognized audio, wherein the to-be-recognized audio includes the voice of the target object.

[0271] In step S1103, the to-be-recognized audio is subjected to voice extraction based on a voice extraction model in the target voice determination model and a voiceprint feature information of the target object, to obtain the voice of the target object; the voiceprint feature information of the target object is obtained by voiceprint feature extraction of a reference voice of the target object based on a voiceprint feature extraction model in the target voice determination model; and the target voice determination model is obtained by training based on a voice determination model training method.

[0272] In an optional embodiment, before performing voice extraction on the to-be-identified audio by using the voiceprint feature information of the target object, a reference voice of the target object can be acquired, and voiceprint feature extraction is performed based on the voiceprint feature extraction model in the target voice determination model obtained through training to obtain the voiceprint feature information of the target object.

[0273] In the embodiments of the present application, when the voiceprint feature extraction model is loaded on the client, the client can perform voiceprint feature extraction based on the voiceprint feature extraction model in the target voice determination model obtained through training to obtain the voiceprint feature information of the target object.

[0274] In the embodiments of the present application, when the voiceprint feature extraction model is loaded on the server, the server can receive the reference voice of the target object transmitted by the client, and perform voiceprint feature extraction based on the voiceprint feature extraction model in the target voice determination model obtained through training to obtain the voiceprint feature information of the target object, and then send the voiceprint feature information of the target object to the client.

[0275] In the embodiments of the present application, the voice extraction model in the target voice determination model can be loaded on the client.

[0276] Optionally, if the to-be-identified audio includes the voice of the target object, the client can perform short-time Fourier transform on the to-be-identified audio to obtain the amplitude of the frequency domain information, and perform voice extraction on the amplitude of the frequency domain information based on the voice extraction model in the target voice determination model and the voiceprint feature information of the target object to obtain the voice of the target object in the frequency domain space. Then, the client performs time-frequency conversion on the voice in the frequency domain space by using inverse short-time Fourier to obtain the voice of the target object in the time domain space. Subsequently, the client can transmit the voice of the target object in the time domain space to another client in a call scenario based on the business requirement.

[0277] Optionally, if the to-be-identified audio does not include the voice of the target object, the client can perform short-time Fourier transform on the to-be-identified audio to obtain the amplitude of the frequency domain information, and perform voice extraction on the amplitude of the frequency domain information based on the voice extraction model in the target voice determination model and the voiceprint feature information of the target object to obtain the voice of the target object in the frequency domain space. Then, the client performs time-frequency conversion on the voice in the frequency domain space by using inverse short-time Fourier to obtain the voice of the target object in the time domain space, which is a mute.

[0278] Figure 12 A speech determination model training device block diagram according to an exemplary embodiment is shown. The device has the function of implementing the data processing method in the above method embodiment, which can be realized by hardware, or by hardware executing corresponding software. Refer to Figure 12 The device includes:

[0279] The model obtaining module 1201 is configured to obtain a candidate speech determination model; the candidate speech determination model is obtained based on training of a first speech set; the first speech set includes a plurality of first sample speeches and a first object identifier corresponding to each first sample speech; each first sample speech includes a first speech, a second speech, and a third speech; the first speech at least includes a speech corresponding to a first object identifier associated with the first speech; the second speech at least includes a speech corresponding to a first object identifier associated with the second speech and a speech corresponding to an object identifier not associated with the second speech; and the third speech is a speech corresponding to a first object identifier associated with the third speech.

[0280] The speech set obtaining module 1202 is configured to obtain a second speech set; the second speech set includes a plurality of second sample speeches and a second object identifier corresponding to each second sample speech; each second sample speech includes a fourth speech, a fifth speech, and a sixth speech; the fourth speech at least includes a speech corresponding to a second object identifier associated with the fourth speech; the fifth speech at least includes a speech corresponding to an object identifier not associated with the fifth speech; and the sixth speech is a mute audio;

[0281] The feature information extraction module 1203 is configured to perform voiceprint feature extraction on the fourth speech in each second sample speech based on a voiceprint feature extraction model in the candidate speech determination model, to obtain voiceprint feature information corresponding to each second sample speech;

[0282] The speech extraction module 1204 is configured to perform speech extraction on the fifth speech in each second sample speech based on a speech extraction model in the candidate speech determination model and the voiceprint feature information corresponding to each second sample speech, to obtain a seventh speech corresponding to each second sample speech; each seventh speech represents a speech corresponding to each second object identifier extracted from the fifth speech in each second sample speech;

[0283] The training module 1205 is configured to train the candidate speech determination model based on the seventh speech and the sixth speech corresponding to each second sample speech, to obtain a target speech determination model.

[0284] In some possible embodiments, the training module is configured to:

[0285] perform voiceprint feature extraction on the first speech in each first sample speech based on a voiceprint feature extraction model in the initial speech determination model, to obtain voiceprint feature information corresponding to each first sample speech;

[0286] determine a predicted object identifier corresponding to each first sample speech based on the voiceprint feature information corresponding to each first sample speech;

[0287] extract the second speech in each first sample speech based on the speech extraction model in the initial speech determination model and the voiceprint feature information corresponding to each first sample speech, to obtain an eighth speech corresponding to each first sample speech; each eighth speech represents the speech corresponding to each first object identifier extracted by the speech extraction model from the second speech in each first sample speech;

[0288] train the initial speech determination model based on the first object identifier, the predicted object identifier, the third speech and the eighth speech corresponding to each first sample speech, to obtain a candidate speech determination model.

[0289] In some possible embodiments, the training module is configured to perform:

[0290] extract frequency domain features of the first speech in each first sample speech, to obtain first speech frequency domain information corresponding to each first sample speech;

[0291] perform log-mel feature extraction or mel cepstrum coefficient feature extraction on the first speech frequency domain information corresponding to each first sample speech, to obtain speech feature information corresponding to each first sample speech;

[0292] extract voiceprint features of the speech feature information corresponding to each first sample speech based on a voiceprint feature extraction model in the initial speech determination model carrying the attention mechanism, to obtain voiceprint feature information corresponding to each first sample speech;

[0293] The voiceprint feature information represents the timbre features of the object indicated by the first object identifier corresponding to the voiceprint feature information.

[0294] In some possible embodiments, the speech extraction model includes a speech extraction sub-model and a mask filtering model.

[0295] The training module is configured to perform:

[0296] extract frequency domain features of the second speech in each first sample speech, to obtain second speech frequency domain information corresponding to each first sample speech;

[0297] extract the second speech corresponding to each first sample speech based on the speech extraction sub-model and the voiceprint feature information corresponding to each first sample speech, to obtain a speech mask corresponding to each first sample speech;

[0298] filter the speech mask corresponding to each first sample speech and the second speech frequency domain information corresponding to each first sample speech based on the mask filtering model, to obtain an eighth speech corresponding to each first sample speech.

[0299] In some possible embodiments, the eighth voice corresponding to each first sample voice is voice information in a frequency domain space.

[0300] The training module is configured to perform:

[0301] determine, based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice, first loss data corresponding to each first sample voice;

[0302] train, based on the first loss data corresponding to each first sample voice, the voiceprint feature extraction model in the initial voice determination model to obtain an updated voiceprint feature extraction model;

[0303] perform time domain feature conversion on the eighth voice corresponding to each first sample voice to obtain a ninth voice corresponding to each first sample voice in a time domain space;

[0304] perform frequency domain feature conversion on the third voice corresponding to each first sample voice to obtain a tenth voice corresponding to each first sample voice in a frequency domain space;

[0305] determine, based on the third voice corresponding to each first sample voice and the ninth voice corresponding to each first sample voice, second loss data corresponding to each first sample voice;

[0306] determine, based on the eighth voice corresponding to each first sample voice and the tenth voice corresponding to each first sample voice, third loss data corresponding to each first sample voice;

[0307] train, based on the second loss data corresponding to each first sample voice and the third loss data corresponding to each first sample voice, the voice extraction model in the initial voice determination model to obtain an updated voice extraction model;

[0308] determine the candidate voice determination model based on the updated voiceprint feature extraction model and the updated voice extraction model.

[0309] In some possible embodiments, the training module is configured to perform:

[0310] perform voiceprint feature extraction on the first voice in each first sample voice based on the voiceprint feature extraction model in the candidate voice determination model to obtain voiceprint feature information corresponding to each first sample voice in the current cycle;

[0311] determine, based on the voiceprint feature information corresponding to each first sample voice in the current cycle, a predicted object identifier corresponding to each first sample voice in the current cycle;

[0312] extract a second voice in each first sample voice based on a voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each first sample voice in the current cycle, to obtain an eighth voice corresponding to each first sample voice in the current cycle;

[0313] determine first loss data corresponding to each first sample voice in the current cycle based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice in the current cycle;

[0314] determine second loss data corresponding to each first sample voice in the current cycle and third loss data corresponding to each first sample voice in the current cycle based on the eighth voice corresponding to each first sample voice in the current cycle and the third voice corresponding to each first sample voice;

[0315] train the voiceprint feature extraction model and the voice extraction model in the candidate voice determination model based on the first loss data corresponding to each first sample voice in the current cycle, the second loss data corresponding to each first sample voice in the current cycle, and the third loss data corresponding to each first sample voice in the current cycle, to obtain an updated candidate voice determination model.

[0316] In some possible embodiments, the first voice includes a voice corresponding to the first object identifier associated with the first voice and a first background noise; and a signal-to-noise ratio of the first voice is greater than a preset signal-to-noise ratio;

[0317] The second voice includes a voice corresponding to the first object identifier associated with the second voice, a voice corresponding to an object identifier not associated with the second voice, and a second background noise;

[0318] The fourth voice includes a voice corresponding to the second object identifier associated with the fourth voice and a third background noise; and a signal-to-noise ratio of the fourth voice is greater than a preset signal-to-noise ratio;

[0319] The fifth voice includes a voice corresponding to an object identifier not associated with the fifth voice and a fourth background noise.

[0320] In some possible embodiments, the voice extraction sub-model includes an encoder, a gated recurrent unit, and a decoder; the encoder includes a first preset number of convolutional layers, the gated recurrent unit includes a second preset number of gated recurrent networks, and the decoder includes a third preset number of transposed convolutional layers;

[0321] The training module is configured to perform:

[0322] extract information from each second voice frequency domain information based on the first preset number of convolutional layers, to obtain a set of convolutional output information corresponding to each first sample voice; a number of convolutional output information in the set of convolutional output information is the same as the first preset number;

[0323] information in the set of convolution output information corresponding to each first sample voice based on the second preset number of gated recurrent unit networks and the voiceprint feature information corresponding to each first sample voice, to obtain a set of gated output information corresponding to each first sample voice; the number of gated output information in the set of gated output information is the same as the second preset number;

[0324] information in the set of output information corresponding to each first sample voice and the set of gating information based on the third preset number of transpose convolution layers, to obtain a voice mask corresponding to each first sample voice.

[0325] It should be noted that the apparatus provided in the above embodiments, in realizing its functions, is only exemplified by the above division of each functional module, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0326] Figure 13 is a block diagram of an object voice extraction device according to an example embodiment. The device has the function of realizing the data processing method in the above method embodiments, which can be realized by hardware or corresponding software executed by hardware. Referring to Figure 13 , the device comprises:

[0327] The audio acquisition module 1301 is configured to execute the acquisition of the audio to be recognized; the audio to be recognized includes the voice of the target object;

[0328] The voice extraction module 1302 is configured to execute voice extraction on the audio to be recognized based on the voice extraction model in the target voice determination model and the voiceprint feature information of the target object, to obtain the voice of the target object; the voiceprint feature information of the target object is obtained by voiceprint feature extraction on the reference voice of the target object based on the voiceprint feature extraction model in the target voice determination model; and the target voice determination model is obtained by training based on the voice determination model training method.

[0329] It should be noted that the apparatus provided in the above embodiments, in realizing its functions, is only exemplified by the above division of each functional module, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0330] Figure 14 is a block diagram of an apparatus 3000 for speech determination model training or audio recognition according to an example embodiment. The apparatus 3000 can be a part of a mobile phone, a computer, a digital broadcasting terminal, a message receiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0331] Referring to Figure 14 , the apparatus 3000 can include one or more of the following components: a processing component 3002, a memory 3004, a power supply component 3006, a multimedia component 3008, an audio component 3010, an input / output (I / O) interface 3012, a sensor component 3014, and a speech determination model training component 3016.

[0332] The processing component 3002 usually controls overall operations of the apparatus 3000, such as operations associated with displaying, making a phone call, data speech determination model training, camera operations, and recording operations. The processing component 3002 can include one or more processors 3020 to execute instructions to complete all or part of steps of the above methods. In addition, the processing component 3002 can include one or more modules to facilitate interaction between the processing component 3002 and other components. For example, the processing component 3002 can include a multimedia module to facilitate the interaction between the multimedia component 3008 and the processing component 3002.

[0333] The memory 3004 is configured to store various types of data to support operations of the apparatus 3000. Examples of these data include instructions for any applications or methods operating on the apparatus 3000, contact data, phonebook data, messages, pictures, videos, etc. The memory 3004 can be implemented by any type of volatile or non-volatile storage devices or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0334] The power supply component 3006 supplies electrical power for various components of the apparatus 3000. The power supply component 3006 can include a power supply management system, one or more power sources, and other components associated with generating, managing and distributing power for the apparatus 3000.

[0335] The multimedia component 3008 includes a screen providing an output interface between the device 3000 and a user. In some embodiments, the screen includes a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors for sensing touch, swiping or gesture on the touch panel. The touch sensor can not only sense a boundary of a touching or swiping action, but also detect duration and pressure related to the touching or swiping action. In some embodiments, the multimedia component 3008 includes a front camera and / or a rear camera. When the device 3000 is in an operation mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zooming capability.

[0336] The audio component 3010 is configured to output and / or input audio signals. For example, the audio component 3010 includes a microphone (MIC) that is configured to receive an external audio signal when the device 3000 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 3004 or transmitted via the voice determination model training component 3016. In some embodiments, the audio component 3010 further includes a speaker for outputting audio signals.

[0337] The I / O interface 3012 provides an interface between the processing component 3002 and peripheral interface modules, such as a keyboard, a click wheel, buttons, and so on. The buttons can include, but are not limited to, a home button, a volume button, a start button, and a lock button.

[0338] The sensor component 3014 includes one or more sensors for providing status assessments of various aspects of the device 3000. For example, the sensor component 3014 can detect an open / closed position of the device 3000, relative positioning of components, such as a display and a keypad of the device 3000, a change of position of the device 3000 or a component of the device 3000, presence or absence of user contact with the device 3000, an orientation or acceleration / deceleration / g-force and a temperature change of the device 3000. The sensor component 3014 can include a proximity sensor configured to detect presence of a nearby object without any physical touch. The sensor component 3014 can further include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 3014 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0339] The voice determination model training component 3016 is configured to facilitate voice determination model training between the apparatus 3000 and other devices in a wired or wireless manner. The apparatus 3000 can access a wireless network based on a voice determination model training criterion, such as WiFi, a carrier network (e.g., 2G, 3G, 4G, or 5G), or a combination thereof. In an example embodiment, the voice determination model training component 3016 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an example embodiment, the voice determination model training component 3016 further includes a near field voice determination model training (NFC) module to facilitate short-range voice determination model training. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0340] In an example embodiment, the apparatus 3000 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic components, for performing the above-described methods.

[0341] Embodiments of the present application also provide a computer-readable storage medium, which can be arranged in an electronic device to save at least one instruction or at least one program for implementing a voice determination model training method. The at least one instruction or the at least one program is loaded and executed by the processor to implement the voice determination model training method provided by the above-described method embodiments.

[0342] Embodiments of the present application also provide a computer program product, which includes a computer program stored in a readable storage medium. At least one processor of a computer device reads and executes the computer program from the readable storage medium, so that the computer device executes the method of any one of the first aspect of the embodiments of the present disclosure.

[0343] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve the desired result. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or can be advantageous.

[0344] The various embodiments described in the specification are presented for the purpose of illustration and description. The same or similar parts and / or methods are mutually referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts are referred to the part of the method embodiments.

[0345] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing the relevant hardware to complete, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0346] The above description is only the preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A voice determination model training method, characterized by, The method comprises the following steps: An alternative voice determination model is obtained; the alternative voice determination model is obtained based on training of a first voice set; The first voice set comprises a plurality of first sample voices and a first object identifier corresponding to each first sample voice; each first sample voice comprises a first voice, a second voice, and a third voice; the first voice at least comprises a voice corresponding to a first object identifier associated with the first voice; the second voice at least comprises a voice corresponding to a first object identifier associated with the second voice and a voice corresponding to an object identifier not associated with the second voice; the third voice is a voice corresponding to a first object identifier associated with the third voice; A second voice set is obtained; the second voice set comprises a plurality of second sample voices and a second object identifier corresponding to each second sample voice; each second sample voice comprises a fourth voice, a fifth voice, and a sixth voice; the fourth voice at least comprises a voice corresponding to a second object identifier associated with the fourth voice; the fifth voice at least comprises a voice corresponding to an object identifier not associated with the fifth voice; the sixth voice is a mute audio; Voiceprint feature information corresponding to each second sample voice is obtained by performing voiceprint feature extraction on the fourth voice in each second sample voice based on a voiceprint feature extraction model in the alternative voice determination model; A seventh voice corresponding to each second sample voice is obtained by performing voice extraction on the fifth voice in each second sample voice based on a voice extraction model in the alternative voice determination model and the voiceprint feature information corresponding to each second sample voice; each seventh voice represents a voice corresponding to each second object identifier extracted from the fifth voice in each second sample voice; The alternative voice determination model is trained based on the seventh voice and the sixth voice corresponding to each second sample voice, and a target voice determination model is obtained. 2.The voice determination model training method of claim 1, wherein, The method further comprises a process of training an initial voice determination model based on the first voice set to obtain the alternative voice determination model; the process comprises: Voiceprint feature information corresponding to each first sample voice is obtained by performing voiceprint feature extraction on the first voice in each first sample voice based on a voiceprint feature extraction model in the initial voice determination model; A predicted object identifier corresponding to each first sample voice is determined based on the voiceprint feature information corresponding to each first sample voice; An eighth voice corresponding to each first sample voice is obtained by performing voice extraction on the second voice in each first sample voice based on a voice extraction model in the initial voice determination model and the voiceprint feature information corresponding to each first sample voice; each eighth voice represents a voice corresponding to each first object identifier extracted from the second voice in each first sample voice by the voice extraction model; The initial voice determination model is trained based on the first object identifier, the predicted object identifier, the third voice, and the eighth voice corresponding to each first sample voice, and an alternative voice determination model is obtained. 3.The voice determination model training method of claim 2, wherein, The voiceprint feature extraction model in the initial voice determination model is used to perform voiceprint feature extraction on the first voice in each first sample voice, to obtain voiceprint feature information corresponding to each first sample voice, including: Frequency domain feature extraction is performed on the first voice in each first sample voice, to obtain first voice frequency domain information corresponding to each first sample voice; Log-Mel feature extraction or Mel cepstrum coefficient feature extraction is performed on the first voice frequency domain information corresponding to each first sample voice, to obtain voice feature information corresponding to each first sample voice; The voiceprint feature extraction model in the initial voice determination model is used to perform voiceprint feature extraction on the voice feature information corresponding to each first sample voice, to obtain voiceprint feature information corresponding to each first sample voice; The voiceprint feature information represents the timbre feature of an object represented by the first object identifier. 4.The voice determination model training method of claim 2, wherein, The voice extraction model includes a voice extraction sub-model and a mask filtering model; The voice extraction model in the initial voice determination model and the voiceprint feature information corresponding to each first sample voice are used to perform voice extraction on the second voice in each first sample voice, to obtain an eighth voice corresponding to each first sample voice, including: Frequency domain feature extraction is performed on the second voice in each first sample voice, to obtain second voice frequency domain information corresponding to each first sample voice; The voice extraction sub-model and the voiceprint feature information corresponding to each first sample voice are used to perform voice extraction on the second voice frequency domain information corresponding to each first sample voice, to obtain a voice mask corresponding to each first sample voice; The mask filtering model is used to filter the voice mask corresponding to each first sample voice and the second voice frequency domain information corresponding to each first sample voice, to obtain an eighth voice corresponding to each first sample voice. 5.The voice determination model training method of claim 2, wherein, The eighth voice corresponding to each first sample voice is voice information in a frequency domain space; The initial voice determination model is trained based on the first object identifier, the predicted object identifier, the third voice, and the eighth voice corresponding to each first sample voice, to obtain a candidate voice determination model, including: First loss data corresponding to each first sample voice is determined based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice; The voiceprint feature extraction model in the initial voice determination model is trained based on the first loss data corresponding to each first sample voice, to obtain an updated voiceprint feature extraction model; The eighth voice corresponding to each first sample voice is converted into a time domain feature, to obtain a ninth voice corresponding to the first sample voice in a time domain space; The third voice corresponding to each first sample voice is converted into a frequency domain feature, to obtain a tenth voice corresponding to the first sample voice in a frequency domain space; determine the second loss data corresponding to each first sample voice based on the third voice corresponding to each first sample voice and the ninth voice corresponding to each first sample voice; determine the third loss data corresponding to each first sample voice based on the eighth voice corresponding to each first sample voice and the tenth voice corresponding to each first sample voice; train the voice extraction model in the initial voice determination model based on the second loss data corresponding to each first sample voice and the third loss data corresponding to each first sample voice, to obtain an updated voice extraction model; determine the candidate voice determination model based on the updated voice feature extraction model and the updated voice extraction model.

6. The speech determination model training method of any one of claims 2-5, wherein, Before the second voice set is obtained, the method further comprises: extract voice feature information corresponding to each first sample voice in the current cycle based on the voice feature extraction model in the candidate voice determination model; determine the predicted object identifier corresponding to each first sample voice in the current cycle based on the voice feature information corresponding to each first sample voice in the current cycle; extract the second voice in each first sample voice based on the voice extraction model in the candidate voice determination model and the voice feature information corresponding to each first sample voice in the current cycle, to obtain the eighth voice corresponding to each first sample voice in the current cycle; determine the first loss data corresponding to each first sample voice in the current cycle based on the first object identifier corresponding to each first sample voice and the predicted object identifier corresponding to each first sample voice in the current cycle; determine the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle based on the eighth voice corresponding to each first sample voice in the current cycle and the third voice corresponding to each first sample voice; train the voice feature extraction model and the voice extraction model in the candidate voice determination model based on the first loss data corresponding to each first sample voice in the current cycle, the second loss data corresponding to each first sample voice in the current cycle and the third loss data corresponding to each first sample voice in the current cycle, to obtain an updated candidate voice determination model.

7. The voice determination model training method according to claim 1, wherein the first voice comprises a voice corresponding to the first object identifier associated with the first voice and a first background noise, and a signal-to-noise ratio of the first voice is greater than a preset signal-to-noise ratio; the second voice comprises a voice corresponding to the first object identifier associated with the second voice, a voice corresponding to an object identifier not associated with the second voice and a second background noise; the fourth voice comprises a voice corresponding to the second object identifier associated with the fourth voice and a third background noise, and a signal-to-noise ratio of the fourth voice is greater than the preset signal-to-noise ratio; ​ The fifth voice includes a voice corresponding to an object identifier not associated with the fifth voice and a fourth background noise. 8.The voice determination model training method of claim 4, wherein, The voice extraction sub-model includes an encoder, a gated recurrent unit, and a decoder; the encoder includes a first preset number of convolutional layers, the gated recurrent unit includes a second preset number of gated recurrent networks, and the decoder includes a third preset number of transposed convolutional layers; The voice extraction sub-model and the voiceprint feature information corresponding to each first sample voice are used to perform voice extraction on the second voice frequency domain information corresponding to each first sample voice to obtain a voice mask corresponding to each first sample voice, including: The first preset number of convolutional layers are used to perform information extraction on the second voice frequency domain information to obtain a convolutional output information set corresponding to each first sample voice; the number of convolutional output information in the convolutional output information set is the same as the first preset number; The second preset number of gated recurrent networks and the voiceprint feature information corresponding to each first sample voice are used to perform information extraction on the convolutional output information in the convolutional output information set corresponding to each first sample voice to obtain a gated output information set corresponding to each first sample voice; the number of gated output information in the gated output information set is the same as the second preset number; The third preset number of transposed convolutional layers are used to perform information extraction on the output information set and the gated information set corresponding to each first sample voice to obtain a voice mask corresponding to each first sample voice.

9. A method of object speech extraction, characterized by, It includes: Obtaining an audio to be identified; the audio to be identified includes a voice of a target object; A voice extraction model in a target voice determination model and voiceprint feature information of the target object are used to perform voice extraction on the audio to be identified to obtain a voice of the target object; the voiceprint feature information of the target object is obtained by performing voiceprint feature extraction on a reference voice of the target object based on a voiceprint feature extraction model in the target voice determination model; The target voice determination model is trained based on the voice determination model training method of any one of claims 1 to 8. 10.A voice determination model training apparatus, comprising: It includes: A model acquisition module configured to execute an acquisition of a candidate voice determination model; The candidate voice determination model is obtained based on a first voice set; The first voice set includes a plurality of first sample voices and a first object identifier corresponding to each first sample voice; each first sample voice includes a first voice, a second voice, and a third voice; the first voice includes at least a voice corresponding to a first object identifier associated with the first voice; the second voice includes at least a voice corresponding to a first object identifier associated with the second voice and a voice corresponding to an object identifier not associated with the second voice; and the third voice is a voice corresponding to a first object identifier associated with the third voice; The voice set obtaining module is configured to perform obtaining a second voice set; the second voice set comprises a plurality of second sample voices and a second object identifier corresponding to each second sample voice; each second sample voice comprises a fourth voice, a fifth voice, and a sixth voice; the fourth voice at least comprises a voice corresponding to a second object identifier associated with the fourth voice; the fifth voice at least comprises a voice corresponding to an object identifier not associated with the fifth voice; and the sixth voice is a mute audio; The feature information extraction module is configured to perform voiceprint feature extraction on the fourth voice in each second sample voice based on a voiceprint feature extraction model in the candidate voice determination model, to obtain voiceprint feature information corresponding to each second sample voice; The voice extraction module is configured to perform voice extraction on the fifth voice in each second sample voice based on a voice extraction model in the candidate voice determination model and the voiceprint feature information corresponding to each second sample voice, to obtain a seventh voice corresponding to each second sample voice; each seventh voice represents a voice corresponding to each second object identifier extracted from the fifth voice in each second sample voice; The training module is configured to perform training on the candidate voice determination model based on the seventh voice and the sixth voice corresponding to each second sample voice, to obtain a target voice determination model.

11. An object voice extraction apparatus characterized by comprising: The method comprises: The audio obtaining module is configured to perform obtaining to-be-recognized audio; the to-be-recognized audio comprises a voice of a target object; The voice extraction module is configured to perform voice extraction on the to-be-recognized audio based on a voice extraction model in the target voice determination model and voiceprint feature information of the target object, to obtain the voice of the target object; the voiceprint feature information of the target object is obtained by performing voiceprint feature extraction on reference voice of the target object based on a voiceprint feature extraction model in the target voice determination model; The target voice determination model is obtained by training based on the voice determination model training method in any one of claims 1 to 8.

12. An electronic device, comprising: The method comprises: A processor; A memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the voice determination model training method in any one of claims 1 to 8 or the object voice extraction method in claim 9.

13. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device can perform the voice determination model training method in any one of claims 1 to 8 or the object voice extraction method in claim 9.

Images