Training method, recognition method and device for speech object recognition model

Abstract

The present disclosure relates to a training method, a recognition method and a device for a speech object recognition model. The training method comprises: obtaining a first speech signal sample, a second speech signal sample and an overlapping speech signal sample of each speech object among a plurality of speech objects; inputting the first speech signal sample and the overlapping speech signal sample of each speech object into a speech extractor to obtain an extracted third speech signal sample; inputting the third speech signal sample corresponding to each speech object into a speech representation extractor to obtain a speech representation corresponding to each speech object; inputting the speech representation corresponding to each speech object into an identity discriminator to obtain an identity prediction probability; calculating a loss according to the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object; and adjusting the parameters of the speech extractor, the speech representation extractor and the identity discriminator by the loss, thereby training the speech object recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and more specifically, to a training method, a recognition method and a device for a speech object recognition model. Background Art

[0002] With the development of science and technology, speech-related technologies have been widely used in various fields. Speech-related technologies may include speech recognition, speech synthesis, voiceprint recognition, etc.

[0003] However, the sound pickup environment in actual scenes is often very complex and may contain a lot of noise, such as natural noise, interfering human voices, etc. When performing voice object identification, the above noise will interfere with the process of voice object identification, resulting in a decrease in the accuracy of voice object identification. For example, during an online meeting, if multiple participants have a debate, the voice recognition device of the online meeting will find it difficult to identify the identity of the current main speaker; or during a family gathering, if multiple family members are talking, it will be difficult for smart speakers or smart TVs to determine which family member is currently using the device through voice recognition.

[0004] Therefore, in the related art, when multiple voice objects are outputting voices, the solution of identifying the identity of the current voice object through voice recognition has the problem of low accuracy. Summary of the invention

[0005] The present disclosure provides a training method, a recognition method and a device for a speech object recognition model to at least solve the problem in the above-mentioned related art that when multiple speech objects output speech, the scheme of identifying the identity of the current speech object through speech recognition has low accuracy.

[0006] According to a first aspect of an embodiment of the present disclosure, a training method for a speech object recognition model is provided, wherein the speech object recognition model includes a speech extractor, a speech representation extractor, and an identity discriminator, and the training method includes: obtaining a first speech signal sample, a second speech signal sample, and an overlapping speech signal sample of each speech object in a plurality of speech objects, wherein the first speech signal sample and the second speech signal sample are clean speech signal samples with different contents of the speaking object, the overlapping speech signal sample is a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, the clean speech signal sample of at least one speech object includes at least the second speech signal sample of the speech object, and the first speech signal sample corresponds to an identification label ; Input the first speech signal sample and the overlapping speech signal samples of each speech object into the speech extractor to obtain the third speech signal sample extracted from the overlapping speech signal samples of each speech object; input the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object; input the speech representation corresponding to each speech object into the identity discriminator to obtain the identity prediction probability corresponding to each speech object; calculate the loss according to the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object; adjust the parameters of the speech extractor, the speech representation extractor and the identity discriminator according to the loss, so as to train the speech object recognition model.

[0007] Optionally, the loss is calculated based on the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object, including: calculating the first loss based on the third speech signal sample and the second speech signal sample corresponding to each speech object; calculating the second loss based on the identity prediction probability and the identification label corresponding to each speech object; and performing weighted summation of the first loss and the second loss to obtain the loss.

[0008] Optionally, the first loss is calculated by a scale-invariant signal-to-distortion ratio loss function.

[0009] Optionally, the second loss is calculated by a cross entropy loss function.

[0010] Optionally, the step of inputting the first speech signal sample and the overlapping speech signal sample of each speech object into the speech extractor to obtain the third speech signal sample extracted from the overlapping speech signal samples of each speech object comprises: performing Fourier transform on the first speech signal sample and the overlapping speech signal sample of each speech object respectively to obtain a first Fourier transform result corresponding to the first speech signal sample and a second Fourier transform result corresponding to the overlapping speech signal sample; inputting the first Fourier transform result and the second Fourier transform result of each speech object into the speech extractor to obtain a speech extraction result for each speech object; and performing inverse Fourier transform on the speech extraction result of each speech object to obtain a third speech signal sample extracted from the overlapping speech signal samples of each speech object.

[0011] Optionally, the step of inputting the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object includes: performing Fourier transform on the third speech signal sample corresponding to each speech object to obtain a third Fourier transform result; obtaining the Mel-cepstral feature corresponding to the third Fourier transform result of each speech object; inputting the Mel-cepstral feature corresponding to each speech object into the speech representation extractor to obtain a frame-level feature; and performing statistical feature pooling on the frame-level feature corresponding to each speech object to obtain a speech representation corresponding to each speech object.

[0012] According to a second aspect of an embodiment of the present disclosure, a method for recognizing a speech object is provided, the method comprising: obtaining a speech signal under test; performing the following operations for each candidate speech signal among a plurality of candidate speech signals: inputting the candidate speech signal into a speech extractor and a speech representation extractor trained according to the training method of the present disclosure, to obtain a first speech representation corresponding to each candidate speech signal, wherein the plurality of candidate speech signals are speech signals pre-registered for a plurality of candidate objects; inputting the candidate speech signal and the speech signal under test into the speech extractor and the speech representation extractor, to obtain a second speech representation corresponding to each candidate speech signal; obtaining the speech representation similarity between the first speech representation and the second speech representation corresponding to each candidate speech signal; and determining the identity of the speech object under test corresponding to the speech signal under test according to the speech representation similarity corresponding to each candidate speech signal.

[0013] Optionally, determining the identity of the tested speech object corresponding to the tested speech signal based on the speech representation similarity corresponding to each candidate speech signal includes: determining the maximum speech representation similarity among multiple speech representation similarities corresponding to the multiple candidate speech signals; judging whether the maximum speech representation similarity is greater than a preset threshold to obtain a judgment result; and determining the identity of the tested speech object based on the judgment result.

[0014] Optionally, determining the identity of the voice object under test based on the judgment result includes: when the judgment result indicates that the maximum voice representation similarity is greater than the preset threshold, determining that the voice object under test is a candidate object corresponding to the maximum voice representation similarity.

[0015] Optionally, determining the identity of the voice object under test based on the judgment result includes: determining that the identity of the voice object under test cannot be identified when the judgment result indicates that the maximum voice representation similarity is less than or equal to the preset threshold.

[0016] According to a third aspect of an embodiment of the present disclosure, a training device for a speech object recognition model is provided, wherein the speech object recognition model comprises a speech extractor, a speech representation extractor and an identity discriminator, and the training device comprises: a speech signal sample acquisition module, configured to acquire a first speech signal sample, a second speech signal sample and an overlapping speech signal sample of each speech object in a plurality of speech objects, wherein the first speech signal sample and the second speech signal sample are clean speech signal samples with different contents of the speaking object, the overlapping speech signal sample is a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, the clean speech signal sample of at least one speech object comprises at least the second speech signal sample of the speech object, and the first speech signal sample corresponds to an identification label; a speech signal sample extraction module, configured to acquire a first speech signal sample, a second speech signal sample and an overlapping speech signal sample of each speech object in a plurality of speech objects, wherein the first speech signal sample and the second speech signal sample are clean speech signal samples with different contents of the speaking object, the overlapping speech signal sample is a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, the clean speech signal sample of at least one speech object comprises at least the second speech signal sample of the speech object, and the first speech signal sample corresponds to an identification label; The first speech signal sample and the overlapping speech signal sample are input into the speech extractor to obtain the third speech signal sample extracted from the overlapping speech signal samples of each speech object; the speech representation extraction module is configured to input the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object; the identity prediction probability acquisition module is configured to input the speech representation corresponding to each speech object into the identity discriminator to obtain the identity prediction probability corresponding to each speech object; the calculation module is configured to calculate the loss according to the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object; the adjustment module is configured to adjust the parameters of the speech extractor, the speech representation extractor and the identity discriminator according to the loss, so as to train the speech object recognition model.

[0017] According to a fourth aspect of an embodiment of the present disclosure, a device for recognizing a speech object is provided, the device comprising: a tested speech signal acquisition module, configured to acquire a tested speech signal; performing the following operations for each candidate speech signal among a plurality of candidate speech signals: a first speech representation acquisition module, configured to input the candidate speech signal into a speech extractor and a speech representation extractor trained according to the training method of the present disclosure, to obtain a first speech representation corresponding to each candidate speech signal, wherein the plurality of candidate speech signals are speech signals pre-registered for a plurality of candidate objects; a second speech representation acquisition module, configured to input the candidate speech signal and the tested speech signal into the speech extractor and the speech representation extractor, to obtain a second speech representation corresponding to each candidate speech signal; a speech representation similarity acquisition module, configured to acquire the speech representation similarity between the first speech representation and the second speech representation corresponding to each candidate speech signal; and a determination module, configured to determine the identity of the tested speech object corresponding to the tested speech signal according to the speech representation similarity corresponding to each candidate speech signal.

[0018] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a training method for a speech object recognition model and a method for recognizing speech objects according to the present disclosure.

[0019] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of a speech object recognition model and the recognition method of a speech object according to the present disclosure.

[0020] According to a seventh aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements a training method for a speech object recognition model and a method for recognizing a speech object according to the present disclosure.

[0021] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0022] The training method of the speech object recognition model disclosed in the present invention uses a large number of pure speech signal samples and overlapping speech signal samples of speech objects to train the speech object recognition model. Therefore, the trained speech object recognition model can accurately extract the speech representation of the specified speech object from the speech signals of multiple speech objects, and when multiple speech objects all output speech, the accuracy of identifying the identity of the current speech object can be guaranteed; and the recognition method of the speech object disclosed in the present invention can extract the second speech representation corresponding to the potential candidate speech signal from the tested speech signal by using the trained speech extractor and speech representation extractor with the help of the candidate speech signal, and then determine the identity of the tested speech object according to the speech representation similarity between the first speech representation corresponding to the candidate speech signal and the second speech representation. Since the recognition method of the speech object disclosed in the present invention can extract the speech representation of the specified speech object from the speech signal of a single speech object or the speech signals of multiple speech objects, the identity of the current speech object can also be accurately identified when multiple speech objects all output speech.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0025] Figure 1 is a flow chart illustrating a method for training a speech object recognition model according to an exemplary embodiment of the present disclosure;

[0026] Figure 2 is a schematic diagram showing a training phase and a testing phase of a speech object recognition model according to an exemplary embodiment of the present disclosure;

[0027] Figure 3 is a flowchart illustrating a method for recognizing a voice object according to an exemplary embodiment of the present disclosure;

[0028] Figure 4 is a block diagram illustrating a training apparatus for a speech object recognition model according to an exemplary embodiment of the present disclosure;

[0029] Figure 5 is a block diagram illustrating a device for recognizing a voice object according to an exemplary embodiment of the present disclosure;

[0030] Figure 6 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0033] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0034] At present, speech-related technologies have been widely used in various fields, including speech recognition, speech synthesis, voiceprint recognition, etc. However, the sound pickup environment in actual scenes is often very complex and may contain a lot of noise, such as natural noise, interfering human voices, etc. When performing speech object identification, the above noises will interfere with the process of speech object identification, resulting in a decrease in the accuracy of speech object identification.

[0035] In order to solve the above problems, in the related art, when training the voiceprint recognition system, a data augmentation training method is often used, for example, adding noise, reverberation, etc. Alternatively, a voice enhancement module is added to the front end of the voiceprint recognition system to filter out unfavorable factors such as noise and reverberation in the test voice. However, the voiceprint recognition system in the related art is only effective in the audio environment of non-human voices such as noise or reverberation. If there is an interfering human voice in the audio environment, for example, when there are multiple people speaking at the same time, the voiceprint recognition system still has difficulty in correctly identifying the identity of the current speaker.

[0036] In order to solve the problem in the related art that when multiple voice objects all output voice, the accuracy of identifying the identity of the current voice object through voice recognition is low, the training method of the voice object recognition model disclosed in the present invention uses a large number of pure voice signal samples and overlapping voice signal samples of voice objects to train the voice object recognition model. Therefore, the trained voice object recognition model can accurately extract the voice representation of the specified voice object from the voice signals of multiple voice objects, and when multiple voice objects all output voice, the accuracy of identifying the identity of the current voice object can be guaranteed; and the recognition method of the voice object disclosed in the present invention can extract the second voice representation corresponding to the potential candidate voice signal from the tested voice signal by using the trained voice extractor and voice representation extractor with the help of the candidate voice signal, and then determine the identity of the tested voice object according to the voice representation similarity between the first voice representation corresponding to the candidate voice signal and the second voice representation. Since the recognition method of the voice object disclosed in the present invention can extract the voice representation of the specified voice object from the voice signal of a single voice object or the voice signals of multiple voice objects, the identity of the current voice object can also be accurately identified when multiple voice objects all output voice.

[0037] Figure 1 is a flowchart illustrating a method for training a speech object recognition model according to an exemplary embodiment of the present disclosure. The speech object recognition model may include a speech extractor, a speech representation extractor, and an identity discriminator.

[0038] Reference Figure 1 In step 101, the first speech signal sample x(t), the second speech signal sample s(t) and the overlapping speech signal sample y(t) of each speech object in the plurality of speech objects may be obtained. The first speech signal sample x(t) and the second speech signal sample s(t) are clean speech signal samples with different contents of the speech object, that is, the first speech signal sample x(t) and the second speech signal sample s(t) are speech signal samples with higher signal-to-noise ratio (SNR) with different contents of the speech object. The overlapping speech signal sample y(t) may be a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, and the clean speech signal samples of at least one speech object include at least the second speech signal sample s(t) of the speech object. The first speech signal sample x(t) corresponds to an identification label label. At this time, each speech object in the plurality of speech objects may correspond to a data pair (x(t), y(t), s(t), label).

[0039] Furthermore, the reason why the first speech signal sample x(t) and the second speech signal sample s(t) are required to be speech signal samples with different contents of the same speech object is that when the trained speech object recognition model is used to identify the identity of the speech object in the subsequent process, the candidate speech signal pre-registered for each speech object in the candidate speech signal library and the tested speech signal need to be formed into a test pair, and the identity of the tested speech object corresponding to the tested speech signal is determined based on the multiple test pairs formed. In an actual test scenario, the candidate speech signal of the candidate object contained in each test pair is different from the content of the potential speech signal of the candidate object in the tested speech signal contained in the test pair. For example, all employees of a company can pre-register their own candidate speech signals, and during the online meeting of the company's employees, the content of the speech of each participating employee during the meeting must be different from the content of the candidate speech signal pre-registered by the employee. Therefore, in the training stage, in order to simulate the real test scenario as much as possible, the first speech signal sample x(t) and the second speech signal sample s(t) used for training can be set as speech signal samples with different contents, so that the accuracy of the speech object recognition model in identifying the identity of the tested speech object can be guaranteed as much as possible.

[0040] It should be noted that in the training phase of the speech object recognition model, the speech data of tens of thousands of speech objects are used to train a speech extractor, a speech representation extractor and an identity discriminator. The speech extractor, speech representation extractor and identity discriminator can all be built based on neural networks.

[0041] In step 102, the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object can be input into the speech extractor to obtain the third speech signal sample s′(t) extracted from the overlapping speech signal sample y(t) of each speech object, that is, the speech signal sample s′(t) of each potential speech object can be extracted from the speech signal sample obtained by overlapping the pure speech signal samples of at least one speech object through the speech extractor.

[0042] According to an exemplary embodiment of the present disclosure, the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object can be Fourier transformed respectively to obtain the first Fourier transform result STFT(x(t)) corresponding to the first speech signal sample x(t) and the second Fourier transform result STFT(y(t)) corresponding to the overlapping speech signal sample y(t). Then, the first Fourier transform result STFT(x(t)) and the second Fourier transform result STFT(y(t)) of each speech object can be input into the speech extractor to obtain the speech extraction result of each speech object. Next, the speech extraction result of each speech object can be inverse Fourier transformed to obtain the third speech signal sample s′(t) extracted from the overlapping speech signal sample y(t) of each speech object.

[0043] Furthermore, the speech extractor can use mask estimation technology based on complex domain features, and the constructed model can use a forward neural network layer and a dilated convolutional network layer. With the help of the first speech signal sample x(t) of a certain speech object, the speech extractor can be used to extract the potential speech signal sample s′(t) of the speech object from the overlapping speech signal samples y(t) of the speech object:

[0044] s′(t)=iSTFT(f(STFT(x(t)),STFT(y(t)))) (1)

[0045] Wherein, f(·) represents the above-mentioned feedforward neural network layer and the atrous convolutional network layer, STFT represents short time Fourier transform, and iSTFT represents invert Short Time Fourier transform.

[0046] It should be noted that the first speech signal sample x(t) can be converted to the time-frequency domain through short-time Fourier transform. If the original audio signal with a length of T is represented as x(t) in the time domain, where t represents time, 0<t≤T, then after short-time Fourier transform, x(t) can be represented in the time-frequency domain as:

[0047] S(n,k)=STFT(x(t)) (2)

[0048] Among them, n is the frame sequence, 0<n≤N, N is the total number of frames; k is the center frequency sequence, 0<k≤K, K is the total number of frequency points. That is, S(n,k) in the complex domain can be used as the input of the speech extractor, which includes the amplitude value |S(n,k)| and the phase value ∠S(n,k) in the time-frequency domain.

[0049] In step 103, the third speech signal sample s′(t) corresponding to each speech object may be input into a speech representation extractor to obtain a speech representation e corresponding to each speech object.

[0050] According to an exemplary embodiment of the present disclosure, the third voice signal sample s′(t) corresponding to each voice object can be Fourier transformed to obtain a third Fourier transform result. That is, the third voice signal sample s′(t) can be first transferred to the time-frequency domain through a short-time Fourier transform, and then the amplitude value |STFT(s′(t))| in the time-frequency domain can be obtained. Then, the Mel-cepstrum feature corresponding to the third Fourier transform result of each voice object can be obtained. That is, based on the amplitude value |STFT(s′(t))| in the time-frequency domain, the Mel-cepstrum feature M(n,l) in the Mel domain can be obtained, where l is the feature dimension.

[0051] M(n,l)=Mel(|STFT(s′(t))|) (3)

[0052] Next, the Mel-cepstral features corresponding to each speech object can be input into the speech representation extractor to obtain frame-level features. That is, a five-layer time-delay network g(·) can be used to process the frame-based Mel-cepstral features M(n,l) to obtain frame-level features R(n,i), where i is the number of nodes in the neural network layer.

[0053] R(n,i)=g(Mel(|STFT(s′(t))|)) (4)

[0054] Then, the frame-level features corresponding to each speech object can be statistically pooled to obtain the speech representation corresponding to each speech object. That is, after obtaining the frame-level features R(n,i), the frame-level features R(n,i) can be statistically pooled to obtain the speech representation e of the speech object at the sentence level. The speech representation e of the speech object at the sentence level can be a vector:

[0055] e=statistic pooing(R(n,i)) (5)

[0056] In step 104, the speech representation e corresponding to each speech object may be input into the identity discriminator to obtain the identity prediction probability corresponding to each speech object. For example, a two-layer feedforward network may be used to transform the speech representation e corresponding to the speech object and predict the identity prediction probability that the speech representation e belongs to each speech object in all speech objects in the training data.

[0057] Reference Figure 2 , Figure 2is a schematic diagram showing the training phase and the testing phase of a speech object recognition model according to an exemplary embodiment of the present disclosure. Figure 2 In the training stage, the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object can be used as the input of the speech extractor. At this time, the output of the speech extractor is the potential third speech signal sample s′(t) of the speech object extracted from the overlapping speech signal sample y(t) of the speech object. Next, the third speech signal sample s′(t) can be used as the input of the speech representation extractor, and then the speech representation extractor can output the speech representation e corresponding to the speech object. Finally, the speech representation e corresponding to the speech object can be used as the input of the identity discriminator to obtain the identity prediction probability corresponding to the speech object, that is, the identity prediction probability of predicting the speech object as each speech object in speech object 1, speech object 2,..., speech object m can be obtained.

[0058] In step 105, the loss may be calculated according to the third speech signal sample s′(t), the second speech signal sample s(t), the identity prediction probability and the identification label corresponding to each speech object.

[0059] According to an exemplary embodiment of the present disclosure, the speech extractor needs to improve the SNR of the speech signal of the extracted potential speech object as much as possible, and the speech representation extractor needs to improve the classification accuracy of the speech object as much as possible. Therefore, a joint training method can be adopted to weightedly sum the loss function corresponding to the speech extractor and the loss function corresponding to the speech representation extractor as the loss function used in actual training. For example, the first loss J1 can be calculated based on the third speech signal sample s′(t) and the second speech signal sample s(t) corresponding to each speech object; the second loss J2 can be calculated based on the identity prediction probability and identification label corresponding to each speech object. Next, the first loss J1 and the second loss J2 can be weightedly summed to obtain the loss J during actual training:

[0060] J=J1+α*J2 (6)

[0061] Here, α is a weight, and illustratively, α may be 10.

[0062] According to an exemplary embodiment of the present disclosure, the first loss J1 may be calculated by a scale-invariant signal to distortion ratio (SISDR) loss function:

[0063]

[0064] According to an exemplary embodiment of the present disclosure, the second loss J2 can be calculated by a cross entropy loss function:

[0065]

[0066] Where C is the total number of speech objects in the training data. When the speech representation e of the speech segment is output by the speech object c, P c is equal to 1, otherwise, P c =0, P(c|e) is the identity prediction probability that the identity discriminator predicts the speech representation e of the speech segment as the identity output by the speech object c.

[0067] In step 106, the parameters of the speech extractor, the speech representation extractor, and the identity discriminator may be adjusted by loss, thereby training the speech object recognition model.

[0068] Reference Figure 3 , Figure 3 is a flowchart illustrating a method for recognizing a voice object according to an exemplary embodiment of the present disclosure.

[0069] Reference Figure 3 In step 301, a measured speech signal z(t) may be obtained. The measured speech signal z(t) may be a speech signal of one speech object or may be speech signals of multiple speech objects.

[0070] It should be noted that each candidate object among the multiple candidate objects needs to pre-register a candidate speech signal to obtain a candidate speech signal library a1(t), a2(t),…, a n After the measured speech signal z(t) is obtained, the candidate speech signals a1(t), a2(t), ..., a1(t) and a2(t) pre-registered for each candidate in the candidate speech signal library can be converted into n (t) and the tested speech signal z(t) form a test pair: (a1(t),z(t)),(a2(t),z(t)),…,(a n (t),z(t)).

[0071] The following operations may be performed for each candidate speech signal in the plurality of candidate speech signals:

[0072] In step 302, each candidate speech signal may be input into Figure 1 The speech extractor and speech representation extractor trained by the training method in the training method are used to obtain the first speech representation e corresponding to the candidate speech signal. r .

[0073] In step 303, each candidate speech signal and the tested speech signal z(t) may be input into a speech extractor and a speech representation extractor to obtain a second speech representation e corresponding to the candidate speech signal. t .

[0074] For example, refer to Figure 2 ,exist Figure 2 In the test phase, the candidate speech signal a1(t) pre-registered by candidate object 1 can be input into the speech extractor to obtain s′ 1r (t), where s′ 1r (t) can be obtained using the following formula:

[0075] s′ 1r (t)=iSTFT(f(STFT(a1(t)),STFT(a1(t)))) (9)

[0076] Then, s′ can be 1r (t) Input the speech representation extractor to obtain the first speech representation corresponding to the candidate speech signal a1(t)

[0077] The candidate speech signal a1(t) pre-registered by the candidate object 1 and the measured speech signal z(t) can also be input into the speech extractor to obtain the potential speech signal s′ of the candidate object 1 extracted from the measured speech signal z(t). 1t (t), where s′ 1t (t) can be obtained using the following formula:

[0078] s′ 1t (t)=iSTFT(f(STFT(a1(t)),STFT(z(t)))) (10)

[0079] Then, s′ can be 1t (t) Input the speech representation extractor to obtain the second speech representation corresponding to the candidate speech signal a1(t) At this point, the speech representation pair corresponding to the test pair (a1(t), z(t)) is obtained Similarly, we can also obtain the corresponding (s′) of the test pair (a2(t), z(t)) 2r (t),s′ 2t (t)), and using (s′ 2r (t),s′ 2t (t)) obtain the speech representation pair A test pair (a n (t),z(t)) corresponds to (s′ nr (t),s′ nt (t)), and use (s′ nr(t),s′ nt (t)) obtain the speech representation pair etc.

[0080] In step 304, the first speech representation corresponding to each candidate speech signal may be obtained. r With the second phonetic representation t For example, the first speech representation corresponding to the candidate speech signal a1(t) can be obtained Second speech representation The first speech representation corresponding to the candidate speech signal a2(t) can be obtained. Second speech representation The similarity of speech representation between them; the candidate speech signal a can be obtained n (t) The corresponding first phonetic representation Second speech representation The similarity of the phonetic representations between them, etc.

[0081] It should be noted that the first speech representation e contained in each speech representation pair can be calculated by cosine similarity (Cosine Similarity) or probability linear discriminant analysis (Probability Linear Discriminant Analysis, PLDA) r With the second phonetic representation t The similarity of the phonetic representations between them.

[0082] In step 305, the identity of the tested speech object corresponding to the tested speech signal z(t) may be determined according to the similarity of the speech representation corresponding to each candidate speech signal.

[0083] According to an exemplary embodiment of the present disclosure, the maximum voice representation similarity among multiple voice representation similarities corresponding to multiple candidate voice signals can be determined. Then, it can be determined whether the maximum voice representation similarity is greater than a preset threshold to obtain a judgment result. Next, the identity of the voice object under test can be determined based on the judgment result.

[0084] According to an exemplary embodiment of the present disclosure, when the judgment result indicates that the maximum speech representation similarity is greater than a preset threshold, the tested speech object may be determined to be a candidate object corresponding to the maximum speech representation similarity.

[0085] According to an exemplary embodiment of the present disclosure, when the judgment result indicates that the above-mentioned maximum voice representation similarity is less than or equal to a preset threshold, it can be determined that the identity of the voice object under test cannot be recognized.

[0086] Figure 4is a block diagram showing a training device for a speech object recognition model according to an exemplary embodiment of the present disclosure, wherein the speech object recognition model includes a speech extractor, a speech representation extractor, and an identity discriminator.

[0087] Reference Figure 4 The device 400 may include a speech signal sample acquisition module 401, a speech signal sample extraction module 402, a speech representation extraction module 403, an identity prediction probability acquisition module 404, a calculation module 405 and an adjustment module 406.

[0088] The speech signal sample acquisition module 401 can acquire the first speech signal sample x(t), the second speech signal sample s(t) and the overlapping speech signal sample y(t) of each speech object in multiple speech objects. Among them, the first speech signal sample x(t) and the second speech signal sample s(t) are pure speech signal samples with different contents of the speech object, that is, the first speech signal sample x(t) and the second speech signal sample s(t) are speech signal samples with higher signal-to-noise ratio (SNR) with different contents of the speech object. The overlapping speech signal sample y(t) can be a speech signal sample obtained by overlapping the pure speech signal samples of at least one speech object, and the pure speech signal samples of at least one speech object include at least the second speech signal sample s(t) of the speech object. The first speech signal sample x(t) corresponds to an identification label label. At this time, each speech object in the multiple speech objects can correspond to a data pair (x(t), y(t), s(t), label).

[0089] Furthermore, the reason why the first speech signal sample x(t) and the second speech signal sample s(t) are required to be speech signal samples with different contents of the same speech object is that when the trained speech object recognition model is used to identify the identity of the speech object in the subsequent process, the candidate speech signal pre-registered for each speech object in the candidate speech signal library and the tested speech signal need to be formed into a test pair, and the identity of the tested speech object corresponding to the tested speech signal is determined based on the multiple test pairs formed. In an actual test scenario, the candidate speech signal of the candidate object contained in each test pair is different from the content of the potential speech signal of the candidate object in the tested speech signal contained in the test pair. For example, all employees of a company can pre-register their own candidate speech signals, and during the online meeting of the company's employees, the content of the speech of each participating employee during the meeting must be different from the content of the candidate speech signal pre-registered by the employee. Therefore, in the training stage, in order to simulate the real test scenario as much as possible, the first speech signal sample x(t) and the second speech signal sample s(t) used for training can be set as speech signal samples with different contents, so that the accuracy of the speech object recognition model in identifying the identity of the tested speech object can be guaranteed as much as possible.

[0090] It should be noted that in the training phase of the speech object recognition model, the speech data of tens of thousands of speech objects are used to train a speech extractor, a speech representation extractor and an identity discriminator. The speech extractor, speech representation extractor and identity discriminator can all be built based on neural networks.

[0091] The speech signal sample extraction module 402 can input the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object into the speech extractor to obtain the third speech signal sample s′(t) extracted from the overlapping speech signal sample y(t) of each speech object, that is, the speech extractor can extract the potential speech signal sample s′(t) of each speech object from the speech signal samples obtained by overlapping the pure speech signal samples of at least one speech object.

[0092] According to an exemplary embodiment of the present disclosure, the speech signal sample extraction module 402 can perform Fourier transform on the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object respectively, and obtain the first Fourier transform result STFT(x(t)) corresponding to the first speech signal sample x(t) and the second Fourier transform result STFT(y(t)) corresponding to the overlapping speech signal sample y(t). Then, the first Fourier transform result STFT(x(t)) and the second Fourier transform result STFT(y(t)) of each speech object can be input into the speech extractor to obtain the speech extraction result of each speech object. Next, the speech extraction result of each speech object can be inverse Fourier transformed to obtain the third speech signal sample s′(t) extracted from the overlapping speech signal sample y(t) of each speech object.

[0093] The speech representation extraction module 403 may input the third speech signal sample s′(t) corresponding to each speech object into the speech representation extractor to obtain the speech representation e corresponding to each speech object.

[0094] According to an exemplary embodiment of the present disclosure, the speech characterization extraction module 403 can perform Fourier transform on the third speech signal sample s′(t) corresponding to each speech object to obtain a third Fourier transform result. Then, the Mel-cepstral feature corresponding to the third Fourier transform result of each speech object can be obtained. Next, the Mel-cepstral feature corresponding to each speech object can be input into the speech characterization extractor to obtain frame-level features. Then, the frame-level features corresponding to each speech object can be statistically pooled to obtain the speech characterization e corresponding to each speech object.

[0095] The identity prediction probability acquisition module 404 can input the speech representation e corresponding to each speech object into the identity discriminator to obtain the identity prediction probability corresponding to each speech object. For example, a two-layer forward network can be used to convert the speech representation e corresponding to the speech object, and predict the identity prediction probability that the speech representation e belongs to each speech object in all speech objects in the training data.

[0096] Return to reference Figure 2 ,exist Figure 2In the training stage, the first speech signal sample x(t) and the overlapping speech signal sample y(t) of each speech object can be used as the input of the speech extractor. At this time, the output of the speech extractor is the potential third speech signal sample s′(t) of the speech object extracted from the overlapping speech signal sample y(t) of the speech object. Next, the third speech signal sample s′(t) can be used as the input of the speech representation extractor, and then the speech representation extractor can output the speech representation e corresponding to the speech object. Finally, the speech representation e corresponding to the speech object can be used as the input of the identity discriminator to obtain the identity prediction probability corresponding to the speech object, that is, the identity prediction probability of predicting the speech object as each speech object in speech object 1, speech object 2,..., speech object m can be obtained.

[0097] The calculation module 405 may calculate the loss according to the third speech signal sample s′(t), the second speech signal sample s(t), the identity prediction probability and the identification label corresponding to each speech object.

[0098] According to an exemplary embodiment of the present disclosure, the speech extractor needs to improve the SNR of the speech signal of the extracted potential speech object as much as possible, and the speech representation extractor needs to improve the classification accuracy of the speech object as much as possible. Therefore, a joint training method can be adopted to weightedly sum the loss function corresponding to the speech extractor and the loss function corresponding to the speech representation extractor as the loss function used in actual training. For example, the calculation module 405 can calculate the first loss J1 based on the third speech signal sample s′(t) and the second speech signal sample s(t) corresponding to each speech object; the calculation module 405 can calculate the second loss J2 based on the identity prediction probability and identification label corresponding to each speech object. Next, the calculation module 405 can perform a weighted summation of the first loss J1 and the second loss J2 to obtain the loss J during actual training:

[0099] J=J1+α*J2

[0100] Here, α is a weight, and illustratively, α may be 10.

[0101] According to an exemplary embodiment of the present disclosure, the first loss J1 may be calculated by a scale-invariant signal-to-distortion ratio (SISDR) loss function.

[0102] According to an exemplary embodiment of the present disclosure, the second loss J2 may be calculated by a cross entropy loss function.

[0103] The adjustment module 406 can adjust the parameters of the speech extractor, the speech representation extractor and the identity discriminator through the loss, so as to train the speech object recognition model.

[0104] Figure 5is a block diagram illustrating a recognition apparatus for a voice object according to an exemplary embodiment of the present disclosure.

[0105] Reference Figure 5 The device 500 may include a measured speech signal acquisition module 501, a first speech representation acquisition module 502, a second speech representation acquisition module 503, a speech representation similarity acquisition module 504 and a determination module 505.

[0106] The measured speech signal acquisition module 501 can acquire the measured speech signal z(t), wherein the measured speech signal z(t) may be a speech signal of one speech object or may be speech signals of multiple speech objects.

[0107] It should be noted that each candidate object among the multiple candidate objects needs to pre-register a candidate speech signal to obtain a candidate speech signal library a1(t), a2(t),…, a n After the measured speech signal z(t) is obtained, the candidate speech signals a1(t), a2(t), ..., a1(t) and a2(t) pre-registered for each candidate in the candidate speech signal library can be converted into n (t) and the tested speech signal z(t) form a test pair: (a1(t),z(t)),(a2(t),z(t)),…,(a n (t),z(t)).

[0108] The following operations may be performed for each candidate speech signal in the plurality of candidate speech signals:

[0109] The first speech representation acquisition module 502 can input each candidate speech signal into the Figure 1 The speech extractor and speech representation extractor trained by the training method in the training method are used to obtain the first speech representation e corresponding to the candidate speech signal. r .

[0110] The second speech representation acquisition module 503 can input each candidate speech signal and the measured speech signal z(t) into the speech extractor and the speech representation extractor to obtain the second speech representation e corresponding to the candidate speech signal. t .

[0111] Return to reference Figure 2 ,exist Figure 2 In the test phase, the candidate speech signal a1(t) pre-registered by candidate object 1 can be input into the speech extractor to obtain s′ 1r (t). Then, s′ can be 1r (t) Input the speech representation extractor to obtain the first speech representation corresponding to the candidate speech signal a1(t)

[0112] The candidate speech signal a1(t) pre-registered by the candidate object 1 and the measured speech signal z(t) can also be input into the speech extractor to obtain the potential speech signal s′ of the candidate object 1 extracted from the measured speech signal z(t). 1t (t). Then, s′ can be 1t (t) Input the speech representation extractor to obtain the second speech representation corresponding to the candidate speech signal a1(t) At this point, the speech representation pair corresponding to the test pair (a1(t), z(t)) is obtained Similarly, we can also obtain the corresponding (s′) of the test pair (a2(t), z(t)) 2r (t),s′ 2t (t)), and using (s′ 2r (t),s′ 2t (t)) obtain the speech representation pair A test pair (a n (t),z(t)) corresponds to (s′ nr (t),s′ nt (t)), and use (s′ nr (t),s′ nt (t)) obtain the speech representation pair etc.

[0113] The speech representation similarity acquisition module 504 may acquire the first speech representation corresponding to each candidate speech signal. r With the second phonetic representation t For example, the first speech representation corresponding to the candidate speech signal a1(t) can be obtained Second speech representation The first speech representation corresponding to the candidate speech signal a2(t) can be obtained Second speech representation The similarity of speech representation between them; the candidate speech signal a can be obtained n (t) The corresponding first phonetic representation Second speech representation The similarity of the phonetic representations between them, etc.

[0114] It should be noted that the first speech representation e contained in each speech representation pair can be calculated by cosine similarity or probabilistic linear discriminant analysis PLDA. r With the second phonetic representation t The similarity of the phonetic representations between them.

[0115] The determination module 505 may determine the identity of the detected speech object corresponding to the detected speech signal z(t) according to the similarity of the speech representation corresponding to each candidate speech signal.

[0116] According to an exemplary embodiment of the present disclosure, the determination module 505 can determine the maximum voice representation similarity among multiple voice representation similarities corresponding to multiple candidate voice signals. Then, it can be determined whether the maximum voice representation similarity is greater than a preset threshold to obtain a judgment result. Next, the identity of the voice object under test can be determined based on the judgment result.

[0117] According to an exemplary embodiment of the present disclosure, when the judgment result indicates that the maximum speech representation similarity is greater than a preset threshold, the determination module 505 may determine that the tested speech object is a candidate object corresponding to the maximum speech representation similarity.

[0118] According to an exemplary embodiment of the present disclosure, when the judgment result indicates that the above-mentioned maximum voice representation similarity is less than or equal to a preset threshold, the determination module 505 may determine that the identity of the voice object under test cannot be recognized.

[0119] Figure 6 is a block diagram illustrating an electronic device 600 according to an exemplary embodiment of the present disclosure.

[0120] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602, wherein the at least one memory 601 stores instructions, and when the instructions are executed by the at least one processor 602, a method for training a speech object recognition model and a method for recognizing a speech object according to an exemplary embodiment of the present disclosure are executed.

[0121] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instructions. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit collection capable of executing the above instructions (or instruction sets) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0122] In the electronic device 600, the processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0123] The processor 602 may execute instructions or codes stored in the memory 601, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0124] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 601 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 602 can read files stored in the memory.

[0125] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0126] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the training method of the voice object recognition model and the recognition method of the voice object according to the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0127] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided, including a computer program, which, when executed by a processor, implements the method for training a speech object recognition model and the method for recognizing a speech object according to the present disclosure.

[0128] According to the training method, recognition method and device of the speech object recognition model disclosed in the present invention, since a large number of pure speech signal samples and overlapping speech signal samples of speech objects are used to train the speech object recognition model, the trained speech object recognition model can accurately extract the speech representation of the specified speech object from the speech signals of multiple speech objects, and when multiple speech objects all output speech, the accuracy of identifying the identity of the current speech object can be guaranteed; and the speech object recognition method disclosed in the present invention can, with the help of the candidate speech signal, use the trained speech extractor and speech representation extractor to extract the second speech representation corresponding to the potential candidate speech signal from the tested speech signal, and then determine the identity of the tested speech object according to the speech representation similarity between the first speech representation corresponding to the candidate speech signal and the second speech representation. Since the speech object recognition method disclosed in the present invention can extract the speech representation of the specified speech object from the speech signal of a single speech object or the speech signals of multiple speech objects, the identity of the current speech object can also be accurately identified when multiple speech objects all output speech.

[0129] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0130] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a speech object recognition model, characterized in that: The speech object recognition model includes a speech extractor, a speech representation extractor and an identity discriminator, and the training method includes: Acquire a first speech signal sample, a second speech signal sample, and an overlapping speech signal sample of each speech object among a plurality of speech objects, wherein the first speech signal sample and the second speech signal sample are clean speech signal samples with different contents of the speech object, the overlapping speech signal sample is a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, the clean speech signal sample of at least one speech object includes at least the second speech signal sample of the speech object, and the first speech signal sample corresponds to an identification label; Inputting the first speech signal sample and the overlapping speech signal sample of each speech object into the speech extractor to obtain a third speech signal sample extracted from the overlapping speech signal sample of each speech object; Inputting the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object; Inputting the speech representation corresponding to each speech object into the identity discriminator to obtain the identity prediction probability corresponding to each speech object; Calculate the loss according to the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object; Adjusting the parameters of the speech extractor, the speech representation extractor and the identity discriminator by means of the loss, thereby training the speech object recognition model; The step of calculating the loss according to the third voice signal sample, the second voice signal sample, the identity prediction probability and the identification label corresponding to each voice object includes: Calculating a first loss according to a third speech signal sample and a second speech signal sample corresponding to each speech object; Calculate the second loss based on the identity prediction probability and identification label corresponding to each speech object; The first loss and the second loss are weightedly summed to obtain the loss.

2. The training method according to claim 1, characterized in that: The first loss is calculated by a scale-invariant signal-to-distortion ratio loss function.

3. The training method according to claim 1, characterized in that: The second loss is calculated by a cross entropy loss function.

4. The training method according to claim 1, characterized in that: The step of inputting the first speech signal sample and the overlapping speech signal sample of each speech object into the speech extractor to obtain a third speech signal sample extracted from the overlapping speech signal sample of each speech object comprises: Performing Fourier transform on the first speech signal sample and the overlapping speech signal sample of each speech object respectively to obtain a first Fourier transform result corresponding to the first speech signal sample and a second Fourier transform result corresponding to the overlapping speech signal sample; Inputting the first Fourier transform result and the second Fourier transform result of each voice object into the voice extractor to obtain a voice extraction result of each voice object; An inverse Fourier transform is performed on the speech extraction result of each speech object to obtain a third speech signal sample extracted from the overlapping speech signal samples of each speech object.

5. The training method according to claim 1, characterized in that: The step of inputting the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object comprises: Performing Fourier transform on the third speech signal sample corresponding to each speech object to obtain a third Fourier transform result; Obtain the Mel-frequency cepstrum feature corresponding to the third Fourier transform result of each speech object; Inputting the Mel-frequency cepstrum features corresponding to each speech object into the speech representation extractor to obtain frame-level features; The frame-level features corresponding to each speech object are statistically pooled to obtain the speech representation corresponding to each speech object.

6. A method for recognizing a speech object, characterized in that: The identification method comprises: Acquire the voice signal under test; For each of the plurality of candidate speech signals, the following operations are performed: Inputting the candidate speech signals into the speech extractor and the speech representation extractor trained by the training method according to any one of claims 1 to 5 to obtain a first speech representation corresponding to each candidate speech signal, wherein the plurality of candidate speech signals are speech signals pre-registered by a plurality of candidate objects; Inputting the candidate speech signals and the tested speech signal into the speech extractor and the speech representation extractor to obtain a second speech representation corresponding to each candidate speech signal; Obtaining a speech representation similarity between a first speech representation and a second speech representation corresponding to each candidate speech signal; The identity of the tested speech object corresponding to the tested speech signal is determined according to the speech representation similarity corresponding to each candidate speech signal.

7. The identification method according to claim 6, characterized in that: The step of determining the identity of the tested speech object corresponding to the tested speech signal according to the speech representation similarity corresponding to each candidate speech signal comprises: Determine a maximum speech representation similarity among a plurality of speech representation similarities corresponding to the plurality of candidate speech signals; Determine whether the maximum speech representation similarity is greater than a preset threshold, and obtain a determination result; Based on the determination result, the identity of the detected voice object is determined.

8. The identification method according to claim 7, characterized in that: The determining the identity of the detected voice object based on the judgment result includes: In a case where the judgment result indicates that the maximum speech representation similarity is greater than the preset threshold, the detected speech object is determined to be a candidate object corresponding to the maximum speech representation similarity.

9. The identification method according to claim 7, characterized in that: The determining the identity of the detected voice object based on the judgment result includes: When the judgment result indicates that the maximum speech representation similarity is less than or equal to the preset threshold, it is determined that the identity of the detected speech object cannot be recognized.

10. A training device for a speech object recognition model, characterized in that: The speech object recognition model includes a speech extractor, a speech representation extractor and an identity discriminator, and the training device includes: A speech signal sample acquisition module is configured to acquire a first speech signal sample, a second speech signal sample and an overlapping speech signal sample of each speech object in a plurality of speech objects, wherein the first speech signal sample and the second speech signal sample are clean speech signal samples with different contents of the speech object, the overlapping speech signal sample is a speech signal sample obtained by overlapping clean speech signal samples of at least one speech object, the clean speech signal sample of at least one speech object includes at least the second speech signal sample of the speech object, and the first speech signal sample corresponds to an identification label; A speech signal sample extraction module is configured to input the first speech signal sample and the overlapping speech signal sample of each speech object into the speech extractor to obtain a third speech signal sample extracted from the overlapping speech signal sample of each speech object; A speech representation extraction module, configured to input the third speech signal sample corresponding to each speech object into the speech representation extractor to obtain the speech representation corresponding to each speech object; An identity prediction probability acquisition module is configured to input the speech representation corresponding to each speech object into the identity discriminator to obtain the identity prediction probability corresponding to each speech object; A calculation module configured to calculate the loss according to the third speech signal sample, the second speech signal sample, the identity prediction probability and the identification label corresponding to each speech object; an adjustment module, configured to adjust the parameters of the speech extractor, the speech representation extractor and the identity discriminator through the loss, so as to train the speech object recognition model; Wherein, the calculation module is configured as follows: Calculating a first loss according to a third speech signal sample and a second speech signal sample corresponding to each speech object; Calculate the second loss based on the identity prediction probability and identification label corresponding to each speech object; The first loss and the second loss are weightedly summed to obtain the loss.

11. A device for recognizing a speech object, characterized in that: The identification device comprises: A voice signal acquisition module is configured to acquire a voice signal under test; For each of the plurality of candidate speech signals, the following operations are performed: A first speech representation acquisition module is configured to input the candidate speech signals into the speech extractor and the speech representation extractor trained by the training method according to any one of claims 1 to 5, and obtain a first speech representation corresponding to each candidate speech signal, wherein the plurality of candidate speech signals are speech signals pre-registered by a plurality of candidate objects; A second speech representation acquisition module is configured to input the candidate speech signal and the tested speech signal into the speech extractor and the speech representation extractor to obtain a second speech representation corresponding to each candidate speech signal; A speech representation similarity acquisition module is configured to acquire the speech representation similarity between the first speech representation and the second speech representation corresponding to each candidate speech signal; The determination module is configured to determine the identity of the tested voice object corresponding to the tested voice signal according to the similarity of the voice representation corresponding to each candidate voice signal.

12. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method of the speech object recognition model as described in any one of claims 1 to 5 or the recognition method of the speech object as described in any one of claims 6 to 9.

13. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method of a speech object recognition model as described in any one of claims 1 to 5 or the recognition method of a speech object as described in any one of claims 6 to 9.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for training a speech object recognition model according to any one of claims 1 to 5 or the method for recognizing a speech object according to any one of claims 6 to 9 is implemented.

Images