Voiceprint recognition model training and related recognition method, electronic device and storage medium

Abstract

Embodiments of the present application provide a voiceprint recognition model training and related recognition method, an electronic device and a storage medium. The voiceprint recognition method comprises: extracting acoustic features of a to-be-recognized speech segment; extracting hidden features of the to-be-recognized speech segment, wherein the hidden features are used to indicate the speaking content corresponding to the to-be-recognized speech segment; splicing the acoustic features and the hidden features of the to-be-recognized speech segment to obtain spliced features of the to-be-recognized speech segment; inputting the spliced features of the to-be-recognized speech segment into a voiceprint recognition model to perform voiceprint recognition on the to-be-recognized speech segment and obtain a voiceprint recognition result. The present scheme can improve the accuracy of voiceprint recognition.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a voiceprint recognition model training and related recognition method, electronic device and storage medium. Background Technology

[0002] Voiceprint recognition (VPR), also known as speaker recognition, is a type of biometric technology. In practice, it uses a voiceprint recognition model to identify the voice characteristics of different speakers in order to recognize or confirm the speaker.

[0003] Currently, voiceprint recognition models extract acoustic features from speech data to indicate the speaker's voice characteristics, and then distinguish the speech information of different speakers based on the extracted acoustic features.

[0004] However, the acoustic features extracted from speech data include not only the speaker's voice features, but also the relevant features of the speech content. Therefore, the acoustic features extracted from speech segments with different content spoken by the same speaker will have significant differences, while the acoustic features extracted from speech segments with the same content spoken by different speakers will have high similarity, resulting in a low accuracy rate for voiceprint recognition. Summary of the Invention

[0005] In view of this, embodiments of this application provide a voiceprint recognition model training and related recognition method, electronic device and storage medium to at least solve or alleviate the above problems.

[0006] According to a first aspect of the embodiments of this application, a voiceprint recognition method is provided, comprising: extracting acoustic features of a speech segment to be recognized; extracting hidden features of the speech segment to be recognized, wherein the hidden features are used to indicate the speaking content corresponding to the speech segment to be recognized; concatenating the acoustic features and hidden features of the speech segment to be recognized to obtain concatenated features of the speech segment to be recognized; inputting the concatenated features of the speech segment to be recognized into a voiceprint recognition model, performing voiceprint recognition on the speech segment to be recognized, and obtaining a voiceprint recognition result.

[0007] According to a second aspect of the embodiments of this application, a speaker recognition method is provided, comprising: concatenating a first speech segment and a second speech segment to obtain a concatenated speech segment; extracting acoustic features of the concatenated speech segment; extracting hidden features of the concatenated speech segment, wherein the hidden features are used to indicate the speaking content corresponding to the first speech segment and the second speech segment; concatenating the acoustic features and hidden features of the concatenated speech segment to obtain concatenated features of the concatenated speech segment; inputting the concatenated features of the concatenated speech segment into a voiceprint recognition model to perform voiceprint recognition on the concatenated speech segment to obtain a voiceprint recognition result of the concatenated speech segment; and determining a speaker recognition result based on the voiceprint recognition result of the concatenated speech segment, wherein the speaker recognition result is used to indicate the probability that the first speech segment and the second speech segment correspond to the same speaker.

[0008] According to a third aspect of the embodiments of this application, a speaker log generation method is provided, comprising: extracting acoustic features of speech to be processed; extracting hidden features of the speech to be processed, wherein the hidden features are used to indicate the speaking content corresponding to the speech to be processed; concatenating the acoustic features and the hidden features of the speech to be processed to obtain concatenated features of the speech to be processed; inputting the concatenated features of the speech to be processed into a voiceprint recognition model to perform voiceprint recognition on the speech to be processed to obtain a voiceprint recognition result of the speech to be processed; and inputting the voiceprint recognition result of the speech to be processed into a feedforward neural network for feature extraction to obtain a speaker log, wherein the speaker log is used to identify speech segments in the speech to be processed divided according to the speaker.

[0009] According to a fourth aspect of the embodiments of this application, a method for training a voiceprint recognition model is provided, comprising: acquiring at least two speech segments from an unlabeled speech dataset; concatenating the at least two speech segments to obtain a first speech sample; extracting acoustic features of the first speech sample; extracting hidden features of the first speech sample, wherein the hidden features are used to indicate the speaking content corresponding to each speech segment in the first speech sample; concatenating the acoustic features and hidden features of the first speech sample to obtain concatenated features of the first speech sample; inputting the concatenated features of the first speech sample into a voiceprint recognition model to be trained to obtain a voiceprint recognition result output by the voiceprint recognition model; determining the voiceprint recognition loss of the voiceprint recognition model based on the voiceprint recognition result; adjusting the parameters of the voiceprint recognition model based on the voiceprint recognition loss until the voiceprint recognition loss is less than a preset first loss threshold, and stopping the above-described training of the voiceprint recognition model.

[0010] According to a fifth aspect of the embodiments of this application, an electronic device is provided, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; the memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the method described in any one of the first to fourth aspects.

[0011] According to a sixth aspect of the embodiments of this application, a computer storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided by any one of the first to fourth aspects described above.

[0012] According to a seventh aspect of the embodiments of this application, a computer program product is provided, including computer instructions that instruct a computing device to perform the method provided by any one of the first to fourth aspects described above.

[0013] As described in the above technical solution, after extracting the acoustic features and hidden features of the speech segment to be identified, the acoustic features and hidden features are concatenated to obtain the concatenated features of the speech segment to be identified. These concatenated features are then input into the voiceprint recognition model to perform voiceprint recognition on the speech segment to be identified, thus obtaining the voiceprint recognition result. Since the acoustic features include the speaker's voiceprint features and related features of the speech content, and the hidden features can indicate the speech content corresponding to the speech segment to be identified, concatenating the acoustic features and hidden features into a concatenated feature is used as the input to the voiceprint recognition model. When the voiceprint recognition model performs voiceprint recognition based on the concatenated feature, it can shield the influence of different speech content and perform voiceprint recognition only based on the speaker's features, thereby improving the accuracy of voiceprint recognition. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0015] Figure 1 This is a schematic diagram of an exemplary system applied in one embodiment of this application;

[0016] Figure 2 This is a flowchart of a voiceprint recognition method according to an embodiment of this application;

[0017] Figure 3 This is a schematic diagram of a voiceprint recognition model according to an embodiment of this application;

[0018] Figure 4This is a flowchart of a speaker recognition method according to an embodiment of this application;

[0019] Figure 5 This is a flowchart of a speaker log generation method according to an embodiment of this application;

[0020] Figure 6 This is a flowchart of a voiceprint recognition model training method according to an embodiment of this application;

[0021] Figure 7 This is a flowchart of a voiceprint recognition model fine-tuning method according to an embodiment of this application;

[0022] Figure 8 This is a flowchart of a voiceprint recognition model fine-tuning method according to another embodiment of this application;

[0023] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0024] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the essence of the present application, well-known methods, processes, and flows are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0025] First, some nouns or terms that appear in the description of the embodiments of this application shall be interpreted as follows.

[0026] Voiceprint recognition: Voiceprint recognition is a type of biometric technology that identifies a speaker by their voice.

[0027] Speaker recognition: By extracting a person's voiceprint features, it can identify whether two audio recordings belong to the same person.

[0028] Speaker Log: In scenarios where multiple people are speaking, an audio clip includes the voices of multiple people, and the speaker's voice and content are distinguished.

[0029] Exemplary System

[0030] Figure 1 An exemplary system for a voiceprint recognition method and a voiceprint recognition model training method applicable to embodiments of this application is shown. For example... Figure 1 As shown, the system includes a cloud server 10, a communication network 20, and at least one user device 30. Figure 1 The example shown is for multiple user devices 30.

[0031] The cloud server 10 can be any suitable device for storing information, data, programs, and / or any other suitable type of content, including but not limited to distributed storage system devices, server clusters, computing cloud server clusters, etc. In some embodiments, the cloud server 10 can perform any suitable function. For example, the cloud server 10 can be used for voiceprint recognition and training words for a voiceprint recognition model. In some embodiments, the cloud server 10 can receive voice data sent by the user device 30, perform voiceprint recognition on the voice data, and send the voiceprint recognition result to the user device 30. In other embodiments, the cloud server 10 can receive model training instructions from the user device 30, train a voiceprint recognition model according to the model training instructions, and then send the trained voiceprint recognition model to the user device 30, or perform voiceprint recognition based on the trained voiceprint recognition model.

[0032] Communication network 20 can be any suitable combination of one or more wired and / or wireless networks. For example, communication network 20 can include any one or more of the following: the Internet, intranet, wide area network (WAN), local area network (LAN), wireless network, digital subscriber line (DSL) network, frame relay network, asynchronous transfer mode (ATM) network, virtual private network (VPN), and / or any other suitable communication network. User equipment 30 can be connected to communication network 20 via one or more communication links (e.g., communication link 112), which can be connected to cloud server 10 via one or more communication links (e.g., communication link 114). Communication links can be any communication link suitable for transmitting data between cloud server 10 and user equipment 30, such as network links, dial-up links, wireless links, hardwired links, any other suitable communication links, or any suitable combination of such links.

[0033] User equipment 30 may include any one or more devices suitable for voice acquisition and running a voiceprint recognition model for voiceprint recognition. User equipment 30 may include any suitable type of device, such as mobile devices, tablet computers, laptop computers, desktop computers, wearable computers, game consoles, media players, or conferencing equipment.

[0034] It should be noted that the use of cloud server 10 for voiceprint recognition and voiceprint recognition model training is only one application scenario of this application embodiment. The voiceprint recognition method and voiceprint recognition model training method provided in this application embodiment can also be implemented by local server, client, IoT device, etc., and this application embodiment does not limit this.

[0035] Voiceprint recognition method

[0036] Figure 2 This is a flowchart of a voiceprint recognition method according to an embodiment of this application. Figure 2 As shown, the voiceprint recognition method includes the following steps:

[0037] Step 201: Extract the acoustic features of the speech segment to be recognized.

[0038] The speech segment to be identified is the speech segment that needs to be identified by voiceprint recognition. The speech segment to be identified includes the voices of one or more speakers.

[0039] In one example, when extracting acoustic features of a speech segment to be recognized, the Fbank features of the speech segment to be recognized can be extracted as the acoustic features of the speech segment to be recognized.

[0040] Speech segments with the same content and different speakers have different acoustic features. Speech segments with different speakers and different content also have different acoustic features. Therefore, the acoustic features extracted from the speech segment to be identified include not only the speaker's voiceprint features, but also the relevant features of the speech content.

[0041] Step 202: Extract hidden features from the speech segment to be recognized.

[0042] After acquiring the speech segment to be recognized, in addition to extracting its acoustic features, it is also necessary to extract its hidden features. These hidden features indicate the corresponding speech content. For two different speech segments, if they correspond to the same speech content, then the hidden features extracted from these two speech segments will be the same.

[0043] Step 203: Concatenate the acoustic features and hidden features of the speech segment to be recognized to obtain the concatenated features of the speech segment to be recognized.

[0044] After extracting the acoustic features and hidden features of the speech segment to be recognized, the acoustic features and hidden features of the speech segment to be recognized are concatenated to obtain the concatenated features of the speech segment to be recognized. The method of concatenating acoustic features and hidden features can be any type of feature concatenation method. For example, concatenating two feature rectangles in the order of acoustic features to hidden features to obtain concatenated features, or concatenating two feature matrices in the order of hidden feature values ​​and acoustic features to obtain concatenated features, or taking a weighted average of acoustic features and hidden features to obtain concatenated features. The embodiments of this application do not limit the method of concatenating acoustic features and hidden features.

[0045] Step 204: Input the splicing features of the speech segment to be identified into the voiceprint recognition model, perform voiceprint recognition on the speech segment to be identified, and obtain the voiceprint recognition result.

[0046] A voiceprint recognition model is a pre-trained neural network model used to perform voiceprint recognition based on input feature data and output the voiceprint recognition result. After obtaining the spliced ​​features of the speech segment to be recognized, the spliced ​​features are input into the voiceprint recognition model, which then performs voiceprint recognition on the speech segment to be recognized, thereby obtaining the voiceprint recognition result output by the voiceprint recognition model.

[0047] Voiceprint recognition results can indicate the speaker corresponding to each frame in the speech segment to be recognized. Based on the voiceprint recognition results, it can be determined which frames in the speech segment to be recognized correspond to the same speaker. Therefore, when the speech segment to be recognized includes the voices of multiple speakers, it can be divided into multiple speech segments according to the speaker. Depending on the actual application scenario, the voiceprint recognition results can be sent to downstream applications as the final output, or they can be used as input to other neural network models for further processing. This application does not limit this aspect.

[0048] It should be understood that the voiceprint recognition model takes spliced ​​features as input, which are composed of acoustic features and hidden features. Therefore, when training the voiceprint recognition model, it is also necessary to train it using samples composed of acoustic features and hidden features. The training process of the voiceprint recognition model will be explained in detail in the following embodiments.

[0049] In this embodiment, after extracting the acoustic features and hidden features of the speech segment to be identified, the acoustic features and hidden features are concatenated to obtain the concatenated features of the speech segment to be identified. These concatenated features are then input into a voiceprint recognition model to perform voiceprint recognition on the speech segment to be identified, thus obtaining the voiceprint recognition result. Since acoustic features include the speaker's voiceprint features and related features of the speech content, and hidden features can indicate the speech content corresponding to the speech segment to be identified, concatenating the acoustic features and hidden features into a concatenated feature is used as input to the voiceprint recognition model. When the voiceprint recognition model performs voiceprint recognition based on the concatenated feature, it can shield the influence of different speech content and perform voiceprint recognition only based on the speaker's features, thereby improving the accuracy of voiceprint recognition.

[0050] In one possible implementation, when extracting the hidden features of the speech segment to be recognized, the Mel-Frequency Cepstral Coefficients (MFCC) features of the speech segment to be recognized are first extracted. Then, the MFCC features are input into the Automatic Speech Recognition (ASR) model. The ASR model extracts the hidden features of the speech content corresponding to each frame in the speech segment to be recognized. Then, the hidden features of the speech content corresponding to each frame in the speech segment to be recognized are sequentially combined to obtain the hidden features of the speech segment to be recognized.

[0051] After extracting the MFCC features of the speech segment to be recognized, the MFCC features are input into a pre-trained ASR model. The ASR model can perform speech recognition on the speech segment based on the MFCC features, thereby determining the speech content included in the speech segment. Furthermore, it can determine the speech content corresponding to each frame in the speech segment and obtain the hidden features of the speech content corresponding to each frame. Since each phonation in the speech content may correspond to multiple frames in the speech segment to be recognized, multiple adjacent frames may correspond to the same hidden features.

[0052] After obtaining the hidden features of the speech content corresponding to each frame in the speech segment to be recognized, the hidden features of the speech content corresponding to each frame can be sequentially combined according to the order of each frame in the speech segment to be recognized, so as to obtain the hidden features of the speech segment to be recognized, so that the hidden features of the speech segment to be recognized can completely indicate the speech content corresponding to the speech segment to be recognized.

[0053] In this embodiment, after extracting the MFCC features of the speech segment to be identified, the MFCC features are used as input to the ASR model. The ASR model extracts the hidden features of the speech content corresponding to each frame in the speech segment to be identified. Then, the hidden features of the speech content corresponding to each frame are combined to form the hidden features of the speech segment to be identified. This allows the hidden features of the speech segment to be identified to indicate the speech content corresponding to the speech segment at the frame level, ensuring that the hidden features can accurately indicate the speech content. This, in turn, ensures the accuracy of the recognition results obtained when performing voiceprint recognition based on the hidden features of the speech segment to be identified.

[0054] Figure 3 This is a schematic diagram of a voiceprint recognition model according to an embodiment of this application. Figure 3 As shown, the voiceprint recognition model includes a first convolutional layer 301, a first dilated convolutional layer 302, a second dilated convolutional layer 303, a third dilated convolutional layer 304, a second convolutional layer 305, a first encoding / decoding layer 306, and a second encoding / decoding layer 307, arranged sequentially. The input to the first convolutional layer 301 is the spliced ​​features of the speech segment to be recognized, and the output of the second encoding / decoding layer 307 is the voiceprint recognition result of the speech segment to be recognized. Following the order from the first convolutional layer 301 to the second encoding / decoding layer 307, the output of the product of the previous layers is used as the input in each subsequent layer.

[0055] In this embodiment, the voiceprint recognition model has a 7-layer structure. The first layer is a convolutional layer, the second to fourth layers are dilated convolutional layers, the fifth layer is a convolutional layer, and the sixth and seventh layers are encoding / decoding layers. This 7-layer structure can fully extract features from the input data, filter out the influence of speech content based on hidden features in the spliced ​​features, and perform voiceprint recognition based on the differences between speaker features, thereby ensuring the accuracy of the voiceprint recognition results. In addition, the second to fourth layers are dilated convolutional layers. Dilated convolutional layers perform dilated convolution on the input data, which can increase the receptive field of the convolutional neural network, thereby performing voiceprint recognition based on multiple adjacent frames in the speech segment to be recognized, further improving the accuracy of the voiceprint recognition results.

[0056] In one possible implementation, the first convolutional layer 301 and the second convolutional layer 305 are used to perform one-dimensional convolution on their respective inputs, and the first dilated convolutional layer 302, the second dilated convolutional layer 303, and the third dilated convolutional layer 304 are used to perform dilated convolution on their respective inputs. The first encoding / decoding layer 306 and the second encoding / decoding layer 307 each include at least one encoder and at least one decoder for encoding and decoding their respective inputs. The first dilated convolutional layer 302, the second dilated convolutional layer 303, and the third dilated convolutional layer 304 may all include an SE-Res2Block model, and the first encoding / decoding layer 306 and the second encoding / decoding layer 307 may all include a Transformer model.

[0057] The SE-Res2Block model comprises dilated convolutions with a 1-frame context between the preceding and following layers. The first layer reduces the feature dimensionality, the second dense layer restores the feature data to its original dimensionality, and then the SE module scales each channel. The entire unit uses a skip connection. Ensembles of the SE-Res2Block model can improve the performance of the voiceprint recognition model while reducing the number of model parameters.

[0058] In this embodiment, after the spliced ​​features of the speech segment to be identified are input into the voiceprint recognition model, one-dimensional convolution, three dilated convolutions, another one-dimensional convolution, and two encoding / decoding operations are performed sequentially. This extracts spoken language features from the input and spliced ​​features while filtering out features related to the spoken content. Voiceprint recognition is then performed based on the speaker's features, ensuring the accuracy of the voiceprint recognition results. All three dilated convolutional layers include the SE-Res2Block model. Using the SE-Res2Block model for dilated convolution not only increases the receptive field, thus improving the performance of the voiceprint recognition model, but also reduces the number of model parameters, thereby reducing the training time of the voiceprint recognition model.

[0059] Speaker recognition methods

[0060] In the application scenario of speaker recognition, based on the voiceprint recognition method provided in the above embodiments, this application provides a speaker recognition method for identifying whether two speech segments correspond to the same speaker. Figure 4 This is a flowchart of the speaker recognition method according to an embodiment of this application, such as... Figure 4 As shown, the speaker recognition method includes the following steps:

[0061] Step 401: Segment the first speech segment and the second speech segment to obtain the segmented speech segment.

[0062] The first and second speech segments are two speech segments that require speaker identification, that is, it is necessary to identify whether the first and second speech segments belong to the same speaker. It should be understood that the first and second speech segments each correspond to only one speaker, that is, the first and second speech segments each contain only the voice of one speaker.

[0063] When splicing the first and second speech segments, a separator can be inserted between them so that after extracting acoustic and hidden features, the voiceprint recognition model can still distinguish the features of the first and second speech segments.

[0064] Step 402: Extract the acoustic features of the spliced ​​speech segments.

[0065] Step 403: Extract hidden features from the spliced ​​speech segments.

[0066] Step 404: Segment the acoustic features and hidden features of the concatenated speech segments to obtain the concatenated features of the concatenated speech segments.

[0067] Step 405: Input the splicing features of the spliced ​​speech segments into the voiceprint recognition model, perform voiceprint recognition on the spliced ​​speech segments, and obtain the voiceprint recognition results of the spliced ​​speech segments.

[0068] It should be noted that steps 402 to 405 above can refer to steps 201 to 204 in the previous embodiment, and steps 402 to 405 will not be described again here.

[0069] Step 406: Determine the speaker identification result based on the voiceprint recognition result of the spliced ​​speech segment.

[0070] After obtaining the voiceprint recognition results of the spliced ​​speech segments, since the voiceprint recognition results can indicate the speaker corresponding to each frame in the spliced ​​speech segments, the speaker indicated by the part of the voiceprint recognition results corresponding to the first speech segment and the speaker indicated by the part of the voiceprint recognition results corresponding to the second speech segment can be determined based on the voiceprint recognition results of the spliced ​​speech segments. Thus, it can be determined whether the first speech segment and the second speech segment correspond to the same speaker, and finally output the probability that the first speech segment and the second speech segment correspond to the same speaker.

[0071] In one example, if the probability indicated by the speaker recognition result is zero, it means that the first speech segment and the second speech segment correspond to different speakers; if the probability indicated by the speaker recognition result is 1, it means that the first speech segment and the second speech segment correspond to the same speaker.

[0072] In this embodiment, two speech segments requiring speaker identification are concatenated to obtain a concatenated speech segment. Acoustic features and hidden features of the concatenated speech segment are then extracted. These acoustic and hidden features are concatenated to obtain concatenated features of the concatenated speech segment. These concatenated features are then input into a voiceprint recognition model to perform voiceprint recognition on the concatenated speech segment, obtaining the voiceprint recognition result. Finally, the speaker identification result is determined based on the voiceprint recognition result of the concatenated speech segment. Since acoustic features include the speaker's voiceprint features and related features of the speech content, and hidden features can indicate the speech content corresponding to the speech segment to be identified, concatenating acoustic and hidden features into a concatenated feature and using it as input to the voiceprint recognition model allows the model to shield against the influence of different speech content when performing voiceprint recognition based solely on speaker features. This improves the accuracy of voiceprint recognition and, consequently, the accuracy of the speaker identification result determined based on the voiceprint recognition result.

[0073] In one possible implementation, when determining the speaker recognition result based on the speaker recognition result of the spliced ​​speech segments, the matrix corresponding to the first speech segment and the matrix corresponding to the second speech segment in the speaker recognition result of the spliced ​​speech segments can be pooled to obtain distributed feature representations. Then, the distributed feature representations are mapped to the speaker recognition result through a fully connected layer.

[0074] The speaker recognition model includes a voiceprint recognition model, a pooling layer, and a fully connected layer. The voiceprint recognition result output by the voiceprint recognition model is input into the pooling layer. The pooling layer performs pooling processing on the voiceprint recognition result to obtain a distributed feature representation. The fully connected layer maps the distributed feature representation to the speaker recognition result.

[0075] In this embodiment, the voiceprint recognition result output by the voiceprint recognition model is pooled through a pooling layer to reduce its dimensionality and compress it, obtaining a distributed feature representation. This distributed feature representation is then input through a fully connected layer for classification, yielding a speaker recognition result that indicates whether the first and second speech segments correspond to the same speaker. By pooling the voiceprint recognition result, its dimensionality can be reduced and compressed, accelerating computation and improving the efficiency of speaker recognition.

[0076] It should be noted that, Figure 4 The speaker recognition method shown is a specific application of the voiceprint recognition method in the embodiments of this application. For specific voiceprint recognition methods, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0077] Speaker log generation method

[0078] In the application scenario of speaker recognition, based on the voiceprint recognition method provided in the above embodiments, this application embodiment provides a speaker log generation method for distinguishing the voices and speaking content of different speakers in the same audio. Figure 5 This is a flowchart of the speaker log generation method according to an embodiment of this application, such as... Figure 5 As shown, the speaker log generation method includes the following steps:

[0079] Step 501: Extract the acoustic features of the speech to be processed.

[0080] The audio to be processed corresponds to multiple speakers, that is, the audio to be processed includes the voice information of multiple roles. For example, the audio to be processed can be the audio collected during a multi-person conference, which includes the voice information of multiple people taking turns to speak, or the audio to be processed can be the audio collected in the summary of the teaching process, which includes the voice information of teachers and students taking turns to speak.

[0081] Step 502: Extract the hidden features of the speech to be processed.

[0082] Step 503: Concatenate the acoustic features and hidden features of the speech to be processed to obtain the concatenated features of the speech to be processed.

[0083] Step 504: Input the splicing features of the speech to be processed into the voiceprint recognition model, perform voiceprint recognition on the speech to be processed, and obtain the voiceprint recognition result of the speech to be processed.

[0084] It should be noted that steps 501 to 504 above can refer to steps 201 to 204 in the foregoing embodiments, and steps 501 to 504 will not be described again here.

[0085] Step 505: Input the voiceprint recognition result of the speech to be processed into the feedforward neural network for feature extraction to obtain the speaker log.

[0086] After obtaining the voiceprint recognition results of the speech to be processed, the voiceprint recognition results are input into a feedforward neural network for feature extraction to obtain a speaker log. The speaker log can identify the speech segments corresponding to different speakers in the speech to be processed. For example, the speaker log can indicate that there are 3 speakers corresponding to the speech to be processed, and indicate that the first 15 seconds of the speech to be processed is the speech segment corresponding to speaker Spk1, 15 seconds to 45 seconds is the speech segment corresponding to speaker Spk2, and 45 seconds to 120 seconds is the speech segment corresponding to speaker Spk3.

[0087] In this embodiment, acoustic features and hidden features of the speech to be processed are extracted, and the acoustic features and hidden features are concatenated to obtain concatenated features of the speech to be processed. These concatenated features are then input into a voiceprint recognition model to perform voiceprint recognition on the speech to be processed, obtaining the voiceprint recognition result. A speaker log is then generated based on the voiceprint recognition result. Since acoustic features include the speaker's voiceprint features and related features of the speech content, and hidden features can indicate the speech content corresponding to the speech segment to be recognized, concatenating acoustic features and hidden features into a concatenated feature is used as input to the voiceprint recognition model. When the voiceprint recognition model performs voiceprint recognition based on the concatenated feature, it can shield the influence of different speech content and perform voiceprint recognition only based on the speaker features, improving the accuracy of voiceprint recognition. This, in turn, improves the accuracy of the speaker log generated based on the voiceprint recognition result.

[0088] It should be noted that, Figure 5 The speaker log generation method shown is a specific application of the voiceprint recognition method in the embodiments of this application. For specific voiceprint recognition methods, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0089] Training methods for voiceprint recognition models

[0090] The training methods for the voiceprint recognition models in the above embodiments are described in detail below. Figure 6 This is a flowchart of a voiceprint recognition model training method according to an embodiment of this application, as follows: Figure 6 As shown, the training method for this voiceprint recognition model includes the following steps:

[0091] Step 601: Obtain at least two speech segments from the unlabeled speech dataset.

[0092] Unlabeled speech datasets contain a large amount of audio data, but do not label the speaker for each audio record. However, it is possible to determine whether two audio records correspond to the same speaker. When extracting speech segments from an unlabeled speech dataset, multiple speech segments can be randomly selected. Different speech segments can be extracted from the same audio record or from different audio records. When two speech segments are extracted from the same audio record, they correspond to the same speaker; when two speech segments are extracted from two audio records with different speakers, they correspond to different speakers.

[0093] Step 602: Segment the acquired speech segments to obtain the first speech sample.

[0094] After acquiring multiple speech segments, the order of each speech segment can be randomly determined, and then the speech segments can be spliced ​​together according to the determined order to obtain the first speech sample used to train the voiceprint recognition model.

[0095] Step 603: Extract the acoustic features of the first speech sample.

[0096] Step 604: Extract the hidden features of the first speech sample, wherein the hidden features are used to indicate the speaking content corresponding to each speech segment in the first speech sample.

[0097] Step 605: Concatenate the acoustic features and hidden features of the first speech sample to obtain the concatenated features of the first speech sample.

[0098] Step 606: Input the splicing features of the first speech sample into the voiceprint recognition model to be trained, and obtain the voiceprint recognition result output by the voiceprint recognition model.

[0099] It should be noted that steps 603 to 606 above can refer to steps 201 to 204 in the foregoing embodiments, and steps 603 to 606 will not be described again here.

[0100] Step 607: Determine the voiceprint recognition loss of the voiceprint recognition model based on the voiceprint recognition results.

[0101] After obtaining the voiceprint recognition result output by the voiceprint recognition model, the voiceprint recognition loss of the voiceprint recognition model can be determined based on the voiceprint recognition result. The voiceprint recognition loss can indicate the accuracy of the voiceprint recognition model in recognizing the voiceprint of the first speech sample.

[0102] Step 608: Adjust the parameters of the voiceprint recognition model according to the voiceprint recognition loss until the voiceprint recognition loss is less than the preset first loss threshold, and stop training the voiceprint recognition model as described above.

[0103] After obtaining the voiceprint recognition loss of the voiceprint recognition model, it is determined whether the voiceprint recognition loss is less than a preset first loss threshold. If the voiceprint recognition loss is less than the first loss threshold, it means that the accuracy of the voiceprint recognition model has reached the expected target, and training of the voiceprint recognition model is stopped, resulting in a voiceprint recognition model that can be used for inference. The obtained voiceprint recognition model can be used for speaker recognition and speaker log generation in the aforementioned embodiments. If the voiceprint recognition loss is greater than or equal to the first loss threshold, it means that the accuracy of the voiceprint recognition model has not reached the expected target, and the parameters of the voiceprint recognition model are adjusted according to the voiceprint recognition loss, and the above steps are repeated until the voiceprint recognition loss is less than the first loss threshold.

[0104] In this embodiment, unlabeled speech segments are acquired and concatenated to obtain a first speech sample. After extracting the acoustic and hidden features of the first speech sample, the acoustic and hidden features are concatenated to obtain the concatenated features of the first speech sample. The voiceprint recognition model is then trained using these concatenated features. Since the concatenated features include acoustic features and hidden features indicating the content of speech, training the voiceprint recognition model with these features allows the model to learn whether the differences between two speakers' features are caused by the sound itself or by the content of speech. Through continuous synchronous learning, the voiceprint recognition model gradually avoids the influence of speech content on speaker features, thereby ensuring that the trained voiceprint recognition model has a high accuracy rate when performing voiceprint recognition.

[0105] The speech segments are obtained from unlabeled speech datasets, eliminating the need for labeling and saving the cost of training the speaker recognition model. Since unlabeled speech data is widely available, massive amounts of such data can be used to train the speaker recognition model, thereby improving its accuracy.

[0106] In one possible implementation, when determining the voiceprint recognition loss of the voiceprint recognition model based on the voiceprint recognition results, the recognition result of the voiceprint recognition model for each frame in the first speech sample can be determined based on the voiceprint recognition results, and then the frame-level adversarial loss can be determined based on the recognition result of the voiceprint recognition model for each frame in the first speech sample, and the frame-level adversarial loss can be determined as the voiceprint recognition loss of the voiceprint recognition model.

[0107] In this embodiment, a frame-level adversarial loss is obtained as the voiceprint recognition loss based on the recognition result of each frame by the voiceprint recognition model. The parameters of the voiceprint recognition model are then adjusted based on this loss. Since the frame-level adversarial loss reflects the accuracy of the voiceprint recognition model at the frame level, it provides a more accurate and comprehensive reflection of the model's performance. Adjusting the model parameters based on this loss allows the trained voiceprint recognition model to achieve a higher accuracy rate.

[0108] Adjusting the model parameters of the voiceprint recognition model based on frame-level adversarial loss can make the features of the same speaker as similar as possible, while making the features of different speakers as far apart as possible, thereby enabling the voiceprint recognition model to learn the ability to distinguish between different speakers.

[0109] In one possible implementation, a voiceprint recognition model for general scenarios can be trained using the voiceprint recognition model training method described in the above embodiments. When applying the voiceprint recognition model to a specific scenario, it is also necessary to fine-tune the voiceprint recognition model to make it suitable for the corresponding application scenario and ensure the accuracy of voiceprint recognition in the corresponding application scenario.

[0110] Figure 7 This is a flowchart of a voiceprint recognition model fine-tuning method according to an embodiment of this application, used to fine-tune a voiceprint recognition model applied in a speaker recognition scenario, such as... Figure 7 As shown, the fine-tuning method for the voiceprint recognition model includes the following steps:

[0111] Step 701: Concatenate the two speech segments to obtain the second speech sample.

[0112] The speech segments used to splice together the second speech sample can be obtained from the speaker recognition scenario, and the obtained speech segments are unlabeled speech segments.

[0113] Step 702: Extract the acoustic features of the second speech sample.

[0114] Step 703: Extract the hidden features of the second speech sample, wherein the hidden features are used to indicate the speaking content corresponding to each speech segment in the second speech sample.

[0115] Step 704: Concatenate the acoustic features and hidden features of the second speech sample to obtain the concatenated features of the second speech sample.

[0116] Step 705: Input the splicing features of the second speech sample into the trained voiceprint recognition model to obtain the voiceprint recognition result of the voiceprint recognition model for the second speech sample.

[0117] The voiceprint recognition model trained in this step refers to the model that, through... Figure 6 The voiceprint recognition model trained by the method shown is shown.

[0118] It should be noted that steps 701 to 705 above can refer to steps 602 to 606 in the foregoing embodiments, and steps 701 to 705 will not be described again here.

[0119] Step 706: Perform pooling processing on the matrices corresponding to the two speech segments in the speaker recognition result of the second speech sample, respectively, to obtain the distributed feature representation corresponding to the second speech sample.

[0120] Step 707: The distributed feature representation corresponding to the second speech sample is mapped to the speaker recognition result corresponding to the second speech sample through a fully connected layer.

[0121] Step 708: Determine the speaker recognition loss based on the speaker recognition result corresponding to the second speech sample.

[0122] In one example, the speaker recognition result corresponding to the second speech sample can be input into the AM-Softmax loss algorithm to calculate the speaker recognition loss.

[0123] Step 709: Adjust the parameters of the pooling layer and the fully connected layer according to the speaker recognition loss, or adjust the parameters of the voiceprint recognition model, the pooling layer and the fully connected layer until the speaker recognition loss is less than the preset second loss threshold, and stop the above training of the voiceprint recognition model, the pooling layer and the fully connected layer.

[0124] After obtaining the speaker recognition loss, if the speaker recognition loss is greater than or equal to the second loss threshold, the parameters of the pooling layer and the fully connected layer can be adjusted only, or the parameters of the voiceprint recognition model, the pooling layer and the fully connected layer can be adjusted. This can be adapted to the needs of different users and specific application scenarios, improve the user experience and enhance the applicability of the method.

[0125] It should be noted that, Figure 7 The voiceprint recognition model fine-tuning method shown is a specific application of the voiceprint recognition model training method in the embodiments of this application. For the specific voiceprint recognition model training method, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0126] Figure 8 This is a flowchart of a voiceprint recognition model fine-tuning method according to another embodiment of this application, used to fine-tune a voiceprint recognition model applied in a speaker log generation scenario, such as... Figure 8 As shown, the fine-tuning method for the voiceprint recognition model includes the following steps:

[0127] Step 801: Extract the acoustic features of the speech to be processed.

[0128] The speech to be processed can be obtained from the speaker log generation scenario. The speech to be processed consists of unlabeled speech segments and speech data streams.

[0129] Step 802: Extract the hidden features of the speech to be processed, wherein the hidden features are used to indicate the speaking content corresponding to the speech to be processed.

[0130] Step 803: Concatenate the acoustic features and hidden features of the speech to be processed to obtain the concatenated features of the speech to be processed.

[0131] Step 804: Input the splicing features of the speech to be processed into the trained voiceprint recognition model to obtain the voiceprint recognition result of the speech to be processed.

[0132] The voiceprint recognition model trained in this step refers to the model that, through... Figure 6 The voiceprint recognition model trained by the method shown is shown.

[0133] It should be noted that steps 801 to 804 above can refer to steps 603 to 606 in the foregoing embodiments, and steps 801 to 804 will not be described again here.

[0134] Step 805: Extract features from the voiceprint recognition results of the speech to be processed using a feedforward neural network to obtain the speaker log corresponding to the speech to be processed.

[0135] Step 806: Determine the speaker log loss based on the speaker log corresponding to the speech to be processed.

[0136] In one example, the speaker log corresponding to the speech to be processed can be input into the PIT loss algorithm to calculate the speaker log loss.

[0137] Step 807: Adjust the parameters of the feedforward neural network based on the speaker log loss, or adjust the parameters of the voiceprint recognition model and the feedforward neural network until the speaker log loss is less than the preset third loss threshold, and stop the above training of the voiceprint recognition model and the feedforward neural network.

[0138] After obtaining the speaker log loss, if the speaker log loss is greater than or equal to the third loss threshold, the parameters of the feedforward neural network can be adjusted only, or the parameters of the voiceprint recognition model and the feedforward neural network can be adjusted. This can be adapted to the needs of different users and specific application scenarios, improve the user experience, and enhance the applicability of the method.

[0139] It should be noted that, Figure 8The voiceprint recognition model fine-tuning method shown is a specific application of the voiceprint recognition model training method in the embodiments of this application. For the specific voiceprint recognition model training method, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0140] electronic devices

[0141] Figure 9 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Specific embodiments of this application do not limit the specific implementation of the electronic device. Figure 9 As shown, the electronic device may include: a processor 902, a communications interface 904, a memory 906, and a communications bus 908. Wherein:

[0142] The processor 902, communication interface 904, and memory 906 communicate with each other via communication bus 908.

[0143] Communication interface 904 is used to communicate with other electronic devices or servers.

[0144] The processor 902 is used to execute program 910, which can specifically execute the relevant steps in any of the aforementioned method embodiments.

[0145] Specifically, program 910 may include program code that includes computer operation instructions.

[0146] The processor 902 may be a CPU, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The smart device includes one or more processors, which may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0147] RISC-V is an open-source instruction set architecture based on the Reduced Instruction Set Computing (RISC) principle. It can be applied to various aspects of microcontrollers and FPGA chips, specifically in areas such as IoT security, industrial control, mobile phones, and personal computers. Because its design considers small size, speed, and low power consumption, it is particularly suitable for modern computing devices such as warehouse-scale cloud computers, high-end mobile phones, and tiny embedded systems. With the rise of AIoT (Artificial Intelligence of Things), the RISC-V instruction set architecture is receiving increasing attention and support and is expected to become the next generation of widely used CPU architecture.

[0148] The computer operation instructions in this application embodiment can be computer operation instructions based on the RISC-V instruction set architecture. Correspondingly, the processor 902 can be designed based on the RISC-V instruction set. Specifically, the processor chip in the electronic device provided in this application embodiment can be a chip designed using the RISC-V instruction set. This chip can execute executable code based on the configured instructions, thereby implementing the methods in the above embodiments.

[0149] Memory 906 is used to store program 910. Memory 906 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0150] Specifically, program 910 can be used to cause processor 902 to execute the methods in any of the foregoing embodiments.

[0151] The specific implementation of each step in program 910 can be found in the corresponding descriptions of the steps and units in any of the foregoing method embodiments, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the foregoing method embodiments, and will not be repeated here.

[0152] The electronic device of this application extracts acoustic features and hidden features from the speech segment to be identified. These acoustic features and hidden features are then concatenated to obtain concatenated features of the speech segment. These concatenated features are then input into a voiceprint recognition model to perform voiceprint recognition on the speech segment, resulting in a voiceprint recognition result. Since acoustic features include the speaker's voiceprint features and related features of the speech content, and hidden features can indicate the speech content corresponding to the speech segment to be identified, concatenating acoustic features and hidden features into a concatenated feature is used as input to the voiceprint recognition model. When the voiceprint recognition model performs voiceprint recognition based on the concatenated feature, it can shield against the influence of different speech content and perform voiceprint recognition solely based on the speaker's features, thereby improving the accuracy of voiceprint recognition.

[0153] Computer storage media

[0154] This application also provides a computer-readable storage medium storing instructions for causing a machine to perform the methods described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling a computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0155] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0156] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0157] Computer program products

[0158] This application also provides a computer program product, including computer instructions that instruct a computing device to perform any corresponding operation in the above-described plurality of method embodiments.

[0159] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0160] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0161] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0162] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A voiceprint recognition method, comprising: Extract the acoustic features of the speech segment to be identified; Extract hidden features from the speech segment to be identified, wherein the hidden features are used to indicate the speaking content corresponding to the speech segment to be identified; The acoustic features and hidden features of the speech segment to be identified are concatenated to obtain the concatenated features of the speech segment to be identified; The splicing features of the speech segment to be identified are input into the voiceprint recognition model to perform voiceprint recognition on the speech segment to be identified, and the voiceprint recognition result is obtained. The step of extracting the hidden features of the speech segment to be recognized includes: extracting the Mel-frequency cepstral coefficient features of the speech segment to be recognized; inputting the Mel-frequency cepstral coefficient features into an automatic speech recognition model, performing speech recognition on the speech segment to be recognized using the Mel-frequency cepstral coefficient features in the automatic speech recognition model to determine the speaking content of the speech segment to be recognized; determining the hidden features of the speaking content; and determining the hidden features of the speech segment to be recognized based on the hidden features of the speaking content.

2. The method according to claim 1, wherein, Determining the hidden features of the speech segment to be identified based on the hidden features of the spoken content includes: The hidden features of the speech content corresponding to each frame in the speech segment to be identified are sequentially combined to obtain the hidden features of the speech segment to be identified.

3. The method according to claim 1, wherein, The voiceprint recognition model includes a first convolutional layer, a first dilated convolutional layer, a second dilated convolutional layer, a third dilated convolutional layer, a second convolutional layer, a first encoding / decoding layer, and a second encoding / decoding layer arranged sequentially. The input of the first convolutional layer is the splicing feature of the speech segment to be recognized, and the output of the second encoding / decoding layer is the voiceprint recognition result.

4. The method according to claim 3, wherein, The first and second convolutional layers are used to perform one-dimensional convolution on the input. The first, second, and third dilated convolutional layers all include the SE-Res2Block model. The first and second encoding / decoding layers both include the Transformer model.

5. A speaker recognition method, comprising: The first and second speech segments are spliced ​​together to obtain a spliced ​​speech segment. Extract the acoustic features of the spliced ​​speech segments; Extract hidden features from the spliced ​​speech segments, wherein the hidden features are used to indicate the speaking content corresponding to the first speech segment and the second speech segment; The acoustic features and hidden features of the spliced ​​speech segments are spliced ​​together to obtain the splicing features of the spliced ​​speech segments; The splicing features of the spliced ​​speech segment are input into the voiceprint recognition model to perform voiceprint recognition on the spliced ​​speech segment, thereby obtaining the voiceprint recognition result of the spliced ​​speech segment. Based on the voiceprint recognition results of the spliced ​​speech segments, a speaker recognition result is determined, wherein the speaker recognition result is used to indicate the probability that the first speech segment and the second speech segment correspond to the same speaker; The step of extracting the hidden features of the concatenated speech segment includes: extracting the Mel-frequency cepstral coefficient features of the concatenated speech segment; inputting the Mel-frequency cepstral coefficient features into an automatic speech recognition model, performing speech recognition on the concatenated speech segment using the Mel-frequency cepstral coefficient features in the automatic speech recognition model to determine the speech content of the concatenated speech segment; determining the hidden features of the speech content; and determining the hidden features of the concatenated speech segment based on the hidden features of the speech content.

6. The method according to claim 5, wherein, The step of determining the speaker identification result based on the voiceprint recognition result of the spliced ​​speech segment includes: Pooling is performed on the matrices corresponding to the first and second speech segments in the speaker recognition results of the spliced ​​speech segments to obtain distributed feature representations; The distributed feature representation is mapped to the speaker recognition result through a fully connected layer.

7. A method for generating speaker logs, comprising: Extract the acoustic features of the speech to be processed; Extract hidden features from the speech to be processed, wherein the hidden features are used to indicate the spoken content corresponding to the speech to be processed; The acoustic features and the hidden features of the speech to be processed are concatenated to obtain the concatenated features of the speech to be processed. The splicing features of the speech to be processed are input into the voiceprint recognition model to perform voiceprint recognition on the speech to be processed, and the voiceprint recognition result of the speech to be processed is obtained. The speaker recognition result of the speech to be processed is input into a feedforward neural network for feature extraction to obtain a speaker log, wherein the speaker log is used to identify speech segments in the speech to be processed according to the speaker. The step of extracting the hidden features of the speech to be processed includes: extracting the Mel-frequency cepstral coefficient features of the speech to be processed; inputting the Mel-frequency cepstral coefficient features into an automatic speech recognition model, performing speech recognition on the speech to be processed using the Mel-frequency cepstral coefficient features in the automatic speech recognition model to determine the speaking content of the speech to be processed; determining the hidden features of the speaking content; and determining the hidden features of the speech to be processed based on the hidden features of the speaking content.

8. A training method for a voiceprint recognition model, comprising: Obtain at least two speech segments from an unlabeled speech dataset; The at least two speech segments are spliced ​​together to obtain a first speech sample; Extract the acoustic features of the first speech sample; Extract hidden features from the first speech sample, wherein the hidden features are used to indicate the speaking content corresponding to each speech segment in the first speech sample; The acoustic features and hidden features of the first speech sample are concatenated to obtain the concatenated features of the first speech sample. The splicing features of the first speech sample are input into the voiceprint recognition model to be trained to obtain the voiceprint recognition result output by the voiceprint recognition model. Based on the voiceprint recognition results, determine the voiceprint recognition loss of the voiceprint recognition model; The parameters of the voiceprint recognition model are adjusted according to the voiceprint recognition loss until the voiceprint recognition loss is less than a preset first loss threshold, at which point the above training of the voiceprint recognition model is stopped. The step of extracting the hidden features of the first speech sample includes: extracting the Mel-frequency cepstral coefficient features of the first speech sample; inputting the Mel-frequency cepstral coefficient features into an automatic speech recognition model, performing speech recognition on the first speech sample using the Mel-frequency cepstral coefficient features in the automatic speech recognition model to determine the speaking content of the first speech sample; determining the hidden features of the speaking content; and determining the hidden features of the first speech sample based on the hidden features of the speaking content.

9. The method according to claim 8, wherein, The step of determining the voiceprint recognition loss of the voiceprint recognition model based on the voiceprint recognition result includes: Based on the voiceprint recognition results, determine the recognition result of the voiceprint recognition model for each frame in the first speech sample; Based on the recognition results of the voiceprint recognition model for each frame in the first speech sample, the frame-level adversarial loss is determined. The frame-level adversarial loss is determined as the voiceprint recognition loss of the voiceprint recognition model.

10. The method according to claim 8 or 9, wherein, The method further includes: The two speech segments are concatenated to obtain the second speech sample; Extract the acoustic features of the second speech sample; Extract hidden features from the second speech sample, wherein the hidden features are used to indicate the speaking content corresponding to each speech segment in the second speech sample; The acoustic features and hidden features of the second speech sample are concatenated to obtain the concatenated features of the second speech sample. The splicing features of the second speech sample are input into the trained voiceprint recognition model to obtain the voiceprint recognition result of the voiceprint recognition model for the second speech sample. The two speech segments in the speaker recognition result of the second speech sample are pooled by the pooling layer to obtain the distributed feature representation of the second speech sample. The distributed feature representation corresponding to the second speech sample is mapped to the speaker recognition result corresponding to the second speech sample through a fully connected layer; Based on the speaker recognition result corresponding to the second speech sample, determine the speaker recognition loss; The parameters of the pooling layer and the fully connected layer are adjusted according to the speaker recognition loss, or the parameters of the voiceprint recognition model, the pooling layer and the fully connected layer are adjusted until the speaker recognition loss is less than a preset second loss threshold, at which point the above training of the voiceprint recognition model, the pooling layer and the fully connected layer is stopped.

11. The method according to claim 8 or 9, wherein, The method further includes: Extract the acoustic features of the speech to be processed; Extract hidden features from the speech to be processed, wherein the hidden features are used to indicate the spoken content corresponding to the speech to be processed; The acoustic features and hidden features of the speech to be processed are concatenated to obtain the concatenated features of the speech to be processed. The splicing features of the speech to be processed are input into the trained voiceprint recognition model to obtain the voiceprint recognition result of the voiceprint recognition model for the speech to be processed. The speaker log corresponding to the speech to be processed is obtained by extracting features from the speaker recognition results of the speech to be processed through a feedforward neural network. Determine the speaker log loss based on the speaker log corresponding to the speech to be processed; The parameters of the feedforward neural network are adjusted based on the speaker log loss, or the parameters of the voiceprint recognition model and the feedforward neural network are adjusted until the speaker log loss is less than a preset third loss threshold, at which point the above training of the voiceprint recognition model and the feedforward neural network is stopped.

12. An electronic device, comprising: The processor, memory, communication interface, and communication bus communicate with each other through the communication bus. The memory is used to store at least one executable instruction that causes the processor to perform the operation corresponding to any one of the methods in claims 1-11.

13. A computer storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as claimed in any one of claims 1-11.

14. A computer program product comprising computer instructions that instruct a computing device to perform the method as claimed in any one of claims 1-11.

Images