A speaker recognition method, apparatus, device and storage medium
By selecting and constructing target speech representation models, and using multi-layer clustering and cluster label alignment to generate high-confidence pseudo-labels, the problem of insufficient purity of pseudo-labels in unsupervised speaker recognition is solved, and robust speaker recognition model training and accurate recognition are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MALANSHAN AUDIO & VIDEO LABORATORY
- Filing Date
- 2026-04-21
- Publication Date
- 2026-07-14
AI Technical Summary
Existing unsupervised speaker recognition technology relies on the quality of single-layer speech embedding and clustering stability. The purity of pseudo-labels is insufficient, and there is a lack of pseudo-supervised training mechanisms for high-confidence label matching, which leads to unstable model training direction and reduced discriminative ability and generalization.
The target layer in the initial speech representation model is selected, the target speech representation model is constructed, and high-confidence pseudo-labels are generated through multi-layer clustering and cluster label alignment. Pseudo-supervised training is then performed to improve the quality of pseudo-labels and build a robust speaker recognition model.
In scenarios without human annotation, improving the quality of pseudo-labels enhances the stability and discriminative ability of model training, enables accurate speaker recognition, reduces noise and pseudo-label interference, and improves the final representation quality of the model.
Smart Images
Figure CN122392518A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a speaker recognition method, apparatus, device, and storage medium. Background Technology
[0002] Speaker recognition, a key technology in speech signal processing, is widely used in call centers, financial customer service, government hotlines, and conference recording. Real-world applications accumulate massive amounts of unlabeled speech data, but manual annotation is costly, time-consuming, and carries significant privacy and compliance risks, making it difficult to construct labeled datasets that meet supervised training requirements. Therefore, unsupervised speaker recognition technology has become a focus of research and application. Current mainstream unsupervised speaker recognition methods extract speaker embeddings from speech and then perform unsupervised clustering, using cluster numbers as pseudo-labels for pseudo-supervised model training. However, this approach heavily relies on the quality of single-layer speech embeddings and cluster stability. Factors such as speaker acoustic similarity, uneven speech duration, complex background noise, device differences, and domain shift can easily lead to boundary samples and mis-clustered samples, resulting in insufficient purity and low reliability of pseudo-labels. Meanwhile, different layers of the pre-trained speech representation model exhibit varying abilities to express speaker attributes. Some layers emphasize low-level acoustic features, while others lean towards semantic information. Using only single-layer embedding clustering fails to fully utilize hierarchical information and is susceptible to layer selection bias, resulting in numerous errors in the pseudo-label supervision signal. Furthermore, existing technologies largely focus on pseudo-label generation, lacking a pseudo-supervised training mechanism that matches high-confidence labels. This leads to a disconnect between label generation and model training, with erroneous pseudo-labels interfering with the model's training direction and reducing its discriminative ability and generalization.
[0003] In summary, improving the quality of pseudo-labels and training speaker recognition models are pressing technical problems that need to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a speaker recognition method, apparatus, device, and storage medium, which can improve the quality of pseudo-labels and enable the training of a speaker recognition model, thereby achieving accurate speaker recognition. The specific solution is as follows: Firstly, this application provides a speaker recognition method, including: Several target layers are selected from the initial speech representation model, and a target speech representation model is constructed based on the target layers; the target layers are network layers in the initial speech representation model that meet preset performance conditions. Several speech samples are input into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer. The target embedding vectors in the same target layer are clustered to determine the target cluster label corresponding to each target embedding vector based on the clustering results. If the target cluster labels corresponding to any of the above speech are consistent, then the speech is determined as a target sample, so as to obtain a number of target samples selected from each of the above speech; Pseudo-labels are generated for each target sample, and an initial speaker recognition model is trained based on each target sample and its corresponding pseudo-labels to obtain a target speaker recognition model, which is then used to perform speaker recognition on the speech to be recognized.
[0005] Optionally, the selection of several target layers in the initial speech representation model includes: Acquire several speaker speech data and input each of the speaker speech data into the initial speech representation model to obtain the first embedding vector corresponding to each of the speaker speech data in each network layer of the initial speech representation model; The performance index of each network layer is determined based on the first embedding vector corresponding to each network layer; the performance index is the same error rate or the correct rate or the minimum detection cost calculated by the minimum detection cost function. If the performance metric of any of the network layers meets the preset performance condition, then the network layer is determined as the target layer.
[0006] Optionally, the step of inputting several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer includes: Each of the aforementioned speech samples is input into the target speech representation model to obtain the feature vector corresponding to each of the aforementioned speech samples in each of the aforementioned target layers, and the target embedding vector corresponding to the feature vector is determined by pooling or convergence operations.
[0007] Optionally, clustering the target embedding vectors of the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results includes: Unsupervised clustering is performed on each of the target embedding vectors in the same target layer to obtain the initial cluster labels corresponding to each of the target embedding vectors in the same layer; The initial cluster labels of different target layers are aligned, and the target cluster label corresponding to each target embedding vector is determined based on the alignment result.
[0008] Optionally, before determining the speech as the target sample, the method further includes: For any given speech, determine the number of identical target cluster labels. If the number is greater than a preset threshold, then the speech is identified as the target sample.
[0009] Optionally, the process of training the initial speaker recognition model based on each of the target samples and the pseudo-labels corresponding to each of the target samples includes: Each of the target samples is input into the initial speaker recognition model to obtain the speaker category prediction result output by the initial speaker recognition model; The target loss is determined between the pseudo-label corresponding to each target sample and the speaker category prediction result corresponding to each target sample, and the initial speaker recognition model is optimized based on the target loss to obtain the target speaker recognition model.
[0010] Optionally, the target speaker recognition model includes an encoder and a classifier; Accordingly, the step of using the target speaker recognition model to perform speaker recognition on the speech to be recognized includes: The encoder of the target speaker recognition model is used to generate the embedding vector corresponding to the speech to be recognized; The classifier of the target speaker recognition model performs speaker recognition based on the embedding vector corresponding to the speech to be recognized, and outputs a corresponding prediction distribution; the prediction distribution includes several speaker categories and the probability corresponding to each speaker category.
[0011] Secondly, this application provides a speaker recognition device, comprising: The target layer filtering module is used to filter several target layers in the initial speech representation model and construct a target speech representation model based on the several target layers; the target layer is a network layer in the initial speech representation model that meets preset performance conditions. The target cluster label determination module is used to input several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer, and to cluster the target embedding vectors in the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results. The target sample determination module is used to determine the speech as a target sample if the target cluster labels corresponding to any of the speech are consistent, so as to obtain a number of target samples selected from each of the speech; The speaker recognition module is used to generate pseudo-labels corresponding to each target sample, and to train an initial speaker recognition model based on each target sample and the pseudo-labels corresponding to each target sample to obtain a target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized.
[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned speaker recognition method.
[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned speaker recognition method.
[0014] In this application, firstly, several target layers in the initial speech representation model are selected, and a target speech representation model is constructed based on these target layers. The target layer is a network layer in the initial speech representation model that meets preset performance conditions. Then, several speech samples are input into the target speech representation model to obtain the target embedding vectors corresponding to each speech sample in each target layer. The target embedding vectors of the same target layer are clustered to determine the target cluster labels corresponding to each target embedding vector based on the clustering results. If the target cluster labels corresponding to any speech sample are consistent, the speech sample is determined as a target sample to obtain several target samples selected from each speech sample. Finally, pseudo labels corresponding to each target sample are generated, and the initial speaker recognition model is trained based on each target sample and the pseudo labels corresponding to each target sample to obtain the target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized. As can be seen from the above, this application first evaluates the performance of each network layer of the initial speech representation model, selects the target layers that meet the preset performance conditions, and constructs the target speech representation model. Then, it inputs several speech samples into the target speech representation model, obtains the target embedding vectors corresponding to each speech sample in different target layers, and then clusters the target embedding vectors of the same target layer to obtain the target cluster labels corresponding to each target embedding vector. Subsequently, speech samples with consistent target cluster labels are selected as high-confidence target samples. Next, reliable pseudo-labels are generated for the target samples, and the target samples and their corresponding pseudo-labels are used as supervised data to train the initial speaker recognition model, ultimately obtaining a target speaker recognition model that can be used for actual speech recognition. In this way, this application can automatically construct a set of pseudo-label samples with high purity and strong stability without relying on any manual identity labeling, but only utilizing the clustering structure between the multi-layer speaker representation of the pre-trained model and the unlabeled speech, and directly use it for speaker recognition model training. In this way, this application can improve the quality of pseudo-labels, effectively reduce the interference of noisy pseudo-labels on the model training process, improve training stability and the final representation quality of the model, significantly enhance the discrimination ability and training stability of the speaker recognition model in scenarios without manual annotation, and achieve accurate speaker recognition. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0016] Figure 1 A flowchart of a speaker recognition method provided in this application; Figure 2 This application provides a schematic diagram of the structure of a speaker recognition device; Figure 3 This application provides a structural diagram of an electronic device. Detailed Implementation
[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0018] Speaker recognition, a key technology in speech signal processing, is widely used in call centers, financial customer service, government hotlines, and conference recording. Real-world applications accumulate massive amounts of unlabeled speech data, but manual annotation is costly, time-consuming, and carries significant privacy and compliance risks, making it difficult to construct labeled datasets that meet supervised training requirements. Therefore, unsupervised speaker recognition technology has become a focus of research and application. Current mainstream unsupervised speaker recognition methods extract speaker embeddings from speech and then perform unsupervised clustering, using cluster numbers as pseudo-labels for pseudo-supervised model training. However, this approach heavily relies on the quality of single-layer speech embeddings and cluster stability. Factors such as speaker acoustic similarity, uneven speech duration, complex background noise, device differences, and domain shift can easily lead to boundary samples and mis-clustered samples, resulting in insufficient purity and low reliability of pseudo-labels. Meanwhile, different layers of the pre-trained speech representation model exhibit varying abilities to express speaker attributes. Some layers emphasize low-level acoustic features, while others lean towards semantic information. Using only single-layer embedding clustering fails to fully utilize hierarchical information and is susceptible to layer selection bias, resulting in numerous errors in the pseudo-label supervision signal. Furthermore, existing technologies largely focus on pseudo-label generation, lacking a pseudo-supervised training mechanism that matches high-confidence labels. This leads to a disconnect between label generation and model training, with erroneous pseudo-labels interfering with the model's training direction and reducing its discriminative ability and generalization. Therefore, this application provides a speaker recognition scheme that improves pseudo-label quality and enables the training of a speaker recognition model, thereby achieving accurate speaker identification.
[0019] See Figure 1 As shown in the figure, an embodiment of the present invention discloses a speaker recognition method, which may include: Step S11: Select several target layers in the initial speech representation model, and construct a target speech representation model based on the several target layers; the target layer is a network layer in the initial speech representation model that meets the preset performance conditions.
[0020] Understandably, since different network layers of the initial speech representation model have different abilities to express speaker attributes, this embodiment does not directly use all network layers in the initial speech representation model to participate in subsequent clustering. Instead, it first evaluates the discriminative ability of each network layer and selects several key target layers with strong speaker discriminative ability to construct the target speech representation model.
[0021] Specifically, a validation test set containing several speaker speech data is first constructed, and each network layer is evaluated using the validation test set. The process may include: firstly, acquiring several speaker speech data and inputting each speaker speech data into the initial speech representation model to obtain the first embedding vector corresponding to each network layer of the initial speech representation model; then, determining the performance index of each network layer based on the first embedding vector corresponding to each network layer; the performance index is an equal error rate or an accuracy rate or the minimum detection cost calculated by the minimum detection cost function; if the performance index of any network layer meets the preset performance condition, then the network layer is determined as the target layer.
[0022] More specifically, for each network layer, the first embedding vector of several speaker speech data is extracted using that layer, and then a speaker similarity score is constructed on the validation test set. The speaker discrimination performance index corresponding to that network layer is then calculated based on the speaker similarity score. The performance index can be equal error rate, minimum detection cost function, validation accuracy, or other indicators that reflect the speaker discrimination ability. In one specific implementation, the equal error rate of each network layer on the validation test set is calculated, and the network layers are sorted from low to high according to their equal error rates. The top K layers are selected as the target layer set. : ; Where K represents the number of target layers, This represents the Kth target layer. The purpose of this step is to filter out network layers that contribute weakly to speaker differentiation or are biased towards non-speaker information, thereby improving the stability of subsequent clustering results.
[0023] Step S12: Input several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer, and cluster the target embedding vectors in the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results.
[0024] In this embodiment, several speech samples are input into a target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer. The specific process may include: inputting each speech sample into the target speech representation model to obtain the feature vector corresponding to each speech sample in each target layer, and determining the target embedding vector corresponding to the feature vector through pooling or convergence operations.
[0025] Specifically, the first step is to obtain the set of unlabeled speech. ;in, Indicates the first Unannotated audio samples This represents the total number of samples. Each speech item is input into a pre-trained target speech representation model, and speech representations at each target layer of the model are extracted. For any target layer... This allows us to obtain the speech representation corresponding to the target layer, i.e., the feature vector. The speech is then obtained through pooling or aggregation operations. The utterance-level embedding at this target layer, i.e., the target embedding vector .
[0026] Next, clustering is performed on each target embedding vector of the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results. The specific process may include: first, performing unsupervised clustering on each target embedding vector of the same target layer to obtain the initial cluster label corresponding to each target embedding vector of the same layer; then aligning the initial cluster labels of different target layers and determining the target cluster label corresponding to each target embedding vector based on the alignment results.
[0027] Specifically, unsupervised clustering is performed on the target embedding vectors of each target layer. For any target layer... The clustering results can be expressed as: ; in, This represents an unsupervised clustering algorithm. Represents the target layer The corresponding sample cluster attribution results.
[0028] Since the cluster numbers in different target layers are generated independently and cannot be directly compared number by number, this embodiment further aligns the initial cluster labels of different target layers based on the overlap, co-occurrence or other matching relationships of cluster members between different target layers. This allows semantically similar clusters in different target layers to be mapped to a unified label reference system, thereby obtaining the target cluster labels corresponding to each target embedding vector based on the alignment results.
[0029] Step S13: If the target cluster labels corresponding to any of the speech are consistent, then the speech is determined as a target sample to obtain several target samples selected from each of the speech.
[0030] In this embodiment, after completing cross-layer cluster label alignment, the cluster affiliation of each speech sample in each target layer is compared. For a given speech sample... If the target cluster labels aligned across multiple target layers satisfy the consistency condition, then the speech sample is considered to be... It has stable attribution from the perspective of multi-layered speaker representation, and can represent speech samples. As the target sample, it is included in the high-confidence pseudo-label sample set. The high-confidence sample set can be represented as: ; in, This indicates the target cluster label after cross-layer label alignment.
[0031] Correspondingly, speech samples that do not meet the consistency condition, i.e., speech sets. Except for high-confidence sample sets The remaining sample set consists of the speech samples other than those in the sample set. .
[0032] In one specific implementation, the consistency condition may not be strict full-layer consistency, but rather consistency that satisfies a preset proportion, for example in... At least one of the target layers has If the target cluster labels of all layers are consistent, it can be regarded as a high-confidence sample. That is, for any speech, the number of identical target cluster labels is determined. If the number is greater than a preset threshold, the speech is determined as the target sample.
[0033] Step S14: Generate pseudo-labels corresponding to each target sample, and train the initial speaker recognition model based on each target sample and the pseudo-labels corresponding to each target sample to obtain a target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized.
[0034] For high-confidence sample sets For each target sample in this embodiment, a corresponding pseudo-label is generated based on the cluster affiliation result of the target sample in the aligned reference frame, i.e., the target cluster label. Thus, a high-confidence pseudo-label training set is formed based on the target samples and the pseudo-labels corresponding to each target sample: ; Next, in this embodiment, an initial speaker recognition model is further constructed, and a pseudo-label training set is used. The initial speaker recognition model is subjected to pseudo-supervised training. The training process may include: first, inputting each target sample into the initial speaker recognition model and obtaining the speaker category prediction result output by the initial speaker recognition model; then, determining the target loss between the pseudo label corresponding to each target sample and the speaker category prediction result corresponding to each target sample, and optimizing the initial speaker recognition model based on the target loss to obtain the target speaker recognition model.
[0035] Specifically, during the training phase, for any target sample Let the initial speaker recognition model output the speaker category prediction result as follows: Target sample The corresponding pseudo-tag is Then, a supervised loss based on pseudo-labels can be used to optimize the model. In a specific implementation, the target loss... for: ; The aforementioned target loss term is used to enable the initial speaker recognition model to learn speaker category boundaries on high-confidence target samples. Since the high-confidence target samples are obtained through multi-layer clustering consistency screening, their label reliability is significantly higher than that of samples obtained directly from single-layer clustering, thus providing a more stable pseudo-supervision signal for the model.
[0036] It should be noted that, to further reduce the impact of residual noise in pseudo-labels on training, this embodiment can also introduce a label robustness handling mechanism during pseudo-supervised training. For example, different training priorities can be assigned to different target samples based on the consistency strength of the target samples across multiple target layers; the training weights of target samples with unstable long-term predictions or abnormally large loss values can be reduced during training; and the consistency of target sample attribution can be periodically reviewed to dynamically remove target samples that may be incorrectly included in the high-confidence set.
[0037] For the remaining sample set In one specific implementation, this embodiment may omit the remaining sample set in the current round. Incorporate supervised training, while only including the remaining sample set Retained as samples to be updated. This is pending further testing of the current speaker recognition model on the high-confidence sample set. After one or more rounds of training, the updated model is used to re-extract the embedding representations of all samples, and clustering and pseudo-label generation are performed again to obtain a new set of pseudo-labeled samples of higher quality. Through this iterative approach, the improved model capabilities can further enhance embedding separability and continuously optimize model performance.
[0038] In another specific implementation, the remaining sample set It can also participate in training as an unsupervised update object, for example, only for feature extraction, cluster update, or auxiliary statistics, without directly undertaking the role of pseudo-supervised classification. Regardless of the specific implementation, the core of this embodiment is still: using a multi-layer clustering consistency mechanism to screen out target samples with high confidence pseudo-labels, and using the target samples as the main source of pseudo-supervision, so as to build a more robust unsupervised speaker recognition model training process.
[0039] In this embodiment, a final target speaker recognition model is obtained after training. The target speaker recognition model includes an encoder and a classifier. In this embodiment, the target speaker recognition model can be used to perform speaker recognition on the speech to be recognized. The specific process may include: first, using the encoder of the target speaker recognition model to generate an embedding vector corresponding to the speech to be recognized; then, using the classifier of the target speaker recognition model to perform speaker recognition based on the embedding vector corresponding to the speech to be recognized, and outputting a corresponding prediction distribution; the prediction distribution includes several speaker categories and the probability corresponding to each speaker category.
[0040] It is understandable that the target speaker recognition model in this embodiment has a lightweight, general-purpose, and easily integrated overall structure. It mainly relies on pre-trained representation extraction, layer selection, unsupervised clustering, cluster alignment, consistency screening, and pseudo-supervised training based on high-confidence samples. It does not require manual labeling or an additional complex external distillation framework, thus it can be easily deployed in existing unsupervised speaker recognition systems. Whether in large-scale server-side training scenarios or in scenarios involving rapid target domain adaptation and incremental learning on the device side, this embodiment has high engineering application value.
[0041] As can be seen from the above, in this embodiment, several target layers in the initial speech representation model are first selected, and a target speech representation model is constructed based on these target layers. The target layer is a network layer in the initial speech representation model that meets the preset performance conditions. Then, several speech samples are input into the target speech representation model to obtain the target embedding vectors corresponding to each speech sample in each target layer. The target embedding vectors of the same target layer are clustered to determine the target cluster labels corresponding to each target embedding vector based on the clustering results. If the target cluster labels corresponding to any speech sample are consistent, the speech sample is determined as a target sample to obtain several target samples selected from each speech sample. Finally, pseudo labels corresponding to each target sample are generated, and the initial speaker recognition model is trained based on each target sample and the pseudo labels corresponding to each target sample to obtain the target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized. As can be seen from the above, in this embodiment, the performance of each network layer of the initial speech representation model is first evaluated, and the target layer that meets the preset performance conditions is selected to construct the target speech representation model. Then, several speech samples are input into the target speech representation model to obtain the target embedding vectors corresponding to each speech sample in different target layers. Subsequently, the target embedding vectors of the same target layer are clustered to obtain the target cluster labels corresponding to each target embedding vector. Then, speech samples with consistent target cluster labels are selected as high-confidence target samples. Next, reliable pseudo-labels are generated for the target samples. The target samples and their corresponding pseudo-labels are used as supervised data to train the initial speaker recognition model, and finally, a target speaker recognition model that can be used for actual speech recognition is obtained. In this way, this embodiment can automatically construct a set of pseudo-label samples with high purity and strong stability without relying on any manual identity labeling, but only using the clustering structure between the multi-layer speaker representation of the pre-trained model and the unlabeled speech, and directly use it for speaker recognition model training. In this way, this embodiment can improve the quality of pseudo-labels, effectively reduce the interference of noisy pseudo-labels on the model training process, improve training stability and the final representation quality of the model, significantly enhance the discrimination ability and training stability of the speaker recognition model in scenarios without manual annotation, achieve accurate speaker recognition, and has high engineering application value.
[0042] Accordingly, see Figure 2 As shown in the illustration, this application also provides a speaker recognition device, which may include: The target layer filtering module 11 is used to filter several target layers in the initial speech representation model and construct a target speech representation model based on the several target layers; the target layer is a network layer in the initial speech representation model that meets preset performance conditions. The target cluster label determination module 12 is used to input several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer, and to cluster the target embedding vectors in the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results. The target sample determination module 13 is used to determine the speech as a target sample if the target cluster labels corresponding to any of the speech are consistent, so as to obtain a number of target samples selected from each of the speech. The speaker recognition module 14 is used to generate pseudo-labels corresponding to each of the target samples, and to train the initial speaker recognition model based on each of the target samples and the pseudo-labels corresponding to each of the target samples to obtain a target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized.
[0043] In some specific embodiments, the target layer filtering module 11 may include: The speaker speech data acquisition unit is used to acquire several speaker speech data and input each of the speaker speech data into the initial speech representation model to obtain the first embedding vector corresponding to each of the speaker speech data in each network layer of the initial speech representation model; A performance index determination unit is used to determine the performance index of each network layer based on the first embedding vector corresponding to each network layer; the performance index is an equal error rate or an accuracy rate or a minimum detection cost calculated by the minimum detection cost function. The target layer determination unit is configured to determine the network layer as the target layer if the performance index of any network layer meets the preset performance condition.
[0044] In some specific embodiments, the target cluster label determination module 12 may include: The target embedding vector determination unit is used to input each of the speech into the target speech representation model to obtain the feature vector corresponding to each of the speech in each of the target layers, and to determine the target embedding vector corresponding to the feature vector through pooling operation or convergence operation.
[0045] In some specific embodiments, the target cluster label determination module 12 may include: An initial cluster label determination unit is used to perform unsupervised clustering on each of the target embedding vectors of the same target layer to obtain the initial cluster labels corresponding to each of the target embedding vectors of the same layer. The target cluster label determination unit is used to align the initial cluster labels of different target layers and determine the target cluster label corresponding to each target embedding vector based on the alignment result.
[0046] In some specific embodiments, the speaker recognition device may further include: The target sample determination module is used to determine the number of identical target cluster labels for any given speech. If the number is greater than a preset threshold, the speech is determined as the target sample.
[0047] In some specific embodiments, the speaker recognition module 14 may include: The prediction result acquisition unit is used to input each of the target samples into the initial speaker recognition model and obtain the speaker category prediction result output by the initial speaker recognition model; The target loss determination unit is used to determine the target loss between the pseudo-label corresponding to each target sample and the speaker category prediction result corresponding to each target sample, and to optimize the initial speaker recognition model based on the target loss to obtain the target speaker recognition model.
[0048] In some specific embodiments, the target speaker recognition model includes an encoder and a classifier; Accordingly, the speaker recognition module 14 may include: An embedding vector generation unit is used to generate an embedding vector corresponding to the speech to be recognized using the encoder of the target speaker recognition model; The prediction distribution output unit is used to perform speaker recognition based on the embedding vector corresponding to the speech to be recognized using the classifier of the target speaker recognition model, and output the corresponding prediction distribution; the prediction distribution includes several speaker categories and the probability corresponding to each speaker category.
[0049] Furthermore, embodiments of this application also disclose an electronic device, Figure 3 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the speaker recognition method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0050] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0051] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0052] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the speaker recognition method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0053] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned speaker recognition method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0054] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0055] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0056] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0057] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0058] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A speaker recognition method, characterized in that, include: Several target layers are selected from the initial speech representation model, and a target speech representation model is constructed based on the several target layers; The target layer is the network layer in the initial speech representation model that meets the preset performance conditions; Several speech samples are input into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer. The target embedding vectors in the same target layer are clustered to determine the target cluster label corresponding to each target embedding vector based on the clustering results. If the target cluster labels corresponding to any of the above speech are consistent, then the speech is determined as a target sample, so as to obtain a number of target samples selected from each of the above speech; Pseudo-labels are generated for each target sample, and an initial speaker recognition model is trained based on each target sample and its corresponding pseudo-labels to obtain a target speaker recognition model, which is then used to perform speaker recognition on the speech to be recognized.
2. The speaker recognition method according to claim 1, characterized in that, The selection of several target layers in the initial speech representation model includes: Acquire several speaker speech data and input each of the speaker speech data into the initial speech representation model to obtain the first embedding vector corresponding to each of the speaker speech data in each network layer of the initial speech representation model; The performance index of each network layer is determined based on the first embedding vector corresponding to each network layer; the performance index is the same error rate or the correct rate or the minimum detection cost calculated by the minimum detection cost function. If the performance metric of any of the network layers meets the preset performance condition, then the network layer is determined as the target layer.
3. The speaker recognition method according to claim 1, characterized in that, The step of inputting several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer includes: Each of the aforementioned speech samples is input into the target speech representation model to obtain the feature vector corresponding to each of the aforementioned speech samples in each of the aforementioned target layers, and the target embedding vector corresponding to the feature vector is determined by pooling operation or convergence operation.
4. The speaker recognition method according to claim 1, characterized in that, The step of clustering the target embedding vectors of the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results includes: Unsupervised clustering is performed on each of the target embedding vectors in the same target layer to obtain the initial cluster labels corresponding to each of the target embedding vectors in the same layer; The initial cluster labels of different target layers are aligned, and the target cluster label corresponding to each target embedding vector is determined based on the alignment result.
5. The speaker recognition method according to claim 1, characterized in that, Before identifying the speech as the target sample, the method further includes: For any given speech, determine the number of identical target cluster labels. If the number is greater than a preset threshold, then the speech is identified as the target sample.
6. The speaker recognition method according to claim 1, characterized in that, The process of training the initial speaker recognition model based on each target sample and the corresponding pseudo-label includes: Each of the target samples is input into the initial speaker recognition model to obtain the speaker category prediction result output by the initial speaker recognition model; The target loss is determined between the pseudo-label corresponding to each target sample and the speaker category prediction result corresponding to each target sample, and the initial speaker recognition model is optimized based on the target loss to obtain the target speaker recognition model.
7. The speaker recognition method according to any one of claims 1 to 6, characterized in that, The target speaker recognition model includes an encoder and a classifier; Accordingly, the step of using the target speaker recognition model to perform speaker recognition on the speech to be recognized includes: The encoder of the target speaker recognition model is used to generate the embedding vector corresponding to the speech to be recognized; The classifier of the target speaker recognition model performs speaker recognition based on the embedding vector corresponding to the speech to be recognized, and outputs a corresponding prediction distribution; the prediction distribution includes several speaker categories and the probability corresponding to each speaker category.
8. A speaker recognition device, characterized in that, include: The target layer filtering module is used to filter several target layers in the initial speech representation model and construct a target speech representation model based on the several target layers; The target layer is the network layer in the initial speech representation model that meets the preset performance conditions; The target cluster label determination module is used to input several speech samples into the target speech representation model to obtain the target embedding vector corresponding to each speech sample in each target layer, and to cluster the target embedding vectors in the same target layer to determine the target cluster label corresponding to each target embedding vector based on the clustering results. The target sample determination module is used to determine the speech as a target sample if the target cluster labels corresponding to any of the speech are consistent, so as to obtain a number of target samples selected from each of the speech; The speaker recognition module is used to generate pseudo-labels corresponding to each target sample, and to train an initial speaker recognition model based on each target sample and the pseudo-labels corresponding to each target sample to obtain a target speaker recognition model, so as to use the target speaker recognition model to perform speaker recognition on the speech to be recognized.
9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the speaker recognition method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program, which, when executed by a processor, implements the speaker recognition method as described in any one of claims 1 to 7.