Voiceprint recognition model training method and device and voiceprint recognition method and device

Abstract

The invention provides a voiceprint recognition model training method and device and a voiceprint recognition method and device, and the training method comprises the steps: carrying out the acoustic feature extraction of domain voice data of a user side through a feature extraction tool authorized by a technology provider, and obtaining incomplete acoustic features, the feature extraction tool is used for outputting at least one intermediate feature as an incomplete acoustic feature in at least one intermediate link of the complete acoustic feature extraction process; transmitting the incomplete acoustic features to a feature discriminator, complementing the incomplete acoustic features, restoring the incomplete acoustic features to generate complete acoustic features, and deploying the feature discriminator to a user side after the feature discriminator is encrypted and packaged by a technology provider; and training a preset voiceprint recognition network based on the complete acoustic features to obtain a voiceprint recognition model. According to the method, the contradiction between technical security and data privacy protection caused by the fact that a complete training system needs to be deployed at a user side in a traditional method is solved, and the deployment flexibility and security of a model training scheme are greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voiceprint recognition, and in particular to a voiceprint recognition model training method, a voiceprint recognition method and device. BACKGROUND

[0002] At present, in order to improve the effect of the voiceprint recognition model by using the data of a specific field, the historical data is usually mixed with the newly added field data, and a complete training system is deployed on the user side to train the model. However, this method requires the complete training system containing the core algorithm of the technology provider to be deployed on the user side, which not only has a complex deployment process and poor flexibility, but also has the risk of leakage of the core model and key technology of the technology provider, and it is difficult to achieve effective security protection. SUMMARY

[0003] The present application provides a voiceprint recognition model training method, a voiceprint recognition method and device to solve the defects in the prior art.

[0004] The present application provides a voiceprint recognition model training method, which comprises the following steps: The acoustic feature extraction tool authorized by the technology provider is used to extract the acoustic features of the field voice data of the user side, and non-complete acoustic features are obtained. The acoustic feature extraction tool is used to output at least one intermediate feature as non-complete acoustic features in at least one intermediate link of the complete acoustic feature extraction process; The non-complete acoustic features are transmitted to the feature discriminator, and the feature discriminator is used to complete the non-complete acoustic features to restore the complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and packaged by the technology provider and deployed to the user side; The preset voiceprint recognition network is trained based on the complete acoustic features to obtain a voiceprint recognition model.

[0005] According to the voiceprint recognition model training method provided by the present application, the field voice data includes positive example data, negative example data and weakly supervised data. The positive example data includes sample voice data of a target speaker, the negative example data includes sample voice data of a non-target speaker, and the weakly supervised data includes sample voice data with incomplete annotation information and / or doubtful annotation accuracy.

[0006] According to the voiceprint recognition model training method provided by the present application, the acoustic feature extraction tool authorized by the technology provider is used to extract the acoustic features of the field voice data of the user side, and non-complete acoustic features are obtained. Before that, it also includes: The weakly supervised data is preprocessed, and data with a confidence higher than a preset confidence is selected from the weakly supervised data as effective supervised data.

[0007] According to a voiceprint recognition model training method provided by the present invention, the step of preprocessing the weakly supervised data and selecting data with a confidence level higher than a preset confidence level from the weakly supervised data as effective supervised data includes: For each weakly supervised data point under the same speaker, a first voiceprint representation and a second voiceprint representation are extracted respectively. The first voiceprint representation is used to represent the acoustic characteristics of the same speaker under different channel conditions, and the second voiceprint representation is used to represent the acoustic characteristics of the same speaker based on deep neural network learning. The first voiceprint representation and the second voiceprint representation corresponding to each weakly supervised data are fused to obtain the fused voiceprint representation of the corresponding weakly supervised data. Clustering is performed on the fused voiceprint representations corresponding to all weakly supervised data under the same speaker to obtain multiple clusters; Weakly supervised data from the cluster with the largest data volume are retained as valid weakly supervised data for the same speaker.

[0008] According to the voiceprint recognition model training method provided by the present invention, the complete acoustic feature extraction process includes: The speech data is processed sequentially by zero-crossing detection, pre-emphasis, windowing, Fourier transform, modulus extraction, triangular filtering, logarithmic energy, first-order difference, removal of silence frames, and cepstral mean-variance normalization.

[0009] According to the voiceprint recognition model training method provided by the present invention, the incomplete acoustic features include intermediate features output after the first-order difference, and / or intermediate features output after removing silent frames.

[0010] According to the voiceprint recognition model training method provided by the present invention, the step of completing the incomplete acoustic features and restoring them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process includes: Identify the type of the incomplete acoustic feature; Based on the type of the incomplete acoustic feature, the incomplete acoustic feature is completed to generate the complete acoustic feature.

[0011] According to the present invention, a method for training a voiceprint recognition model, wherein training a preset voiceprint recognition network based on the complete acoustic features to obtain a voiceprint recognition model includes: The preset voiceprint recognition network is trained based on the complete acoustic features to obtain a user domain model. The user domain model is sent to the technology provider, so that the technology provider can fuse the user domain model with a general domain model pre-trained by the technology provider to generate and return the voiceprint recognition model.

[0012] According to a voiceprint recognition model training method provided by the present invention, the step of training a preset voiceprint recognition network based on the complete acoustic features to obtain a user domain model includes: The front-end model is obtained by training a pre-defined front-end voiceprint recognition network using the complete acoustic features of weakly supervised data. Using positive and negative examples, the pre-defined backend voiceprint recognition network is trained to obtain the backend model. The user domain model is obtained by combining the front-end model and the back-end model.

[0013] The present invention also provides a voiceprint recognition method, comprising the following steps: Determine the target speech data to be recognized; Using a voiceprint recognition model, the target speech data is subjected to voiceprint recognition to obtain the acoustic features of the target speech data; The voiceprint recognition model is trained based on the voiceprint recognition model training method described in any of the above.

[0014] The present invention also provides a voiceprint recognition model training device, comprising the following modules: An extraction module is used to extract acoustic features from the domain speech data of the user terminal using a feature extraction tool authorized by the technology provider to obtain incomplete acoustic features. The feature extraction tool is used to output at least one intermediate feature as an incomplete acoustic feature at at least one intermediate step in the complete acoustic feature extraction process. The restoration module is used to transmit the incomplete acoustic features to the feature discriminator, which completes the incomplete acoustic features and restores them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and encapsulated by the technology provider and then deployed to the user terminal. The training module is used to train a preset voiceprint recognition network based on the complete acoustic features to obtain a voiceprint recognition model.

[0015] The present invention also provides a voiceprint recognition device, comprising the following modules: The determination module is used to determine the target speech data to be recognized; The recognition module is used to perform voiceprint recognition on the target speech data using a voiceprint recognition model to obtain the acoustic features of the target speech data; The voiceprint recognition model is trained based on the voiceprint recognition model training method described in any of the above.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the voiceprint recognition model training method or voiceprint recognition method as described above.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voiceprint recognition model training method or voiceprint recognition method as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the voiceprint recognition model training method or voiceprint recognition method as described above.

[0019] The voiceprint recognition model training, method, and apparatus provided by this invention employ a two-stage feature processing scheme. This scheme extracts incomplete acoustic features using authorized tools on the user end, then completes them into complete acoustic features using an encrypted feature discriminator, and finally trains the model based on these complete features. This achieves efficient and flexible model training using domain data on the user end while ensuring the security of the technology provider's core technology. Because the core feature completion logic is encrypted and encapsulated within the feature discriminator, and the complete acoustic feature extraction process is broken down, the technology provider's core algorithm is not fully exposed on the user end. This effectively avoids the risk of core technology leakage and resolves the contradiction between technical security and data privacy protection caused by the need to deploy a complete training system on the user end in traditional methods. This significantly improves the deployment flexibility and security of the model training scheme. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the voiceprint recognition model training method provided by the present invention.

[0022] Figure 2 This is a flowchart illustrating the effective supervisory data acquisition method provided by the present invention.

[0023] Figure 3 This is a schematic diagram of the complete acoustic feature extraction process provided by the present invention.

[0024] Figure 4 This is a flowchart illustrating the model fusion method provided by the present invention.

[0025] Figure 5 This is a flowchart illustrating the user domain model training method provided by the present invention.

[0026] Figure 6 This is a flowchart illustrating another voiceprint recognition model training method provided by the present invention.

[0027] Figure 7 This is a flowchart illustrating the voiceprint recognition method provided by the present invention.

[0028] Figure 8 This is a schematic diagram of the structure of the voiceprint recognition model training device provided by the present invention.

[0029] Figure 9 This is a schematic diagram of the voiceprint recognition device provided by the present invention.

[0030] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0032] With the rapid development of deep learning and artificial intelligence, voiceprint recognition technology has been increasingly applied to domain-specific products, providing users with the ability to convert speech into speaker representation information. In practical applications, due to confidentiality and other objective reasons, the domain data of user scenarios can only be deployed on compliant servers at the user end to establish voiceprint recognition training systems. Meanwhile, for technology providers, voiceprint recognition training systems involve core technologies and trade secrets, requiring effective security protection measures. Therefore, a voiceprint recognition system that meets the above objective conditions must possess robust security mechanisms, strong knowledge transfer capabilities, flexible deployment, and controllable access permissions.

[0033] Currently, the method of iteratively training voiceprint recognition models using domain data from user scenarios typically involves mixing historical data with newly added domain data. This approach requires deploying a complete training system containing the technology provider's core algorithms on the user end. This not only involves a complex and inflexible deployment process, but also poses a risk of leakage of the technology provider's core models and key technologies, making effective security protection difficult. Furthermore, it also presents serious problems in areas such as access control.

[0034] To address this issue, this invention provides a voiceprint recognition model training method. The method aims to extract incomplete acoustic features using authorized feature extraction tools on the user end, and then use an encrypted feature discriminator to complete and restore the features. This approach ensures the security of the technology provider's core technology while fully utilizing user-end domain data for model training. It satisfies data security compliance requirements, effectively protects core technologies, and offers advantages such as flexible deployment and controllable permissions. Furthermore, the voiceprint recognition model training method provided by this invention is applied to the user end, which can be understood as a server deploying the voiceprint recognition training system, such as a private server or local data center set up by the user for data security or confidentiality reasons. Additionally, the technology provider of this invention can be understood as an entity that possesses the core algorithms, models, and related technologies of voiceprint recognition; for example, it could be a service provider offering voiceprint recognition solutions.

[0035] in, Figure 1 This is a flowchart illustrating the voiceprint recognition model training method provided by the present invention, as shown below. Figure 1 As shown, the method includes steps 110, 120 and 130.

[0036] Step 110: Use a feature extraction tool authorized by the technology provider to extract acoustic features from the domain speech data of the user terminal to obtain incomplete acoustic features. The feature extraction tool is used to output at least one intermediate feature as an incomplete acoustic feature at at least one intermediate stage in the complete acoustic feature extraction process.

[0037] Specifically, domain-specific voice data can be understood as voice data collected in specific application scenarios, such as voice generated by users (e.g., bank customers, specific personnel) in specific industry applications like finance, security, and the judiciary. Domain-specific voice data typically has a high degree of scenario relevance and confidentiality.

[0038] Acoustic features are key data used to characterize speaker identity information in speech data. A typical complete acoustic feature extraction process is usually public and standardized, and may include a series of steps such as pre-emphasis of speech data, framing, windowing, fast Fourier transform, calculation of the Mel frequency spectrum, and discrete cosine transform, ultimately outputting complete acoustic features. These complete acoustic features can be Mel frequency cepstral coefficients (MFCC) or FBank features; this embodiment does not specifically limit them.

[0039] In this embodiment, to protect the core technology of the technology provider, the complete feature extraction process is not deployed directly on the user's end. Instead, an authorized feature extraction tool is provided. This feature extraction tool can be a software development kit, a command-line tool, or a dynamic link library. Furthermore, authorization here can refer to controlling usage permissions through licenses, dongles, or online authentication. In other words, users need authorization from the technology provider to use the feature extraction tool.

[0040] It should be noted that the feature extraction tool performs an incomplete acoustic feature extraction process. Specifically, the feature extraction tool interrupts and outputs a result at at least one intermediate step in the complete acoustic feature extraction process; this output is the incomplete acoustic feature. Because this feature is an intermediate product, it usually does not contain enough information for a high-quality speaker recognition model to be directly trained, and it also hides the core technical steps that the technology provider will subsequently use to optimize the feature.

[0041] The incomplete acoustic features can be Mel-spectral features or spectral features after fast Fourier transform; this embodiment does not specifically limit them.

[0042] This embodiment breaks down the complete acoustic feature extraction process and executes only a portion of it on the user's end. Users can generate intermediate acoustic features that do not contain all of the technology provider's core technologies using their private domain speech data locally. This not only ensures the security and privacy of the user's domain data but also guarantees the security of the technology provider's core algorithms.

[0043] Step 120: Transmit the incomplete acoustic features to the feature discriminator, which completes the incomplete acoustic features and restores them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and packaged by the technology provider and then deployed to the user terminal.

[0044] Specifically, the feature discriminator can be understood as a model or module pre-trained by the technology provider to achieve a specific transformation function. The core function of the feature discriminator is to complete the incomplete acoustic features generated in step 110 to restore the complete acoustic features. For example, if the incomplete acoustic features are Mel-spectral features, the feature discriminator can perform functions similar to discrete cosine transform and other possible subsequent optimization transforms, ultimately outputting complete acoustic features that are completely consistent with those produced by the technology provider's internal standard process.

[0045] As an alternative embodiment, the feature discriminator can be a deep neural network model, a component of a generative adversarial network, or an optimized transformation matrix. Internally, the feature discriminator incorporates the technology provider's core technologies and knowledge regarding how to generate high-quality, final, complete acoustic features from intermediate, incomplete acoustic features.

[0046] To protect the core technology of the technology provider, the feature discriminator is encrypted and encapsulated by the technology provider before being deployed to the user end. Encryption and encapsulation can be understood as a combination of one or more security technologies. For example, the model file of the feature discriminator itself can be encrypted, making it impossible to read and parse directly; the feature discriminator can also be obfuscated or compiled into a binary executable file, making its internal structure and parameters invisible; it can also be packaged in a secure container and its runtime environment and access permissions restricted to prevent reverse engineering or unauthorized calls.

[0047] Because the feature discriminator is encrypted and encapsulated as a "black box" and deployed on the user's end, users can only call it to perform feature completion functions, but cannot explore its internal implementation principles and algorithm details. In this way, the data processing flow is still completed within the user's end, ensuring the privacy of domain data, while the core algorithms of the technology provider are also effectively protected.

[0048] Step 130: Train the preset voiceprint recognition network based on complete acoustic features to obtain a voiceprint recognition model.

[0049] After processing in step 120, high-quality, complete acoustic features consistent with the technology provider's standards have been obtained. At this point, these features can be used to train a pre-defined voiceprint recognition network.

[0050] The preset voiceprint recognition network can be a basic model that has been pre-trained by the technology provider using massive amounts of general data, or it can be a brand-new model that is randomly initialized and conforms to a specific network architecture.

[0051] The voiceprint recognition model training method provided in this embodiment employs a two-stage feature processing approach. This approach extracts incomplete acoustic features using authorized tools on the user end, then completes them into complete acoustic features using an encrypted feature discriminator, and finally trains the model based on these complete features. This achieves efficient and flexible model training using domain data from the user end while ensuring the security of the technology provider's core technology. Because the core feature completion logic is encrypted and encapsulated within the feature discriminator, and the complete acoustic feature extraction process is broken down, the technology provider's core algorithm is not fully exposed on the user end. This effectively avoids the risk of core technology leakage and resolves the contradiction between technical security and data privacy protection caused by the need to deploy a complete training system on the user end in traditional methods. This significantly improves the deployment flexibility and security of the model training solution.

[0052] Based on the above embodiments, the domain speech data includes positive example data, negative example data, and weakly supervised data. Positive example data includes sample speech data of the target speaker, negative example data includes sample speech data of non-target speakers, and weakly supervised data includes sample speech data with incomplete annotation information and / or questionable annotation accuracy.

[0053] Specifically, positive example data can be understood as sample speech data that clearly belongs to the target speaker. Here, the target speaker is the specific individual that the voiceprint recognition model needs to learn and identify. For example, when building a voice unlocking model for a specific user's mobile phone, the multiple everyday conversational speech clips recorded by that user are positive example data. The role of positive example data is to provide the model with an accurate depiction of the target speaker's acoustic features.

[0054] Counterexample data can be understood as sample speech data that clearly belongs to a non-target speaker. Here, a non-target speaker can be any individual other than the target speaker. For example, in the above scenario of unlocking a mobile phone by voice, speech data recorded by other people would be counterexample data. The purpose of counterexample data is to improve the model's discriminative ability. By learning the differences between the target speaker's voice and that of others, the model can effectively resist impersonation attacks.

[0055] Weakly supervised data can be understood as sample speech data with incomplete annotation information and / or questionable annotation accuracy. In practical applications, obtaining a large amount of accurately labeled positive and negative examples is costly and time-consuming, while weakly supervised data is usually widely available and abundant, and utilizing it can significantly improve the generalization ability of the model.

[0056] As an alternative embodiment, data with incomplete annotation information can be long audio segments containing the voices of multiple speakers, such as a conference recording or a multi-person call recording. For this type of data, it may be known that the participants include the target speaker, but it is not possible to pinpoint exactly who said each sentence.

[0057] As an alternative implementation, data whose labeling accuracy is questionable may be initially labeled using automated tools or labeled by non-professionals, and its labels may contain a certain percentage of errors.

[0058] In the subsequent model training step 130, different training strategies can be designed for these three types of data. For example, positive and negative examples can be used for comparative learning, so that the features output by the model in the vector space are close to each other for features belonging to the same speaker and far apart for features belonging to different speakers. At the same time, a large amount of weakly supervised data can be used for semi-supervised or self-supervised learning, allowing the model to learn more robust and universal acoustic representations that are not affected by the specific speaker's identity, thereby enhancing the model's adaptability to noise, accents, and channel variations.

[0059] This embodiment refines domain-specific speech data into positive examples, negative examples, and weakly supervised data, and combines this with corresponding training strategies, enabling the model training process to fully utilize various types and qualities of data from the real world. By utilizing not only high-quality labeled data to ensure the model's accuracy in distinguishing between different data types, but also a large amount of weakly supervised data to improve the model's generalization ability and robustness, the cost and reliance on fine-grained data annotation are significantly reduced while maintaining model performance.

[0060] Based on the above embodiments, in order to further improve the utilization efficiency and value of weakly supervised data and avoid the negative impact on model training caused by the introduction of poorly labeled data, this embodiment further optimizes the process of utilizing weakly supervised data.

[0061] Specifically, acoustic features are extracted from the user's domain-specific speech data using feature extraction tools authorized by the technology provider, resulting in incomplete acoustic features. This previously included: Weakly supervised data is preprocessed, and data with confidence levels higher than the preset confidence level are selected as effective supervised data.

[0062] Specifically, preprocessing can be understood as a series of automated cleaning and filtering operations performed on the raw weakly supervised data during the data preparation stage of formal model training. Its core purpose is to automatically extract valuable and highly credible training samples from the massive amount of unlabeled weakly supervised data.

[0063] The confidence level here can be understood as a quantitative indicator that measures how likely the annotation information of weakly supervised data is correct, such as measuring the probability that the speech belongs to the target speaker.

[0064] As an alternative implementation, a pre-trained auxiliary voiceprint recognition model can be used to calculate the confidence score. Specifically, each piece of speech data from the weakly supervised data is input into the auxiliary model to extract its voiceprint feature vector; then, the similarity is compared with the known voiceprint feature vector of the target speaker, and the similarity score can be used as the confidence score that the speech belongs to the target speaker.

[0065] As an alternative implementation, for long multi-person speech segments labeled "containing the target speaker," they can first be segmented using speaker log technology to obtain multiple speech segments belonging to different speakers. In this case, the confidence level can be the probability or score of assigning a segment to the cluster representing the target speaker.

[0066] Furthermore, the preset reliability is an adjustable threshold, such as 0.8 or 0.9. This threshold represents a trade-off between data volume and data quality. A higher threshold results in higher quality, more accurately labeled data, but reduces the amount of usable data; a lower threshold increases the amount of usable data but may introduce some mislabeled noise. In practice, this threshold can be set empirically or dynamically optimized by testing model performance at different thresholds on the validation set.

[0067] After screening, data with a confidence level higher than the preset confidence level are considered valid supervised data. For example, a weakly supervised data point with a calculated confidence level of 0.95 for the target speaker, which is higher than the preset threshold of 0.9, is selected as valid supervised data and can be used as a new positive example in subsequent training. Conversely, if its confidence level is only 0.6, it is discarded or not included in this round of supervised training.

[0068] This embodiment achieves automated and low-cost efficient utilization of massive amounts of low-quality weakly supervised data by adding a preprocessing step to the weakly supervised data before formal training and using a confidence screening mechanism to purify the data. Because this step, performed before formal feature extraction and training, removes a large amount of inaccurately labeled or irrelevant noise data at a low computational cost, it significantly expands the scale of the high-quality training dataset without increasing any manual annotation costs, further improving the accuracy and robustness of the final voiceprint recognition model.

[0069] Based on any of the above embodiments, preprocessing of weakly supervised data, and filtering data with confidence levels higher than a preset confidence level from the weakly supervised data as effective supervised data, includes: For each weakly supervised data point under the same speaker, a first voiceprint representation and a second voiceprint representation are extracted respectively. The first voiceprint representation is used to represent the acoustic characteristics of the same speaker under different channel conditions, and the second voiceprint representation is used to represent the acoustic characteristics of the same speaker based on deep neural network learning. The first and second voiceprint representations corresponding to each weakly supervised data are fused to obtain the fused voiceprint representation of the corresponding weakly supervised data. Clustering is performed on the fused voiceprint representations corresponding to all weakly supervised data under the same speaker to obtain multiple clusters; Weakly supervised data from the cluster with the largest amount of data are retained as valid weakly supervised data for the same speaker.

[0070] Considering that weakly supervised data may contain mislabeled speech from others or speech from the target speaker under extreme noise / channel conditions, this embodiment jointly characterizes each weakly supervised speech from both the channel characteristics dimension and the deep identity dimension to more accurately identify them.

[0071] Specifically, for each weakly supervised data point from the same speaker, a first voiceprint representation and a second voiceprint representation are extracted. The core of the first voiceprint representation lies in capturing the conversation / channel variability in the speech data.

[0072] The core of the second voiceprint representation lies in using deep learning networks to extract more discriminative and in-depth speaker identity information. As an optional embodiment, this second voiceprint representation can be extracted by a deep neural network trained on massive amounts of data. Because deep neural networks can learn to decouple the essential features most relevant to the speaker's identity from mixed speech signals through multi-layer nonlinear transformations, the extracted second voiceprint representation can focus more on capturing the speaker's essential and stable acoustic characteristics, and is more robust to short-term noise and channel variations.

[0073] It should be noted that by extracting the first and second voiceprint representations, this embodiment can provide a more comprehensive and robust description of the speech samples from two complementary dimensions: channel and identity. This provides more discriminative feature inputs for subsequent clustering analysis and improves the accuracy of the screening.

[0074] After extracting the first and second voiceprint representations for each weakly supervised data point from the same speaker, the first and second voiceprint representations are fused to obtain the fused voiceprint representation for each weakly supervised data point. As an optional embodiment, the first and second voiceprint representations can be concatenated to obtain the fused voiceprint features for each weakly supervised data point. Alternatively, the first and second voiceprint representations can be weighted and summed to obtain the fused voiceprint features for each weakly supervised data point.

[0075] After obtaining the fused voiceprint representations of all weakly supervised data under the same speaker, a clustering algorithm is used to group these fused voiceprint representations together in the feature space, so as to aggregate samples with similar acoustic characteristics together and separate out anomalous samples with large differences in acoustic characteristics.

[0076] As an optional implementation, algorithms such as hierarchical clustering, DBSCAN, or K-Means can be used to cluster all fused voiceprint representations of the same speaker, resulting in multiple clusters. Among these, the speech data of samples belonging to the real target speaker and recorded under normal conditions will have highly similar features, thus forming one or several large, dense clusters. For mislabeled speech of others, or speech under extreme channel conditions, their features will differ significantly from the mainstream samples, thus forming some small clusters or becoming outliers.

[0077] Finally, the cluster with the largest amount of data is most likely to represent the speaker's true and mainstream acoustic characteristics. Therefore, in this embodiment, the weak supervision data in the cluster with the largest amount of data is retained as the effective weak supervision data for the same speaker. Data in all other smaller clusters are considered noise or mislabeled and are discarded.

[0078] in, Figure 2 This is a flowchart illustrating the effective supervisory data acquisition method provided by the present invention, as shown below. Figure 2 As shown, firstly, regarding those initially attributed to the same speaker (such as...) Figure 2 The weakly supervised data under “ID-1” in the dataset are extracted in parallel from two dimensions. On the one hand, the first voiceprint representation is extracted through the TV voiceprint system, which is used to capture the changes in acoustic characteristics caused by different channel conditions. On the other hand, the second voiceprint representation is extracted through the NN voiceprint system, which is used to capture more discriminative and deeper speaker identity information based on deep neural network learning.

[0079] Then, the first and second voiceprint representations extracted from the same weakly supervised data in the above steps are fused to obtain a fused voiceprint representation containing both channel and identity information.

[0080] Next, for the fused voiceprint representations corresponding to all weakly supervised data under this speaker (ID-1), a hierarchical clustering algorithm is used for cluster analysis to obtain multiple clusters.

[0081] Finally, the weak supervision data corresponding to the cluster with the largest data volume is retained as the effective weak supervision data for that speaker, while other smaller clusters are discarded as noise or mislabeled. The effective weak supervision data for other speakers (such as "ID-N") is obtained using the same method as ID-1.

[0082] Based on any of the above embodiments Figure 3 This is a schematic diagram of the complete acoustic feature extraction process provided by the present invention, as shown below. Figure 3 As shown, the complete acoustic feature extraction process includes: The speech data is processed sequentially by zero-crossing detection, pre-emphasis, windowing, Fourier transform, modulus extraction, triangular filtering, logarithmic energy, first-order difference, removal of silence frames, and cepstral mean-variance normalization.

[0083] Specifically, zero-crossing detection is a preliminary analysis of speech by counting the number of times the speech signal waveform crosses the horizontal axis (zero level). It can be used as an auxiliary feature to distinguish between voiceless and voiced sounds.

[0084] Pre-emphasis is achieved by processing the original speech signal with a high-pass filter to enhance the energy of the high-frequency components of the speech signal, making the spectrum more balanced.

[0085] Before performing a Fourier transform, the speech signal is typically segmented into a series of short, stationary frames. Windowing operations (such as Hamming or Heining windows) aim to smoothly decay the two ends of each frame to zero, reducing spectral leakage effects from the subsequent Fourier transform and thus obtaining a more accurate spectral estimate. The Fourier transform converts the time-domain speech signal frames to the frequency domain, obtaining the energy and phase information of each frequency component.

[0086] The result of the Fourier transform is a complex number. Since the human ear is not sensitive to the phase information of speech, but is more sensitive to the energy distribution of frequency, the modulus of the Fourier transform result is taken to obtain the energy spectrum of the signal, and the phase information is discarded.

[0087] Human hearing perceives frequencies non-linearly, with higher resolution in the low-frequency region than in the high-frequency region. By filtering the energy spectrum obtained in the previous step using a set of triangular filters arranged according to the Mel scale—a process known as triangular filtering—it is possible to simulate the auditory characteristics of the human ear, mapping the linear frequency spectrum onto the non-linear Mel spectrum to obtain characteristics that better reflect human auditory perception.

[0088] The human ear's perception of sound intensity is also logarithmic. Taking the logarithm of the energy of the Mel spectrum (i.e., logarithmic energy) can compress the dynamic range of the features and make the feature distribution more consistent with the human auditory perception model.

[0089] The acoustic features obtained after logarithmic energy are static, describing only the information of the current frame. Speech is dynamic, and its changing trends themselves contain important speaker information. By using first-order differencing—that is, calculating the difference between feature vectors between adjacent frames—difference features can be obtained, capturing the dynamic information of acoustic features changing over time and enhancing the representational power of the features.

[0090] Speech often contains silent or noisy segments that lack meaningful speaker information and can interfere with model training. Identifying and removing these silent frames ensures that only valid speech data is used for training.

[0091] Different recording equipment, channels, and environments can introduce convolutional noise into speech signals, causing shifts in acoustic features. Cepstral mean-variance normalization (CMVN) calculates the mean and variance of all feature vectors within a utterance, then subtracts the mean from each feature vector and divides by the variance. This eliminates linear distortions caused by factors such as channel conditions and enhances the robustness of features to different recording conditions.

[0092] In one specific embodiment, the incomplete acoustic feature can be an intermediate feature output after performing the "first-order difference" step. This intermediate feature already contains both static and dynamic information of the speech, possessing relatively rich basic acoustic information. However, this feature has not yet undergone silence frame removal and cepstral mean-variance normalization processing. In this case, the feature discriminator encapsulates the technology provider's high-performance speech activity detection algorithm and core cepstral mean-variance normalization algorithm. The technology provider can securely deploy its proprietary, accurate judgment capability for speech segments in complex noise environments, along with its efficient channel compensation technology, at the user end.

[0093] In another specific embodiment, the incomplete acoustic features can be intermediate features output after the "silence frame removal" step. At this point, the intermediate features output by the feature extraction tool are already "pure," meaning they only contain features of valid speech segments, but their feature distribution may still be affected by different recording devices and channels, and have not yet undergone final normalization. In this case, the core function of the feature discriminator focuses on performing efficient cepstral mean-variance normalization (CMVN) operations, encapsulating the technology provider's core technologies regarding channel robustness.

[0094] It is important to understand that technology providers can choose to output one or more intermediate features, depending on the focus of the technology they wish to protect and the flexibility of deployment. For example, a feature extraction tool can simultaneously output features after first-order differencing and features after removing silent frames, for subsequent feature discriminators to perform more complex fusion and processing.

[0095] Based on any of the above embodiments, incomplete acoustic features are supplemented to restore and generate complete acoustic features consistent with the output of the complete acoustic feature extraction process, including: Identify the types of incomplete acoustic features; Based on the type of incomplete acoustic features, the incomplete acoustic features are completed to generate complete acoustic features.

[0096] Considering that incomplete acoustic features output by feature extraction tools can be products of different intermediate stages in the complete acoustic feature extraction process, and their degree of incompleteness is variable, it is necessary to consider the type of incomplete acoustic features when completing them. The type of incomplete acoustic feature refers to the specific stage or state of the feature in the complete acoustic feature extraction process.

[0097] The type of incomplete acoustic feature can be identified by reading the metadata attached to it, such as the type identifier, or by analyzing the properties of the incomplete acoustic feature itself, such as feature dimension, numerical range, or statistical distribution.

[0098] After identifying the type of incomplete acoustic feature, the feature discriminator determines the subsequent completion processing path based on the type of the incomplete acoustic feature and completes the incomplete acoustic feature. For example, if the type of incomplete acoustic feature is identified as Mel spectrum, the feature discriminator calls its internally encapsulated processing path, which includes a series of operations such as "logarithmic energy, first-order difference, removal of silence frames, and cepstral mean-variance normalization," to process the incomplete acoustic feature to generate a complete acoustic feature. If the type of incomplete acoustic feature is identified as an intermediate feature output after removing silence frames, the feature discriminator only needs to call its internally encapsulated "cepstral mean-variance normalization" processing module to complete the completion.

[0099] Based on any of the above embodiments, a preset voiceprint recognition network is trained based on complete acoustic features to obtain a voiceprint recognition model, including: The user domain model is obtained by training a pre-defined voiceprint recognition network based on complete acoustic features. The user domain model is sent to the technology provider, who then fuses it with a general domain model pre-trained by the technology provider to generate and return a voiceprint recognition model.

[0100] In the first stage, on the user end, the preset voiceprint recognition network is trained based on the complete acoustic features generated in step 120 to obtain a user domain model. This user domain model is a model that has been deeply optimized on the user's private domain data. It fully learns the acoustic environment, speaker distribution, and other characteristics of the user's specific scenario, and therefore has high performance in this specific domain.

[0101] It should be noted that this training process is conducted entirely within a compliant server on the user's end. The user's domain-specific speech data and the complete acoustic features it generates never leave the user's local machine, thereby maximizing the protection of the user's data security and privacy.

[0102] In the second phase, the user sends the trained user domain model (i.e., the model's network structure and weight parameters, not the user's raw data) to the technology provider's server. The technology provider then merges the received user domain model with a pre-trained general domain model on its server. This general domain model is trained using massive, multi-scenario, and diverse datasets (such as open-source data) covering different genders, ages, and accents. This model possesses strong generalization ability and robustness, but its ability to fine-tune the recognition of a specific domain may not be as good as that of the specially trained user domain model.

[0103] The fusion aims to combine the generalization ability of a general domain model with the specificity ability of a user domain model to obtain a comprehensive voiceprint recognition model.

[0104] As an optional implementation, the fusion method can be a model-weighted average. For example, let the parameters of the final voiceprint recognition model M_final be W_final, the parameters of the user domain model be W_user, and the parameters of the general domain model be W_univ, then the fusion process can be expressed as: W_final = α × W_user + (1-α) × W_univ, where α is a fusion coefficient between 0 and 1, used to adjust the weights of the two models. The value of α can be set empirically or dynamically determined based on the model's performance on the validation set. After fusion, the technology provider encrypts the generated final, higher-performing voiceprint recognition model and returns it to the user for deployment and use.

[0105] This embodiment achieves model performance optimization under the premise of absolute user data security through a collaborative training workflow of "local training and cloud fusion". Since users only need to upload the model parameters trained on local data, rather than sensitive raw speech data, the issue of data privacy compliance is perfectly solved. At the same time, by performing the fusion operation in the cloud, the technology provider not only protects its core general domain model from being leaked, but also endows the user's dedicated model with the powerful generalization ability of the general model. This effectively avoids the overfitting or "catastrophic forgetting" problems that may occur in the user's domain model due to the limited amount of domain data. In the end, it provides users with a voiceprint recognition model with high domain adaptability and high generalization robustness, and optimal performance.

[0106] in, Figure 4 This is a flowchart illustrating the model fusion method provided by the present invention, as shown below. Figure 4 As shown, in the left-hand flow, the input speech is processed using a user domain model, which consists of a Wav2vec pre-trained front-end and a Transformer-TDNN back-end connected in series, ultimately outputting user domain voiceprint features adapted to the user's specific scenario. In the right-hand flow, the same input speech is processed in parallel using a general domain model with the same or similar structure, outputting general domain voiceprint features.

[0107] Then, the user domain voiceprint features and general domain voiceprint features extracted from the two parallel processes are fused to obtain the final voiceprint features. Compared with the features extracted by any single model, the voiceprint features have stronger representation ability and robustness, and can be directly used for subsequent voiceprint comparison or recognition tasks to obtain higher accuracy.

[0108] Based on any of the above embodiments, a user domain model is obtained by training a preset voiceprint recognition network based on complete acoustic features, including: The front-end model is obtained by training a pre-defined front-end voiceprint recognition network using the complete acoustic features of weakly supervised data. Using positive and negative examples, the pre-defined backend voiceprint recognition network is trained to obtain the backend model. By combining the front-end model and the back-end model, we obtain the user domain model.

[0109] Given that different types of data vary significantly in size, annotation quality, and the effective information they contain, different training strategies are used for weakly supervised data, positive examples, and negative examples.

[0110] Specifically, in the first stage, the goal is to train a front-end model capable of extracting general and robust acoustic representations using a large amount of weakly supervised data with low annotation quality.

[0111] Among them, the front-end voiceprint recognition network can be understood as the main part of the complete voiceprint recognition network responsible for feature extraction and encoding, such as a deep neural network backbone composed of multiple convolutional layers or self-attention layers.

[0112] Because weakly supervised data is vast in quantity but may contain inaccurate or incomplete labels, it is unsuitable for direct training of classification tasks requiring precise labels. However, weakly supervised data contains rich information about the acoustic environment, channel variations, and accents. By training on weakly supervised data using self-supervised or semi-supervised learning methods, the front-end model can learn how to extract general acoustic representations from complex speech signals that are insensitive to speaker identity but robust to environmental changes.

[0113] The second phase aims to train a backend model with strong discriminative capabilities using precisely labeled positive and negative examples.

[0114] In this context, the backend voiceprint recognition network can be understood as the classification head or metric learning head part of the complete voiceprint recognition network, responsible for aggregating the features extracted from the frontend and mapping them to the final voiceprint embedding vector space. For example, the backend voiceprint recognition network may include statistical pooling layers and several fully connected layers.

[0115] Because positive and negative examples have clear speaker labels, they are suitable for supervised learning. For example, metric learning training can be performed using contrastive loss, triplet loss, or various angle-based loss functions. The training objective is to maximize the distance between different speaker voiceprints in the voiceprint embedding space, while minimizing the distance between voiceprints of the same speaker, so that the backend model can focus on learning how to distinguish between different people.

[0116] Finally, the front-end and back-end models are combined to form a complete end-to-end network architecture, resulting in the user domain model. As an alternative implementation, the front-end model can be trained or a pre-trained model can be loaded separately first. Then, the parameters of the front-end model are fixed, and the back-end model is trained using only positive and negative examples. Finally, the front-end and back-end models are fine-tuned end-to-end as a whole with a small learning rate to obtain the user domain model.

[0117] This embodiment achieves maximum and optimal utilization of data from different types and of varying quality domains through a phased and differentiated training strategy. The front-end model learns robust general acoustic representations using massive amounts of weakly supervised data, effectively improving its adaptability to complex environments. Meanwhile, the back-end model focuses on enhancing its discriminative ability using high-quality labeled data. This effectively resolves the contradiction between the scarcity of well-labeled data and the abundance of poorly labeled data in the domain dataset, resulting in a final user domain model that possesses both excellent robustness and accurate recognition capabilities.

[0118] in, Figure 5 This is a flowchart illustrating the user domain model training method provided by the present invention, as shown below. Figure 5 As shown, firstly, incomplete acoustic features 1 and incomplete acoustic features 2 are generated by the feature extraction tool authorized by the technology provider. These incomplete acoustic features are input into the encrypted feature discriminator. The feature discriminator identifies and completes the different types of incomplete acoustic features received, and restores and generates complete acoustic features consistent with the output of the complete acoustic feature extraction process.

[0119] Then, based on the generated complete acoustic features, a pre-defined voiceprint recognition network is trained. Specifically, this network can be composed of a "Wav2vec pre-trained front-end" as the front-end voiceprint recognition network and a "Transformer-TDNN back-end" as the back-end voiceprint recognition network. The training process can utilize weakly supervised data to train the Wav2vec pre-trained front-end and utilize positive and negative examples to train the Transformer-TDNN back-end, thus obtaining a user domain model.

[0120] It should be noted that, to ensure the security of the technology provider's core technologies, the entire training process deployed on the user end, including the feature discriminator, model network structure, and related training logic, can be encapsulated and protected through code encryption. After generating the final user-domain model, model encryption can also be selected.

[0121] Figure 6 This is a flowchart illustrating another voiceprint recognition model training method provided by the present invention, as shown below. Figure 6 As shown, firstly, on the user side, using a feature extraction tool authorized by the "feature extraction engine", acoustic features are extracted from the user's domain speech data (positive example data, negative example data, and weakly supervised data) to obtain one or more types of incomplete acoustic features, such as feature 1 and feature 2.

[0122] Then, these incomplete acoustic features are input into a training tool that integrates a feature discriminator. This tool automatically identifies the type of the input incomplete acoustic features and completes them into complete acoustic features suitable for training. This completes the training of a pre-defined voiceprint recognition network, ultimately generating a user domain model on the user's end. Specifically, feature 1 is input into training tool 1 to obtain the corresponding complete acoustic feature, and feature 2 is input into training tool 2 to obtain the corresponding complete acoustic feature.

[0123] Next, the user domain model trained on the user's end is sent to the technology provider. The technology provider then fuses the received user domain model with its pre-trained general domain model to generate a voiceprint recognition model.

[0124] Finally, the technology provider packages and encrypts the voiceprint recognition model to protect the core technology and model parameters. The encrypted model is then returned to the user's end, serving as the core of the voiceprint recognition engine and providing high-precision, robust recognition capabilities for upper-layer voiceprint recognition products.

[0125] Based on any of the above embodiments Figure 7 This is a flowchart illustrating the voiceprint recognition method provided by the present invention. This method is applied to the user terminal, such as... Figure 7 As shown, the method includes: Step 710: Determine the target speech data to be recognized; Step 720: Using a voiceprint recognition model, perform voiceprint recognition on the target speech data to obtain the acoustic features of the target speech data; The voiceprint recognition model is trained based on the voiceprint recognition model training method described in any of the above embodiments.

[0126] In this embodiment, the voiceprint recognition model trained according to the above embodiments is deployed on the user end. That is, the final generated, encrypted, and packaged voiceprint recognition model is installed and run on an application server or terminal device that provides specific voiceprint recognition functionality, such as the backend server of a financial app, a smart door lock, a smart vehicle system, or a personal mobile phone. Upon receiving target speech data, the trained voiceprint recognition model is used to perform voiceprint recognition on the target speech data. This converts the target speech data into high-dimensional, fixed-length voiceprint features that stably represent the speaker's identity information. Further identity verification or identification tasks are then performed based on these voiceprint features.

[0127] Specifically, in step 710, the target voice data can be a voice segment input by the user in real time through a microphone, or it can be a pre-recorded audio file that needs to be analyzed. This embodiment does not make any specific limitations on this.

[0128] In step 720, the process of using a voiceprint recognition model to perform voiceprint recognition on the target speech data refers to inputting the target speech data into the deployed voiceprint recognition model for forward calculation. The model will eventually output acoustic features that represent the identity of the speaker, i.e., voiceprint features.

[0129] Once the voiceprint feature vector is obtained, it can support various downstream applications, such as comparing the extracted voiceprint features of the target speech data with the voiceprint features of a pre-registered, known target speaker. If the similarity score is higher than a preset threshold, it is determined to be the same person, and the identity verification is successful; otherwise, it fails.

[0130] The voiceprint recognition model training device provided by the present invention is described below. The voiceprint recognition model training device described below and the voiceprint recognition model training method described above can be referred to in correspondence.

[0131] Based on any of the above embodiments Figure 8 This is a schematic diagram of the voiceprint recognition model training device provided by the present invention. The device is deployed at the user end and includes: The extraction module 810 is used to extract acoustic features from the domain speech data of the user terminal using a feature extraction tool authorized by the technology provider to obtain incomplete acoustic features. The feature extraction tool is used to output at least one intermediate feature as an incomplete acoustic feature at at least one intermediate step in the complete acoustic feature extraction process. The restoration module 820 is used to transmit incomplete acoustic features to the feature discriminator, which completes the incomplete acoustic features and restores them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and encapsulated by the technology provider and then deployed to the user end. Training module 830 is used to train a preset voiceprint recognition network based on complete acoustic features to obtain a voiceprint recognition model.

[0132] Based on any of the above embodiments Figure 9 This is a schematic diagram of the voiceprint recognition device provided by the present invention. The device is deployed at the user end and includes: The determination module 910 is used to determine the target speech data to be recognized; The recognition module 920 is used to perform voiceprint recognition on the target speech data using a voiceprint recognition model to obtain the acoustic features of the target speech data. The voiceprint recognition model is trained based on the voiceprint recognition model training method described in any of the above embodiments.

[0133] Figure 10 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 10As shown, the electronic device may include a processor 1010, a communications interface 1020, a memory 1030, and a communication bus 1040. The processor 1010, communications interface 1020, and memory 1030 communicate with each other via the communication bus 1040. The processor 1010 can call logical instructions from the memory 1030 to execute a voiceprint recognition model training method or a voiceprint recognition method.

[0134] Furthermore, the logical instructions in the aforementioned memory 1030 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0135] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the voiceprint recognition model training method or voiceprint recognition method provided by the above methods.

[0136] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the voiceprint recognition model training method or voiceprint recognition method provided by the above methods.

[0137] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0138] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a voiceprint recognition model, characterized in that, Applied to the user end, including: Acoustic features are extracted from the domain speech data of the user terminal using a feature extraction tool authorized by the technology provider to obtain incomplete acoustic features. The feature extraction tool is used to output at least one intermediate feature as an incomplete acoustic feature at at least one intermediate step in the complete acoustic feature extraction process. The incomplete acoustic features are transmitted to the feature discriminator, which completes the incomplete acoustic features and restores them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and encapsulated by the technology provider and then deployed to the user terminal. The preset voiceprint recognition network is trained based on the complete acoustic features to obtain a voiceprint recognition model.

2. The voiceprint recognition model training method according to claim 1, characterized in that, The domain speech data includes positive example data, negative example data, and weakly supervised data. The positive example data includes sample speech data of the target speaker, the negative example data includes sample speech data of non-target speakers, and the weakly supervised data includes sample speech data with incomplete annotation information and / or questionable annotation accuracy.

3. The method according to claim 2, characterized in that, The step of using a feature extraction tool authorized by the technology provider to extract acoustic features from the domain speech data of the user terminal to obtain incomplete acoustic features also includes: The weakly supervised data is preprocessed, and data with a confidence level higher than a preset confidence level are selected from the weakly supervised data as effective supervised data.

4. The voiceprint recognition model training method according to claim 3, characterized in that, The preprocessing of the weakly supervised data, and the selection of data with a confidence level higher than a preset confidence level as effective supervised data, includes: For each weakly supervised data point under the same speaker, a first voiceprint representation and a second voiceprint representation are extracted respectively. The first voiceprint representation is used to represent the acoustic characteristics of the same speaker under different channel conditions, and the second voiceprint representation is used to represent the acoustic characteristics of the same speaker based on deep neural network learning. The first voiceprint representation and the second voiceprint representation corresponding to each weakly supervised data are fused to obtain the fused voiceprint representation of the corresponding weakly supervised data. Clustering is performed on the fused voiceprint representations corresponding to all weakly supervised data under the same speaker to obtain multiple clusters; Weakly supervised data from the cluster with the largest data volume are retained as valid weakly supervised data for the same speaker.

5. The voiceprint recognition model training method according to any one of claims 1 to 4, characterized in that, The complete acoustic feature extraction process includes: The speech data is processed sequentially by zero-crossing detection, pre-emphasis, windowing, Fourier transform, modulus extraction, triangular filtering, logarithmic energy, first-order difference, removal of silence frames, and cepstral mean-variance normalization.

6. The voiceprint recognition model training method according to claim 5, characterized in that, The incomplete acoustic features include intermediate features output after the first-order difference, and / or intermediate features output after removing the silence frames.

7. The voiceprint recognition model training method according to any one of claims 1 to 4, characterized in that, The step of completing the incomplete acoustic features and restoring them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process includes: Identify the type of the incomplete acoustic feature; Based on the type of the incomplete acoustic feature, the incomplete acoustic feature is completed to generate the complete acoustic feature.

8. The voiceprint recognition model training method according to any one of claims 1 to 4, characterized in that, The process of training a preset voiceprint recognition network based on the complete acoustic features to obtain a voiceprint recognition model includes: The preset voiceprint recognition network is trained based on the complete acoustic features to obtain a user domain model. The user domain model is sent to the technology provider, so that the technology provider can fuse the user domain model with a general domain model pre-trained by the technology provider to generate and return the voiceprint recognition model.

9. The voiceprint recognition model training method according to claim 8, characterized in that, The process of training a preset voiceprint recognition network based on the complete acoustic features to obtain a user domain model includes: The front-end model is obtained by training a pre-defined front-end voiceprint recognition network using the complete acoustic features of weakly supervised data. Using positive and negative examples, the pre-defined backend voiceprint recognition network is trained to obtain the backend model. The user domain model is obtained by combining the front-end model and the back-end model.

10. A voiceprint recognition method, characterized in that, Applied to the user end, including: Determine the target speech data to be recognized; Using a voiceprint recognition model, the target speech data is subjected to voiceprint recognition to obtain the acoustic features of the target speech data; The voiceprint recognition model is trained based on the voiceprint recognition model training method according to any one of claims 1 to 9.

11. A voiceprint recognition model training device, characterized in that, Deployed on the user end, including: An extraction module is used to extract acoustic features from the domain speech data of the user terminal using a feature extraction tool authorized by the technology provider to obtain incomplete acoustic features. The feature extraction tool is used to output at least one intermediate feature as an incomplete acoustic feature at at least one intermediate step in the complete acoustic feature extraction process. The restoration module is used to transmit the incomplete acoustic features to the feature discriminator, which completes the incomplete acoustic features and restores them to generate complete acoustic features consistent with the output of the complete acoustic feature extraction process. The feature discriminator is encrypted and encapsulated by the technology provider and then deployed to the user terminal. The training module is used to train a preset voiceprint recognition network based on the complete acoustic features to obtain a voiceprint recognition model.

12. A voiceprint recognition device, characterized in that, Deployed on the user end, including: The determination module is used to determine the target speech data to be recognized; The recognition module is used to perform voiceprint recognition on the target speech data using a voiceprint recognition model to obtain the acoustic features of the target speech data; The voiceprint recognition model is trained based on the voiceprint recognition model training method according to any one of claims 1 to 9.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the voiceprint recognition model training method as described in any one of claims 1 to 9, or implements the voiceprint recognition method as described in claim 10.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the voiceprint recognition model training method as described in any one of claims 1 to 9, or implements the voiceprint recognition method as described in claim 10.

Images