Speaker confirmation method and device, equipment and storage medium

Abstract

The invention discloses a speaker confirmation method, device and equipment and a storage medium, and relates to the technical field of information processing, and the method comprises the steps: segmenting an effective audio corresponding to a to-be-recognized speaker into a plurality of subsentence audios; inputting each clause audio into a preset speaker confirmation model to obtain a plurality of speaker feature vectors, the preset speaker confirmation model being constructed based on the high-quality sample audio and audio implicit vector information; determining an average feature vector of the speakers according to the feature vectors of the speakers; and performing identity recognition on the to-be-recognized speaker based on the speaker average feature vector and the standard speaker feature vector. According to the technical scheme, the technical problem that in the prior art, when a speaker confirmation model carries out speaker recognition in a low-quality channel in an actual application scene, the accuracy is not high is solved.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a speaker confirmation method, device, equipment and storage medium. Background Art

[0002] In consumer finance scenarios, for the purpose of protecting property safety and anti-fraud, it is necessary to confirm the identity of the user. One of the steps of identity confirmation is to confirm through the user's call voice whether the user's voice is the same as the speaker when the user registered.

[0003] In the existing scheme, a staged speaker verification model can be used to compare the registered audio with the audio to be verified, so as to determine whether the two audios are spoken by the same speaker. Specifically, the audio pairs to be compared can be respectively input into the feature extraction model to extract the audio vector, and then the similarity calculation method such as cosine distance is used to calculate the similarity score of a pair of audio feature vectors, and the speaker recognition is implemented based on the similarity score. However, the training and test sets used by most models are lossless audio in the original WAV format with a sampling rate of 16k. In the low-quality channels in actual application scenarios, such as telephone call channels, it is more common to use audio with a lower sampling rate or compressed audio for the convenience of storage. At this time, the accuracy of model recognition is not high and the recognition effect is poor.

[0004] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present application is to provide a speaker verification method, apparatus, device and storage medium, aiming to solve the technical problem that the accuracy of the speaker verification model in the prior art is low when performing speaker recognition on a low-quality channel in actual application scenarios.

[0006] To achieve the above objectives, the present application proposes a method for speaker confirmation, the method comprising: Divide the valid audio corresponding to the speaker to be identified into a number of sentence audios; Inputting each sentence audio into a preset speaker verification model to obtain a plurality of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; Determine a speaker average feature vector based on the feature vectors of each speaker; The speaker to be identified is identified based on the speaker average feature vector and the standard speaker feature vector.

[0007] In one embodiment, before the step of dividing the valid audio corresponding to the speaker to be identified into a plurality of sentence audios, the method further includes: Performing audio restoration processing and implicit vector extraction processing on low-quality audio respectively through a preset audio restoration model to obtain high-quality audio and implicit vector information, wherein the implicit vector information is audio information when audio restoration is performed on the low-quality audio; An initial speaker verification model is trained based on the high-quality audio and the latent vector information to obtain a preset speaker verification model.

[0008] In one embodiment, the initial speaker confirmation model is provided with: an audio feature extraction module, a long and short speech statistics aggregation module and a feature projection module; the step of training the initial speaker confirmation model based on the high-quality audio and the latent vector information to obtain a preset speaker confirmation model includes: Adding an implicit vector to the high-quality audio based on the implicit vector information by the audio feature extraction module to obtain processed high-quality audio; The audio feature extraction module performs attention transformation on the processed high-quality audio in the channel dimension and the time dimension to obtain input audio features of several different channels; Performing feature segmentation on the input audio feature by the long and short speech statistical aggregation module to obtain short audio features corresponding to the input audio feature; Generate an input audio feature matrix corresponding to the input audio feature and a short audio feature matrix corresponding to the short audio feature; Performing feature splicing and fusion processing on the input audio feature matrix and the short audio feature matrix to obtain an aggregated audio feature matrix; Projecting the aggregated audio feature matrix into a low-dimensional space through the feature projection module to obtain projected audio features; The initial speaker verification model is trained based on the projected audio features to obtain a preset speaker verification model.

[0009] In one embodiment, the step of training the initial speaker verification model based on the projected audio features to obtain a preset speaker verification model includes: Determining the feature similarity between the projected audio feature and the audio feature to be compared by using the initial speaker verification model; Obtaining a speaker recognition result according to the feature similarity; determining a false rejection rate and a false acceptance rate of the initial speaker verification model based on the speaker recognition result; The initial speaker verification model is optimized based on the false rejection rate and the false acceptance rate to obtain a preset speaker verification model.

[0010] In one embodiment, before the step of performing audio restoration processing and latent vector extraction processing on low-quality audio by using a preset audio restoration model to obtain high-quality audio and latent vector information, the step further includes: Collect several high-quality sample audios; Performing audio compression processing on the high-quality sample audio to obtain low-quality sample audio; An initial audio restoration model is trained based on the low-quality sample audio to obtain a preset audio restoration model.

[0011] In one embodiment, the initial audio restoration model is provided with: an audio encoder, a diffusion restoration module and an audio decoder; the step of training the initial audio restoration model based on the low-quality sample audio to obtain a preset audio restoration model includes: Inputting the low-quality sample audio into an initial audio restoration model; Performing compression encoding processing on the low-quality sample audio by the audio encoder to obtain a low-quality audio latent vector; Performing audio restoration on the low-quality audio latent vector by the diffusion restoration module to obtain a high-quality audio latent vector; Decoding the high-quality audio latent vector by the audio decoder to obtain high-quality predicted audio; The initial audio restoration model is optimized based on the high-quality sample audio and the high-quality predicted audio to obtain a preset audio restoration model.

[0012] In one embodiment, the step of optimizing the initial audio restoration model based on the high-quality sample audio and the high-quality predicted audio to obtain a preset audio restoration model includes: Acquire a first audio spectrum corresponding to the high-quality sample audio and a second audio spectrum corresponding to the high-quality predicted audio; determining a degree of spectral convergence between the first audio spectrum and the second audio spectrum; The initial audio restoration model is optimized based on the frequency spectrum convergence to obtain a preset audio restoration model.

[0013] In addition, to achieve the above-mentioned purpose, the present application also proposes a speaker confirmation device, the device comprising: An audio segmentation module is used to segment the valid audio corresponding to the speaker to be identified into a number of sentence audios; A feature extraction module, used to input each sentence audio into a preset speaker verification model to obtain a number of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; A feature vector determination module, used to determine a speaker average feature vector based on the feature vectors of each speaker; The speaker recognition module is used to identify the speaker to be recognized based on the speaker average feature vector and the standard speaker feature vector.

[0014] In addition, to achieve the above objectives, the present application also proposes a speaker confirmation device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speaker confirmation method described above.

[0015] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. When the computer program is executed by a processor, the steps of the speaker confirmation method described above are implemented.

[0016] The present application provides a speaker verification method, which discloses the following steps: dividing the effective audio corresponding to a speaker to be identified into a plurality of sentence audios; inputting each sentence audio into a preset speaker verification model to obtain a plurality of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; determining the speaker average feature vector according to each speaker feature vector; and identifying the speaker to be identified based on the speaker average feature vector and the standard speaker feature vector; compared with the speaker verification model in the prior art, the training and test sets used are both original WAV format lossless audio with a sampling rate of 16k, and the accuracy rate is not high when the audio with a lower sampling rate or compressed audio is recognized in a low-quality channel in an actual application scenario. Since the present invention can pre-construct a preset speaker verification model based on high-quality sample audio and audio latent vectors, and then obtain the speaker feature vector corresponding to the effective audio of the speaker to be identified based on the preset speaker verification model, and finally perform speaker recognition based on the corresponding speaker average feature vector and the standard speaker feature vector, the technical problem of low accuracy of the speaker verification model in the prior art when performing speaker recognition in a low-quality channel in an actual application scenario is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0019] Figure 1 A flowchart of the first embodiment of the speaker confirmation method of the present application is provided; Figure 2 This is a schematic diagram of the overall flow chart of the speaker confirmation method of this application; Figure 3 A flowchart of the second embodiment of the speaker confirmation method of the present application is provided; Figure 4 This is a schematic diagram of the structure of a speaker verification model in the speaker verification method of this application; Figure 5 A flowchart of the third embodiment of the speaker confirmation method of the present application is provided; Figure 6 This is a schematic diagram of the structure of the audio recovery model in the speaker verification method of this application; Figure 7 This is a schematic diagram of the module structure of the speaker verification device according to an embodiment of the present application; Figure 8 Schematic diagram of the device structure of the hardware operating environment involved in the speaker confirmation method in the embodiment of the present application.

[0020] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution of the embodiment of the present application is: dividing the valid audio corresponding to the speaker to be identified into a number of sentence audios; inputting each sentence audio into a preset speaker confirmation model to obtain a number of speaker feature vectors, wherein the preset speaker confirmation model is constructed based on high-quality sample audio and audio latent vector information; determining the speaker average feature vector based on each speaker feature vector; and identifying the speaker to be identified based on the speaker average feature vector and the standard speaker feature vector.

[0024] Since most speaker verification models in the prior art use lossless audio in the original WAV format with a sampling rate of 16k for training and testing, and in low-quality channels in actual application scenarios, such as telephone call channels, it is more common to use audio with a lower sampling rate or compressed audio for easy storage. In this case, the accuracy of model recognition is not high.

[0025] The present application provides a solution, which can pre-build a preset speaker confirmation model based on high-quality sample audio and audio latent vectors, then obtain a speaker feature vector corresponding to the valid audio of the speaker to be identified based on the preset speaker confirmation model, and finally perform speaker recognition based on the corresponding speaker average feature vector and standard speaker feature vector, thereby solving the technical problem that the speaker confirmation model in the prior art has low accuracy when performing speaker recognition on a low-quality channel in actual application scenarios.

[0026] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of realizing the above functions, a speaker confirmation device, etc. The following takes the speaker confirmation device as an example (hereinafter referred to as the device) to illustrate this embodiment and the following embodiments.

[0027] Based on this, the present application embodiment provides a method for speaker confirmation, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the speaker confirmation method of the present application.

[0028] In this embodiment, the speaker confirmation method includes steps S10 to S40: Step S10: dividing the valid audio corresponding to the speaker to be identified into a plurality of sentence audios.

[0029] It should be noted that the above-mentioned effective audio can be the audio obtained by removing the silent part and the non-speech part in the audio of the speaker to be identified using effective voice detection. In practical applications, this solution can be applied to but not limited to speaker recognition in financial call scenarios.

[0030] It should be understood that the above-mentioned sentence audio can be audio obtained after segmenting the effective audio. In this embodiment, for the financial call scenario, the device can obtain the channel audio of the speaker to be identified in real time during the call, and use effective voice detection to remove the silent part and non-voice part in the audio to obtain the effective audio of the speaker to be identified, and then use effective voice detection to segment it into multiple audio sentence samples to obtain the above-mentioned sentence audio.

[0031] Step S20: Input each sentence audio into a preset speaker verification model to obtain a number of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information.

[0032] It should be noted that the above-mentioned high-quality sample audio can be a high-sampling rate, lossless sample audio used to train the speaker confirmation model; the above-mentioned audio latent vector information can be audio information that helps to train the speaker confirmation model in the process of restoring low-quality audio to high-quality audio.

[0033] In practical applications, before training a preset speaker verification model, audio restoration processing can be performed on low-quality sample audio collected from actual scenarios to restore it to high-quality sample audio, and audio information of the low-quality sample audio that helps train the speaker verification model during the audio restoration process is collected to obtain audio latent vector information, so as to improve the recognition effect of the speaker verification model for low-quality channel audio through high-quality sample audio and audio latent vector information, wherein the latent vector information is generally one or a group of vectors of implicit audio features obtained through training in the audio recovery model.

[0034] It should be noted that the above-mentioned preset speaker confirmation model can be constructed and trained using a deep learning-based model. In this embodiment, the device can input high-quality sample audio and audio latent vector information into the preset speaker confirmation model. At this time, the model can extract the features that can distinguish the speakers for speaker comparison and confirmation, that is, the above-mentioned speaker feature vector.

[0035] Step S30: determining the speaker average feature vector according to the feature vectors of each speaker.

[0036] It is understandable that the above speaker average feature vector is the average vector of speaker feature vectors. In this embodiment, using the speaker average feature vector to identify the speaker can reduce the difference between different audio sample features, thereby improving the accuracy of speaker identification.

[0037] Step S40: performing identity recognition on the speaker to be recognized based on the speaker average feature vector and the standard speaker feature vector.

[0038] It should be noted that the above-mentioned standard speaker feature vector may be a feature vector corresponding to the comparison audio pre-recorded by all speakers during registration.

[0039] In actual applications, the device can compare the speaker average feature vector of the speaker to be identified with the feature vector corresponding to the audio used for comparison to determine whether the valid audio input into the preset speaker confirmation model and the audio to be compared belong to the same speaker, thereby achieving speaker recognition.

[0040] In the specific implementation, refer to Figure 2 , Figure 2 This is a schematic diagram of the overall flow chart of the speaker confirmation method of this application. Figure 2 As shown, this solution can first collect high-quality audio (i.e., high sampling rate, lossless audio), and generate low-quality audio (i.e., low sampling rate, lossy audio) through downsampling and audio compression processing, and then the audio recovery model can be obtained based on the high-quality audio and low-quality audio training. Before constructing the speaker confirmation model, the low-quality audio can be restored to high-quality audio through the audio recovery model and the audio latent vector information can be obtained. Then, the high-quality audio and the audio latent vector information are used as input to construct and train the speaker confirmation model, and the above-mentioned preset speaker confirmation model is obtained after the training is completed. It should be noted that since the high-quality audio and the speaker confirmation model are sensitive to noise (including stationary noise, instantaneous noise, other speech noise, etc.), before the audio is input into the speaker confirmation model, the noise can be reduced by an adaptive filter or a noise reduction model, wherein the stationary noise can be estimated by sampling in advance during the period without speech through an adaptive filter, and the noise can be canceled during the period with speech; the instantaneous noise cannot be canceled by filter estimation, and the noise reduction model is usually used to estimate and eliminate the instantaneous noise of the audio segment. Thereafter, the input speech of the speaker to be identified (i.e., the above-mentioned valid audio) can be firstly segmented into a plurality of sentence audios, and each sentence audio can be input into a preset speaker confirmation model. At this time, the preset speaker confirmation model can output a speaker feature vector, and then, the speaker average feature vector of the speaker feature vector can be calculated. Finally, the speaker average feature vector can be compared with the standard speaker feature vector corresponding to the speech to be compared, so as to confirm whether the speaker to be identified and the speaker to whom the speech to be compared belongs are the same person, thereby realizing the recognition of the speaker to be identified.

[0041] In this embodiment, for the financial telephone scenario, first, high-quality speech of multiple speakers can be collected with a sampling rate of 44k wav format audio as the standard, wherein the number of speakers is more than a thousand, and each speaker has at least 10 minutes of effective speech as the goal, and the voice collection channels can include network video / live broadcast data, open source data, self-recorded data, etc. For the collected high-quality voice audio, it can be compressed with a sampling rate of 8k and MP3 format as the goal to obtain a one-to-one high-quality and low-quality audio data set. Then, the device can use the high-quality audio data set as input and the low-quality audio data set as the target to train the audio recovery model, wherein, since the audio recovery model uses a reversible diffusion process, the high-to-low quality transformation will be learned during the training process, and the model will be reversed when applied to restore the low-quality audio to high quality. When performing financial telephone identity confirmation, due to telephone channel restrictions and storage costs, the audio format of financial telephone scene conversations is basically an mp3 format with an 8k sampling rate. This embodiment can classify the conversation audio under the name of each speaker according to the speaker's ID card / mobile phone number as the primary key. In addition, for dual-channel audio, the channel where the speaker is located can be taken, and the silent and non-voice parts in the audio can be removed using effective voice detection to obtain the effective audio of each speaker. Among them, the data quantity target in the financial telephone identity confirmation scene data set is more than 10,000 people, and each person has at least 30 seconds of effective voice as the target. The collection channel can be an actual telephone identity confirmation scene. After collecting and obtaining the financial telephone identity confirmation scene data set, the speaker confirmation model can be trained based on the data set. The training goal is to find a transformation model that makes the distance between multiple speaker clusters as large as possible. The evaluation index is the model's false positive rate and false negative rate as low as possible. Specifically, this solution can select effective audio of multiple speakers, cross-construct a test data set for one-to-one comparison between two speakers, and use the model to predict the speaker feature vector, and select a threshold to measure the evaluation index. After training the above audio recovery model and speaker confirmation model, the two models can be used together for speaker confirmation in financial phone scenarios. Specifically, the speaker's vocal tract audio is obtained in real time during the call, and divided into multiple audio sentence samples using effective speech detection. Then the model is applied to obtain multiple feature vectors respectively, and the average vector is calculated for multiple feature vectors to reduce the difference between the features of different audio samples. Speaker confirmation is divided into two stages: registration and confirmation. In the registration stage, the above average vector is associated with the customer's identity identifier and saved after the customer's first call. In the confirmation stage, the average vector obtained is compared with the registered average vector during each subsequent call of the customer to determine whether the calling customer is the same person.

[0042] The present embodiment provides a speaker verification method, which discloses dividing the valid audio corresponding to the speaker to be identified into a plurality of sentence audios; inputting each sentence audio into a preset speaker verification model to obtain a plurality of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; determining the speaker average feature vector according to each speaker feature vector; and identifying the speaker to be identified based on the speaker average feature vector and the standard speaker feature vector; compared with the speaker verification model in the prior art, the training and test sets used are both original wav format lossless audio with a sampling rate of 16k, and the accuracy rate is not high when the audio with a lower sampling rate or compressed audio is recognized in a low-quality channel in an actual application scenario. Since the present embodiment can pre-construct a preset speaker verification model based on high-quality sample audio and audio latent vectors, and then obtain the speaker feature vector corresponding to the valid audio of the speaker to be identified based on the preset speaker verification model, and finally perform speaker recognition based on the corresponding speaker average feature vector and the standard speaker feature vector, the technical problem of low accuracy of the speaker verification model in the prior art when performing speaker recognition in a low-quality channel in an actual application scenario is solved.

[0043] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , Figure 3 A flowchart of the second embodiment of the speaker confirmation method of the present application is provided.

[0044] In this embodiment, before step S10, the speaker confirmation method further includes steps S01-S02: Step S01: Performing audio restoration processing and implicit vector extraction processing on low-quality audio through a preset audio restoration model to obtain high-quality audio and implicit vector information, wherein the implicit vector information is audio information when audio restoration is performed on the low-quality audio.

[0045] It should be understood that the low-quality audio may be a lossy audio with a low sampling rate; correspondingly, the high-quality audio may be a lossless audio with a high sampling rate obtained by restoring the low-quality audio. In the process of restoring the low-quality audio to high-quality audio by using a preset audio restoration model, the audio restoration model may be spliced ​​to obtain implicit vector information, so as to improve the effect of the speaker verification model on the low-quality channel audio by using the high-quality audio and the implicit vector information.

[0046] Step S02: training an initial speaker verification model based on the high-quality audio and the latent vector information to obtain a preset speaker verification model.

[0047] Furthermore, the initial speaker verification model is provided with: an audio feature extraction module, a long and short speech statistics aggregation module and a feature projection module; the step S02 includes: Step S021: adding an implicit vector to the high-quality audio based on the implicit vector information through the audio feature extraction module to obtain processed high-quality audio.

[0048] It should be noted that the above audio feature extraction module can be used to extract audio features and latent vector features that can distinguish the speaker, so as to facilitate subsequent comparison and confirmation of the speaker.

[0049] Step S022: The audio feature extraction module performs attention transformation on the processed high-quality audio in the channel dimension and the time dimension to obtain input audio features of several different channels.

[0050] In this embodiment, the initial speaker verification model is provided with: an audio feature extraction module, a long and short speech statistics aggregation module, and a feature projection module. Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the structure of the speaker confirmation model in the speaker confirmation method of this application. Figure 4 As shown, the audio feature extraction module may include a convolution part and a channel attention part, wherein the convolution part has 4 layers, each layer includes 3, 4, 6, and 3 convolution modules respectively, and the output is standardized and implicit vectors are added between layers to obtain processed high-quality audio; the channel attention part performs attention transformation on the input high-quality audio in two dimensions, channel and timing, captures the correlation between high-quality audio in different channels and timing, and finally obtains the input audio features of several different channels.

[0051] Step S023: feature segmentation is performed on the input audio feature by the long and short speech statistics aggregation module to obtain short audio features corresponding to the input audio feature.

[0052] It should be noted that the long and short speech statistics aggregation module can be used to unify audio features. In practical applications, after the audio is processed by the audio feature extraction module, the features obtained still have different time lengths. At this time, it is necessary to convert it into a vector of fixed dimension through aggregation before it can be used for comparison and confirmation of the speaker. Therefore, this embodiment can further process the input audio features through the long and short speech statistics aggregation module. Specifically, the long and short speech statistics aggregation module can perform fixed segmentation on the input long speech and divide it into short speech, thereby obtaining short audio features corresponding to the input audio features.

[0053] Step S024: Generate an input audio feature matrix corresponding to the input audio feature and a short audio feature matrix corresponding to the short audio feature.

[0054] It should be noted that the above-mentioned input audio feature matrix can be a feature matrix corresponding to the input audio features; correspondingly, the above-mentioned short audio feature matrix can be a feature matrix corresponding to the short audio features. In this embodiment, after the long speech is divided into short speech, the long and short speech statistical aggregation module can calculate the average value of multiple different channels of long speech and short speech, the period estimation of the variance 20, 50, and 100 millisecond windows, and 6 statistics of the maximum fluctuation, so that each audio feature is converted into a feature matrix with the same length and the number of features equal to 6, and the above-mentioned input audio feature matrix and short audio feature matrix are obtained.

[0055] Step S025: performing feature splicing and fusion processing on the input audio feature matrix and the short audio feature matrix to obtain an aggregated audio feature matrix.

[0056] It should be noted that the above-mentioned aggregated audio feature matrix can be a feature matrix obtained by concatenating and fusing the input audio feature matrix and the short audio feature matrix. In practical applications, although long speech contains more speaker information, it is also more easily affected by the content; although the short speech voiceprint feature is purer, it has less information. Therefore, in order to improve the accuracy of speaker recognition, the matrices obtained from long and short speech can be concatenated and fused to obtain a more comprehensive and stable aggregated audio feature matrix.

[0057] Step S026: Projecting the aggregated audio feature matrix to a low-dimensional space through the feature projection module to obtain projected audio features.

[0058] It should be understood that since the aggregated feature matrix is ​​usually of high dimension and difficult to distinguish, if a large number of speakers need to be distinguished during the training process, it is necessary to project the high-dimensional features into a low-dimensional space that is easy to distinguish. The projection methods usually include full connection, cosine similarity projection, and inter-class angular margin projection. Figure 4 As shown, this embodiment can use a projection method of hyperspherical projection plus inter-class angle, which is used to increase the inter-class distance and reduce the intra-class distance during training. The projected features are finally adjusted to the required feature vector size using a fully connected layer, thereby reducing the amount of calculation when comparing speakers, thereby improving speaker recognition efficiency.

[0059] Step S027: training an initial speaker verification model based on the projected audio features to obtain a preset speaker verification model.

[0060] Further, the step S027 includes: determining the feature similarity between the projected audio features and the audio features to be compared through the initial speaker confirmation model; obtaining a speaker recognition result according to the feature similarity; determining a false rejection rate and a false acceptance rate of the initial speaker confirmation model based on the speaker recognition result; optimizing the initial speaker confirmation model based on the false rejection rate and the false acceptance rate to obtain a preset speaker confirmation model.

[0061] It can be understood that the above-mentioned audio features to be compared can be features corresponding to the audio used to compare with the audio of the speaker to be identified, such as features corresponding to the audio provided by the speaker when registering; the above-mentioned feature similarity can be a value used to characterize the degree of similarity between the projected audio features and the audio features to be compared.

[0062] It should be understood that the device can compare the audio of the speaker to be identified with the audio to be compared to determine whether the two audios belong to the same speaker, thereby obtaining the speaker recognition result. Specifically, if the feature similarity between the projected audio features and the audio features to be compared exceeds a certain set threshold (such as 95%), it can be determined that the audio of the speaker to be identified and the audio to be compared belong to the same person; otherwise, it is determined that the audio of the speaker to be identified and the audio to be compared do not belong to the same person.

[0063] It should be noted that the above-mentioned false rejection rate can be the ratio of the speaker verification model treating two sounds that should belong to the same speaker as different people (false rejection), where false rejection rate = number of test pairs of false rejection / total number of test pairs*100%; the above-mentioned false acceptance rate can be the ratio of the speaker verification model treating two sounds that should not belong to the same speaker as the same person (false acceptance), where false acceptance rate = number of test pairs of false acceptance / total number of test pairs×100%.

[0064] In practical applications, the current false rejection rate and false acceptance rate of the model can be calculated based on the speaker recognition results, and the model effect of the speaker confirmation model can be evaluated based on the false rejection rate and the false acceptance rate. Generally speaking, the lower the values ​​of the false rejection rate and the false acceptance rate, the better the effect of the model. In this embodiment, when the false rejection rate and the false acceptance rate reach the expected values, the training of the initial speaker confirmation model can be completed to obtain the preset speaker confirmation model.

[0065] In this embodiment, it is disclosed that audio recovery processing and implicit vector extraction processing are performed on low-quality audio through a preset audio recovery model to obtain high-quality audio and implicit vector information, where the implicit vector information is the audio information when audio recovery is performed on the low-quality audio; an initial speaker confirmation model is trained based on the high-quality audio and the implicit vector information to obtain a preset speaker confirmation model. Since this embodiment can jointly train the speaker confirmation model based on the high-quality audio and the implicit vector information, the recognition effect of the speaker confirmation model for low-quality channel audio is improved.

[0066] Based on the first embodiment and / or the second embodiment of the present application, in the third embodiment of the present application, the same or similar contents as those in the above embodiments can be referred to the above introduction, and will not be described in detail later. Figure 5 , Figure 5 This is a flowchart of the third embodiment of the speaker confirmation method of the present application.

[0067] In this embodiment, before step S10, the speaker confirmation method further includes steps S11 to S13: Step S11: Collect a number of high-quality sample audios.

[0068] It should be understood that the high-quality sample audio can be a high-sampling rate lossless audio for training the audio recovery model. The high-quality sample audio in this embodiment can be a 16k sampling rate audio, or a maximum of 44.1k or 48k sampling rate lossless audio.

[0069] Step S12: Perform audio compression processing on the high-quality sample audio to obtain low-quality sample audio.

[0070] It is understandable that the low-quality sample audio mentioned above can be the audio obtained by downsampling and compressing the high-quality sample audio. In this embodiment, the audio compression processing can be the processing of obtaining audio with lower sampling rate, quality and size by methods such as audio downsampling and audio compression algorithm compression of the high-quality sample audio. Among them, the audio downsampling method is a method of converting high-sampling rate audio (such as 48k, 44k, 16k, etc.) into low-sampling rate audio such as 8k audio using a downsampling algorithm, and the downsampling algorithm includes linear interpolation, sinusoidal interpolation, averaging, convolution, etc.; the audio compression method is a method of compressing lossless audio with loss using methods such as MP3 compression and ACC compression.

[0071] Step S13: training an initial audio restoration model based on the low-quality sample audio to obtain a preset audio restoration model.

[0072] Furthermore, the initial audio restoration model is provided with: an audio encoder, a diffusion restoration module and an audio decoder; the step S13 comprises: Step S131: inputting the low-quality sample audio into an initial audio restoration model.

[0073] It can be understood that the initial audio recovery model in this embodiment can be a model based on deep learning, and its model structure can use convolution and deconvolution, diffusion model, variational autoencoder, etc. This embodiment does not limit this.

[0074] Step S132: compress and encode the low-quality sample audio through the audio encoder to obtain a low-quality audio latent vector.

[0075] It should be noted that the low-quality audio latent vectors mentioned above may be a set of latent vectors obtained by convolution compression encoding the low-quality sample audio. In this embodiment, the audio encoder may be used to obtain a set of latent vectors by convolution compression encoding the input original low-quality audio waveform, so as to facilitate the processing of each module later.

[0076] In this embodiment, the audio encoder includes 4 bidirectional dilated convolution layers, each with 64 convolution kernels. "Bidirectional" means that the convolution kernel slides on the audio waveform from two temporal directions, namely from the beginning to the end of the audio, and from the end to the beginning. Its function is to capture the forward and backward information of the audio at a certain time point, that is, to obtain the context dependency; "dilation" means that the size of each convolution kernel is the same (all 3), but the interval between the sampling points of different convolution kernels gradually increases from 0 to 63. Its function is to increase the receptive field (range) of the convolution without increasing the size or resolution of the convolution kernel, thereby reducing the amount of model calculation and improving the efficiency of audio restoration.

[0077] Step S133: Perform audio restoration on the low-quality audio latent vector through the diffusion recovery module to obtain a high-quality audio latent vector.

[0078] In this embodiment, the diffusion recovery module can perform convolution and deconvolution processing on the low-quality audio latent vector to gradually restore it to the latent vector of high-quality audio (that is, the above-mentioned high-quality latent vector).

[0079] In practical applications, refer to Figure 6 , Figure 6 FIG. 1 is a schematic diagram of the structure of the audio recovery model in the speaker verification method of this application. Figure 6As shown, after inputting the low-quality sample audio into the initial audio recovery model, the audio encoder in the initial audio recovery model can first perform convolution compression encoding on the low-quality audio waveform to obtain a set of implicit vectors z, and then the diffusion recovery module can gradually restore the implicit vector z of the low-quality audio to the implicit vector Zt of the high-quality audio through multiple convolution and deconvolution modules. Among them, the "diffusion" of the diffusion recovery module represents that the audio recovery process is gradual. Each layer of modules predicts the audio difference, and the difference between the implicit vectors of high-quality and low-quality audio gradually decreases, and finally the implicit vector of high-quality audio is obtained. Each convolution and deconvolution module in the diffusion recovery module contains 4 convolution kernels, with sizes of 3, 7, 11 and 17 respectively. During the convolution process, the convolution kernels are used from large to small, and during the deconvolution, they are used from small to large to ensure that the size of the input and output implicit vectors remains unchanged.

[0080] Step S134: Decode the high-quality audio latent vector through the audio decoder to obtain high-quality predicted audio.

[0081] It should be noted that if Figure 6 As shown, the audio decoder can restore the implicit vector Zt of the high-quality audio output by the diffusion recovery module into a high-quality audio waveform through the transformers module, and finally output the above-mentioned high-quality predicted audio. In this embodiment, the audio decoder includes 2 attention layers and 2 fully connected layers, which are used to learn the implicit features and recovery parameters of different audio time positions to help better reconstruct high-quality audio.

[0082] Step S135: Optimizing the initial audio restoration model based on the high-quality sample audio and the high-quality predicted audio to obtain a preset audio restoration model.

[0083] Furthermore, the step S135 includes: obtaining a first audio spectrum corresponding to the high-quality sample audio, and a second audio spectrum corresponding to the high-quality predicted audio; determining a spectral convergence between the first audio spectrum and the second audio spectrum; and optimizing the initial audio restoration model based on the spectral convergence to obtain a preset audio restoration model.

[0084] It should be noted that the spectrum convergence degree may be a value used to measure the error between the original audio spectrum and the restored audio spectrum (ie, the error between the first audio spectrum and the second audio spectrum).

[0085] In a specific implementation, this embodiment can use the spectrum convergence to evaluate the effect of the audio restoration model, which evaluates the model restoration effect by measuring the error between the original audio spectrum and the restored audio spectrum. The calculation method is as follows: First, we will restore the spectrum Divided into two parts: Target spectrum and interference error e; Secondly, the following calculations are performed using the original spectrum x and the restored spectrum:

[0086] in, express The transpose of represents the square of the L2 norm of x (i.e., signal energy); Then calculate , and calculate The energy corresponding to e and ; Finally, calculate the value of SDR: , wherein the smaller the SDR is, the closer the original and restored audio are, and the better the model effect is. In this embodiment, when the above-mentioned spectrum convergence reaches the expected value, the initial audio restoration model is trained to obtain a preset audio restoration model.

[0087] In this embodiment, it is disclosed that a number of high-quality sample audios are collected; audio compression processing is performed on the high-quality sample audios to obtain low-quality sample audios; an initial audio recovery model is trained based on the low-quality sample audios to obtain a preset audio recovery model, so that the preset audio recovery model can be used to subsequently achieve audio recovery from a low-quality channel to high-quality audio, thereby avoiding the problem of poor model recognition effect of the speaker confirmation model in a low-quality channel.

[0088] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the speaker confirmation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0089] This application also provides a speaker confirmation device, please refer to Figure 7 , the speaker confirmation device comprises: An audio segmentation module 10 is used to segment the valid audio corresponding to the speaker to be identified into a plurality of sentence audios; A feature extraction module 20, for inputting each sentence audio into a preset speaker verification model to obtain a plurality of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; A feature vector determination module 30, configured to determine a speaker average feature vector based on the feature vectors of each speaker; The speaker recognition module 40 is used to identify the speaker to be recognized based on the speaker average feature vector and the standard speaker feature vector.

[0090] The speaker verification device provided by the present application adopts the speaker verification method in the above embodiment, which can solve the technical problem that the accuracy of the speaker verification model in the prior art is not high when performing speaker recognition in a low-quality channel in an actual application scenario. Compared with the prior art, the beneficial effects of the speaker verification device provided by the present application are the same as the beneficial effects of the speaker verification method provided by the above embodiment, and other technical features in the speaker verification device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0091] The present application provides a speaker confirmation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the speaker confirmation method in the above-mentioned embodiment 1.

[0092] Reference below Figure 8 , which shows a schematic diagram of the structure of a speaker confirmation device suitable for implementing the embodiment of the present application. The speaker confirmation device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 8 The speaker confirmation device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0093] like Figure 8As shown, the speaker confirmation device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the speaker confirmation device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the speaker confirmation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a speaker confirmation device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems may be implemented or have alternatively.

[0094] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0095] The speaker confirmation device provided by the present application adopts the speaker confirmation method in the above embodiment, which can solve the technical problem of speaker confirmation. Compared with the prior art, the beneficial effects of the speaker confirmation device provided by the present application are the same as the beneficial effects of the speaker confirmation method provided by the above embodiment, and other technical features in the speaker confirmation device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0096] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0097] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0098] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the speaker confirmation method in the above-mentioned embodiment.

[0099] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0100] The computer-readable storage medium may be included in the speaker verification device; or may exist independently without being assembled into the speaker verification device.

[0101] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by a speaker confirmation device, the speaker confirmation device: divides the valid audio corresponding to the speaker to be identified into a plurality of sentence audios; inputs each sentence audio into a preset speaker confirmation model to obtain a plurality of speaker feature vectors, wherein the preset speaker confirmation model is constructed based on high-quality sample audio and audio latent vector information; determines a speaker average feature vector based on each speaker feature vector; and identifies the speaker to be identified based on the speaker average feature vector and a standard speaker feature vector.

[0102] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0103] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0104] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0105] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned speaker confirmation method, and can solve the technical problem that the accuracy of the speaker confirmation model in the prior art is not high when performing speaker recognition in a low-quality channel in an actual application scenario. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the speaker confirmation method provided by the above-mentioned embodiment, and will not be repeated here.

[0106] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for speaker confirmation, characterized in that: The method includes: Divide the valid audio corresponding to the speaker to be identified into a number of sentence audios; Inputting each sentence audio into a preset speaker verification model to obtain a plurality of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; Determine a speaker average feature vector based on the feature vectors of each speaker; The speaker to be identified is identified based on the speaker average feature vector and the standard speaker feature vector.

2. The method according to claim 1, characterized in that Before the step of dividing the valid audio corresponding to the speaker to be identified into a plurality of sentence audios, the method further includes: Performing audio restoration processing and implicit vector extraction processing on low-quality audio respectively through a preset audio restoration model to obtain high-quality audio and implicit vector information, wherein the implicit vector information is audio information when audio restoration is performed on the low-quality audio; An initial speaker verification model is trained based on the high-quality audio and the latent vector information to obtain a preset speaker verification model.

3. The method according to claim 2, characterized in that The initial speaker confirmation model is provided with: an audio feature extraction module, a long and short speech statistics aggregation module and a feature projection module; the step of training the initial speaker confirmation model based on the high-quality audio and the implicit vector information to obtain a preset speaker confirmation model includes: Adding an implicit vector to the high-quality audio based on the implicit vector information by the audio feature extraction module to obtain processed high-quality audio; The audio feature extraction module performs attention transformation on the processed high-quality audio in the channel dimension and the time dimension to obtain input audio features of several different channels; Performing feature segmentation on the input audio feature by the long and short speech statistical aggregation module to obtain short audio features corresponding to the input audio feature; Generate an input audio feature matrix corresponding to the input audio feature and a short audio feature matrix corresponding to the short audio feature; Performing feature splicing and fusion processing on the input audio feature matrix and the short audio feature matrix to obtain an aggregated audio feature matrix; Performing concatenation processing on the aggregated audio feature matrix and the implicit vector information to obtain a target aggregated audio feature matrix; Projecting the aggregated audio feature matrix into a low-dimensional space through the feature projection module to obtain projected audio features; The initial speaker verification model is trained based on the projected audio features to obtain a preset speaker verification model.

4. The method according to claim 3, characterized in that The step of training the initial speaker confirmation model based on the projected audio features to obtain a preset speaker confirmation model includes: Determining the feature similarity between the projected audio feature and the audio feature to be compared by using the initial speaker verification model; Obtaining a speaker recognition result according to the feature similarity; determining a false rejection rate and a false acceptance rate of the initial speaker verification model based on the speaker recognition result; The initial speaker verification model is optimized based on the false rejection rate and the false acceptance rate to obtain a preset speaker verification model.

5. The method according to claim 2, characterized in that Before the step of performing audio restoration processing and latent vector extraction processing on the low-quality audio by using a preset audio restoration model to obtain high-quality audio and latent vector information, the step further includes: Collect several high-quality sample audios; Performing audio compression processing on the high-quality sample audio to obtain low-quality sample audio; An initial audio restoration model is trained based on the low-quality sample audio to obtain a preset audio restoration model.

6. The method according to claim 5, characterized in that The initial audio restoration model is provided with: an audio encoder, a diffusion restoration module and an audio decoder; the step of training the initial audio restoration model based on the low-quality sample audio to obtain a preset audio restoration model includes: Inputting the low-quality sample audio into an initial audio restoration model; Performing compression encoding processing on the low-quality sample audio by the audio encoder to obtain a low-quality audio latent vector; Performing audio restoration on the low-quality audio latent vector by the diffusion restoration module to obtain a high-quality audio latent vector; Decoding the high-quality audio latent vector by the audio decoder to obtain high-quality predicted audio; The initial audio restoration model is optimized based on the high-quality sample audio and the high-quality predicted audio to obtain a preset audio restoration model.

7. The method according to claim 6, characterized in that The step of optimizing the initial audio restoration model based on the high-quality sample audio and the high-quality predicted audio to obtain a preset audio restoration model comprises: Acquire a first audio spectrum corresponding to the high-quality sample audio and a second audio spectrum corresponding to the high-quality predicted audio; determining a degree of spectral convergence between the first audio spectrum and the second audio spectrum; The initial audio restoration model is optimized based on the frequency spectrum convergence to obtain a preset audio restoration model.

8. A speaker verification device, characterized in that: The device comprises: An audio segmentation module, used to segment the valid audio corresponding to the speaker to be identified into a number of sentence audios; A feature extraction module, used to input each sentence audio into a preset speaker verification model to obtain a number of speaker feature vectors, wherein the preset speaker verification model is constructed based on high-quality sample audio and audio latent vector information; A feature vector determination module, used to determine a speaker average feature vector based on the feature vectors of each speaker; The speaker recognition module is used to identify the speaker to be recognized based on the speaker average feature vector and the standard speaker feature vector.

9. A speaker verification device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the speaker confirmation method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the speaker identification method according to any one of claims 1 to 7 are implemented.

Images