Cross-channel voiceprint recognition model training method, voiceprint recognition method and device

Abstract

The present invention discloses a training method for a cross-channel voiceprint recognition model, a voiceprint recognition method, and a device. The method comprises collecting different channel data sets, processing them to obtain training data, and using an improved fbank feature extraction algorithm to extract fbank features from the training data to obtain fbank features. A second fbank feature and a fourth fbank feature are used to train a voiceprint recognition model to obtain a first model and a second model, respectively. The first model and the second model are combined through a fusion structure to obtain a third model. The first fbank feature is used to train the third model to obtain a fourth model, and the third fbank feature is used to train the fourth model to obtain a cross-channel voiceprint recognition model. The present invention uses an improved fbank feature extraction algorithm to extract fbank features, and combines the output of the first model with the output of the second model through a fusion structure. This method can effectively improve the recognition accuracy of the voiceprint recognition model in cross-channel scenarios, thereby resolving problems such as a complex training process and low accuracy of the cross-channel voiceprint recognition model.

Description

Technical Field

[0001] The present invention relates to the field of voiceprint recognition, and in particular to a training method for a cross-channel voiceprint recognition model, a voiceprint recognition method, and a device. Background Art

[0002] Every person's voice contains unique biometric characteristics. Voiceprint recognition is a technology that uses a speaker's voice to identify them. Like fingerprint recognition, voiceprint recognition offers high security and reliability, and can be applied in all situations requiring identity verification, such as in criminal investigation, banking, securities, insurance, and other financial sectors.

[0003] Compared to traditional identity recognition technologies, voiceprint recognition offers advantages in that voiceprint extraction is simple, cost-effective, and unique, making it difficult to forge or counterfeit. In recent years, with the development of deep learning technology, voiceprint recognition technology has significantly improved and has been successfully applied in many scenarios.

[0004] Cross-channel scenarios in voiceprint recognition include cross-channel between devices and cross-channel due to different audio sampling rates. Cross-channel between devices refers to the situation where different devices are used to collect the voiceprint registration audio and the verification audio. Different audio sampling devices can affect voiceprint recognition accuracy. Cross-channel between sampling rates refers to the difference in voiceprint recognition performance due to different audio sampling rates. The audio sampling rates for voiceprint recognition are generally 8k and 16k. If the voiceprint registration audio uses an 8k sampling rate and the voiceprint verification audio uses a 16k sampling rate, the recognition results will be affected. Cross-channel recognition is unavoidable in real-life scenarios and has a certain impact on voiceprint recognition performance. Therefore, resolving the cross-channel issue is a prerequisite for the widespread application of voiceprint recognition technology. However, current deep learning methods have limitations in model training. Data from telephone and network channels must be separated. Model training on voice data from different channels is necessary to achieve a high recognition accuracy, which has hindered the widespread application of voiceprint recognition systems. Summary of the Invention

[0005] In response to the above-mentioned technical problems, the embodiment of the present application aims to propose a training method for a cross-channel voiceprint recognition model, a voiceprint recognition method and a device to solve the technical problems mentioned in the above background technology section.

[0006] In a first aspect, the present invention provides a method for training a cross-channel voiceprint recognition model, comprising the following steps:

[0007] S1, collecting data sets, the data sets including a first data set and a second data set based on a first channel, and a third data set and a fourth data set based on the second channel, performing data cleaning and data enhancement processing on the data sets to obtain training data, the training data including first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively;

[0008] S2, using an improved fbank feature extraction algorithm to extract fbank features from the training data to obtain fbank features, where the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively;

[0009] S3, using the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively, to obtain the first model and the second model, and the first model and the second model are combined through the fusion structure to obtain the third model;

[0010] S4: Use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

[0011] Preferably, collecting the data set in step S1 specifically includes:

[0012] collecting a first speech from a first channel;

[0013] Adjusting the sampling rate of the first speech to a preset frequency to obtain a first data set;

[0014] Extracting a portion of the first speech from the first data set as a second data set;

[0015] collecting a second voice from a second channel;

[0016] Adjusting the sampling rate of the second speech to a preset frequency to obtain a third data set;

[0017] A portion of the second speech is extracted from the third data set as a fourth data set.

[0018] Preferably, the data cleaning method in step S1 includes: removing silent segments in the speech by using a windowing method; the data enhancement method includes: adding noise, changing the speed or adding reverberation to the cleaned speech.

[0019] Preferably, the improved fbank feature extraction algorithm in step S2 specifically includes:

[0020] Pre-emphasize, frame, and window the speech signals in the training data;

[0021] Performing Fourier transform on the pre-emphasized, framed, and windowed speech signal to obtain the spectrum of each frame of the speech signal, and taking the modulus square of the spectrum of the speech signal to obtain the power spectrum of the speech signal;

[0022] Perform Mel filtering on the power spectrum to extract the Mel filtering output corresponding to the training data;

[0023] Merge the Mel filter output corresponding to the second training data and the Mel filter output corresponding to the fourth training data, and calculate the mean and standard deviation;

[0024] The Mel filter output corresponding to the training data is dynamically regularized according to the mean and standard deviation to obtain the dynamic regularization output:

[0025]

[0026] Where M(t) is the Mel filter output corresponding to the training data, d(t) is the dynamic regularization output, μ and σ are the mean and standard deviation respectively;

[0027] Take the logarithm of the dynamic warping output to obtain the fbank feature.

[0028] Preferably, the voiceprint recognition model includes Res2Net50, and the loss function used in the training process of the cross-channel voiceprint recognition model is the cross entropy loss function, and the formula is as follows:

[0029]

[0030] Where L is the loss function, The model predicts the output, y is the label, and the value is 0 or 1.

[0031] Preferably, the fusion structure includes a first adder, a first pooling layer, a first convolutional layer, a first normalization layer, a first activation function layer, a second convolutional layer, a second pooling layer, a third convolutional layer, a second normalization layer, a second activation function layer, a fourth convolutional layer, a second adder, a third activation function layer, a first multiplier, a second multiplier and a third adder, wherein the first pooling layer, the first convolutional layer, the first normalization layer, the first activation function layer and the second convolutional layer are connected in sequence, the second pooling layer, the third convolutional layer, the second normalization layer, the second activation function layer and the fourth convolutional layer are connected in sequence, the output of the first model and the output of the second model are added by the first adder to obtain a first addition result, and the first addition results are respectively input into the first The pooling layer and the second pooling layer are respectively extracted through the first convolution layer, the first normalization layer, the first activation function layer, the second convolution layer and the third convolution layer, the second normalization layer, the second activation function layer, and the fourth convolution layer to obtain the first feature and the second feature. The first feature and the second feature are added by the second adder to obtain the second addition result, and the second addition result is input into the third activation function layer. The output result of the third activation function layer is multiplied with the output of the first model and the output of the second model by the first multiplier and the second multiplier to obtain the first multiplication result and the second multiplication result. The output result of the third activation function layer, the first multiplication result, and the second multiplication result are added by the third adder to obtain the output of the third model.

[0032] Preferably, the first activation function layer and the second activation function layer use RELU, and the third activation function layer uses sigmoid.

[0033] In a second aspect, the present invention provides a training device for a cross-channel voiceprint recognition model, comprising:

[0034] a training data acquisition module configured to collect data sets, the data sets including a first data set and a second data set based on the first channel, and a third data set and a fourth data set based on the second channel, and perform data cleaning and data enhancement processing on the data sets to obtain training data, the training data including first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively;

[0035] a feature extraction module configured to extract fbank features from the training data using an improved fbank feature extraction algorithm to obtain fbank features, wherein the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively;

[0036] A model construction module is configured to use the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively to obtain a first model and a second model, and the first model and the second model are combined through a fusion structure to obtain a third model;

[0037] The model training module is configured to use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

[0038] In a third aspect, the present invention provides a cross-channel voiceprint recognition method, which uses a cross-channel voiceprint recognition model trained by the training method of the cross-channel voiceprint recognition model according to the first aspect, including the following steps:

[0039] T1, obtain the registered voice, adjust the sampling rate of the registered voice to the preset frequency, and use the improved fbank feature extraction algorithm to extract the fbank features corresponding to the registered voice. Input the fbank features corresponding to the registered voice into the cross-channel voiceprint recognition model to obtain the first feature vector;

[0040] T2: Obtain the verification voice, adjust the sampling rate of the verification voice to the preset frequency, and use the improved fbank feature extraction algorithm to extract the fbank features corresponding to the verification voice. Input the fbank features corresponding to the verification voice into the cross-channel voiceprint recognition model to obtain the second feature vector;

[0041] T3: perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

[0042] In a fourth aspect, the present invention provides a cross-channel voiceprint recognition device, which uses a cross-channel voiceprint recognition model trained by the training method of the cross-channel voiceprint recognition model according to the first aspect, including:

[0043] a registration module configured to obtain a registration voice, adjust the sampling rate of the registration voice to a preset frequency, extract fbank features corresponding to the registration voice using an improved fbank feature extraction algorithm, and input the fbank features corresponding to the registration voice into a cross-channel voiceprint recognition model to obtain a first feature vector;

[0044] a verification module configured to obtain a verification voice, adjust the sampling rate of the verification voice to a preset frequency, extract fbank features corresponding to the verification voice using an improved fbank feature extraction algorithm, and input the fbank features corresponding to the verification voice into a cross-channel voiceprint recognition model to obtain a second feature vector;

[0045] The comparison module is configured to perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

[0046] In a fifth aspect, the present invention provides an electronic device comprising one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0047] In a sixth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] (1) The present invention uses data cleaning and combines multiple data enhancement methods to enhance the data set, which can greatly increase the robustness of the training model in various application scenarios.

[0050] (2) The present invention adopts an improved fbank feature extraction algorithm to improve the recognition accuracy of the voiceprint recognition model in cross-channel scenarios.

[0051] (3) The present invention adopts a fusion structure to combine the output of the first model and the output of the second model to obtain a third model, and trains the third model to obtain a fourth model. The fourth model is fine-tuned to obtain a cross-channel voiceprint recognition model, which can effectively improve the cross-channel recognition accuracy of the trained cross-channel voiceprint recognition model and reduce the training difficulty. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 is a diagram of an exemplary device architecture to which an embodiment of the present application may be applied;

[0054] Figure 2 This is a flowchart of the training method for the cross-channel voiceprint recognition model of Example 1 of the present application;

[0055] Figure 3This is a flow chart of an improved fbank feature extraction algorithm for the training method of the cross-channel voiceprint recognition model of Example 1 of the present application;

[0056] Figure 4 This is a schematic diagram of the fusion structure of the training method for the cross-channel voiceprint recognition model of Example 1 of the present application;

[0057] Figure 5 This is a schematic diagram of a training device for a cross-channel voiceprint recognition model according to Example 1 of the present application;

[0058] Figure 6 This is a flowchart of the cross-channel voiceprint recognition method of Example 2 of the present application;

[0059] Figure 7 This is a schematic diagram of a cross-channel voiceprint recognition device according to Example 2 of the present application;

[0060] Figure 8 It is a structural diagram of a computer device suitable for implementing the electronic device of the embodiment of the present application. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the present invention more apparent, the present invention will be further described in detail below with reference to the accompanying drawings. It is apparent that the embodiments described are only some, not all, of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0062] Figure 1 An exemplary device architecture 100 is shown to which the cross-channel voiceprint recognition model training method or the cross-channel voiceprint recognition model training device of the embodiments of the present application can be applied.

[0063] like Figure 1 As shown, the device architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0064] Users can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications, such as data processing applications and file processing applications, can be installed on terminal devices 101, 102, 103.

[0065] Terminal devices 101, 102, and 103 can be hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices, including but not limited to smartphones, tablet computers, laptop computers, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, software or software modules used to provide distributed services), or they can be implemented as a single software or software module. No specific limitations are given here.

[0066] The server 105 may be a server that provides various services, such as a background data processing server that processes files or data uploaded by the terminal devices 101, 102, and 103. The background data processing server may process the acquired files or data and generate processing results.

[0067] It should be noted that the training method of the cross-channel voiceprint recognition model provided in the embodiment of the present application can be executed by the server 105 or by the terminal devices 101, 102, and 103. Accordingly, the training device of the cross-channel voiceprint recognition model can be set in the server 105 or in the terminal devices 101, 102, and 103.

[0068] It should be understood that Figure 1 The number of terminal devices, networks, and servers in the above description is merely illustrative. Any number of terminal devices, networks, and servers may be provided as needed. If the processed data does not need to be acquired remotely, the above-described apparatus architecture may not include a network, but only require servers or terminal devices.

[0069] Example 1

[0070] Figure 2 A training method for a cross-channel voiceprint recognition model provided in Example 1 of the present application is shown, comprising the following steps:

[0071] S1. Collect data sets, where the data sets include a first data set and a second data set based on a first channel, and a third data set and a fourth data set based on the second channel. Perform data cleaning and data enhancement on the data sets to obtain training data. The training data include first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively.

[0072] In a specific embodiment, collecting a data set in step S1 specifically includes:

[0073] collecting a first speech from a first channel;

[0074] Adjusting the sampling rate of the first speech to a preset frequency to obtain a first data set;

[0075] Extracting a portion of the first speech from the first data set as a second data set;

[0076] collecting a second voice from a second channel;

[0077] Adjusting the sampling rate of the second speech to a preset frequency to obtain a third data set;

[0078] A portion of the second speech is extracted from the third data set as a fourth data set.

[0079] Specifically, the preset frequency can be set to 16000Hz. In other embodiments, it can also be set to other values ​​according to specific scenarios. The first channel can be a telephone channel, and the first voice can be collected by recording a phone call, and the sampling rate of all the first voices collected from the first channel is upsampled from below 16000Hz to 16000Hz to obtain a first data set. Part of the first voice is extracted from the first data set to form a second data set; the second channel can be a network channel, and the second voice can be collected through the recorder of the mobile phone, and the sampling rate of all the second voices collected from the network channel is downsampled from above 16000Hz to 16000Hz to obtain a third data set, and part of the second voice is extracted from the third data set to form a fourth data set.

[0080] In a specific embodiment, the data cleaning method in step S1 includes: removing silent segments in the speech by using a windowing method; the data enhancement method includes: adding noise, changing the speed, or adding reverberation to the cleaned speech.

[0081] Specifically, before training the voiceprint recognition model, data cleaning and data enhancement operations need to be performed on the first, second, third, and fourth datasets. During the data cleaning process, a sliding window method is used to remove the silent segments in each voice, and all voices are divided into voice files with an effective duration of 30 seconds. The file names are used as labels for model training. The sliding window processing process is as follows:

[0082] (1) Assume a voice Audio = [X1, X2, X3, ..., X n ], where X is the speech sampling point, n = T*16000, and T is the speech duration in seconds;

[0083] (2) Calculate the speech energy value of the sliding window P with a window length of m: m = t * 16000, t = 0.1s;

[0084] (3) Let k be equal to 1, 2, 3, ..., (nm) in sequence, and calculate the speech energy E1, E2, E3, ..., En-m ;

[0085] (4) When the energy E calculated by the sampling points in the sliding window is less than the threshold K, the speech segment is considered to be a silent segment. All sampling points corresponding to the silent segment are deleted, and the remaining sampling points are saved as a speech file.

[0086] After data cleaning, the speech is enhanced. The data enhancement adopts the following three methods:

[0087] (1) Add noise, specifically, randomly add office noise, speech noise, outdoor traffic noise, etc. to each voice;

[0088] (2) Speed ​​change, specifically, the voice playback speed can be randomly changed to 0.9 times or 1.1 times;

[0089] (3) Add reverberation. Specifically, you can add reverberation to each voice.

[0090] After data enhancement, each voice can be expanded to 4, which can greatly increase the robustness of the training model in various application scenarios.

[0091] S2, using an improved fbank feature extraction algorithm to extract fbank features from the training data respectively to obtain fbank features, where the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively.

[0092] In a specific embodiment, the improved fbank feature extraction algorithm in step S2 specifically includes:

[0093] Pre-emphasize, frame, and window the speech signals in the training data;

[0094] Performing Fourier transform on the pre-emphasized, framed, and windowed speech signal to obtain the spectrum of each frame of the speech signal, and taking the modulus square of the spectrum of the speech signal to obtain the power spectrum of the speech signal;

[0095] Perform Mel filtering on the power spectrum to extract the Mel filtering output corresponding to the training data;

[0096] Merge the Mel filter output corresponding to the second training data and the Mel filter output corresponding to the fourth training data, and calculate the mean and standard deviation;

[0097] The Mel filter output corresponding to the training data is dynamically regularized according to the mean and standard deviation to obtain the dynamic regularization output:

[0098]

[0099] Wherein, M(t) is the Mel filter output corresponding to the training data, d(t) is the dynamic regularization output, μ and σ are the mean and standard deviation respectively;

[0100] Take the logarithm of the dynamic warping output to obtain the fbank feature.

[0101] Specifically, the improved fbank feature extraction algorithm is used to extract fbank features from the first data set, the second data set, the third data set, and the fourth data set. The feature extraction process is as follows: Figure 3 As shown. Among them, the dynamic regularization part is the improved part of the improved fbank feature extraction algorithm proposed in the embodiment of this application, and the rest is the same as the existing fbank feature extraction algorithm. The improved fbank feature extraction algorithm proposed in the embodiment of this application can effectively improve the cross-channel recognition accuracy of the trained voiceprint recognition model. The specific content of the dynamic regularization part is as follows:

[0102] The Mel filter output corresponding to the second training data is the Mel filter output obtained by extracting the n speech sounds of the second training data through the steps of pre-emphasis, framing, windowing, Fourier transform, and squaring, which is recorded as

[0103] M tel1 , M tel2 , M tel3 ,…,M teln The Mel filter output corresponding to the fourth training data is the Mel filter output obtained by extracting the n voices of the fourth training data through the steps of pre-emphasis, framing, windowing, Fourier transform, and square extraction, which is recorded as M net1 , M net2 , M net3 ,…,M netn The Mel filter output corresponding to the second training data and the Mel filter output corresponding to the fourth training data are combined to obtain a Mel filter output combination, and the mean μ and standard deviation σ of the Mel filter output combination are calculated:

[0104]

[0105]

[0106] Among them, M i =M tel1 , M tel2 , M tel3 ,…,M teln , M net1 , M net2 , M net3 ,…,M netn , N=2n.

[0107] The dynamic warping output is calculated using the following dynamic warping calculation formula:

[0108]

[0109] Wherein, M(t) is the Mel filter output corresponding to the training data, and d(t) is the dynamic regularization output. Specifically, M(t) is the Mel filter output corresponding to the first training data, the second training data, the third training data, or the fourth training data.

[0110] S3, use the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively, to obtain the first model and the second model, and the first model and the second model are combined through the fusion structure to obtain the third model.

[0111] In a specific embodiment, the voiceprint recognition model includes Res2Net50, and the loss function used in the training process of the cross-channel voiceprint recognition model is the cross entropy loss function, which is formulated as follows:

[0112]

[0113] Where L is the loss function, The model predicts the output, y is the label, and the value is 0 or 1.

[0114] Specifically, the second fbank features extracted from the second training data are used to train the voiceprint recognition model to obtain the first model, and the fourth fbank features extracted from the fourth training data are used to train the voiceprint recognition model to obtain the second model. Both the first model and the second model are trained using Res2Net50 as the base network. They can use the same network structure, but are trained using fbank features extracted from training data from different channels. The loss function for model training uses the cross entropy loss function.

[0115] In a specific embodiment, reference Figure 4The fusion structure includes a first adder, a first pooling layer, a first convolution layer, a first normalization layer, a first activation function layer, a second convolution layer, a second pooling layer, a third convolution layer, a second normalization layer, a second activation function layer, a fourth convolution layer, a second adder, a third activation function layer, a first multiplier, a second multiplier and a third adder, wherein the first pooling layer, the first convolution layer, the first normalization layer, the first activation function layer and the second convolution layer are connected in sequence, the second pooling layer, the third convolution layer, the second normalization layer, the second activation function layer and the fourth convolution layer are connected in sequence, the output of the first model and the output of the second model are added by the first adder to obtain a first addition result, and the first addition result is respectively input into the first pooling layer. The first feature and the second feature are extracted by the first convolution layer, the first normalization layer, the first activation function layer, the second convolution layer, the third convolution layer, the second normalization layer, the second activation function layer, and the fourth convolution layer respectively. The first feature and the second feature are added by the second adder to obtain the second addition result, and the second addition result is input into the third activation function layer. The output result of the third activation function layer is multiplied by the output of the first model and the output of the second model by the first multiplier and the second multiplier respectively to obtain the first multiplication result and the second multiplication result. The output result of the third activation function layer, the first multiplication result and the second multiplication result are added by the third adder to obtain the output of the third model.

[0116] In a specific embodiment, the first activation function layer and the second activation function layer use RELU, and the third activation function layer uses sigmoid.

[0117] Specifically, a fusion structure is used to combine the outputs of the first and second models into a new model, called the third model. By combining the outputs of the first and second models, trained on datasets from different channels, and forming a third model, further training the third model can effectively improve the cross-channel recognition accuracy of the voiceprint recognition model.

[0118] S4: Use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

[0119] Specifically, the third model is trained using the first fbank features corresponding to the first training data until the results converge. This training results in a fourth model, which is then fine-tuned using the fourth fbank features corresponding to the third training data to reduce the loss function and improve the accuracy. This results in a final model, referred to as the fifth model, which is the cross-channel voiceprint recognition model proposed in the embodiments of this application. The cross-entropy loss function is also used as the loss function during the training of the third and fourth models.

[0120] Further references Figure 5 As an implementation of the methods shown in the above figures, this application provides an embodiment of a training device for a cross-channel voiceprint recognition model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0121] The present embodiment provides a training device for a cross-channel voiceprint recognition model, including:

[0122] a training data acquisition module 1 configured to collect a data set, the data set including a first data set and a second data set based on the first channel, and a third data set and a fourth data set based on the second channel, and perform data cleaning and data enhancement processing on the data set to obtain training data, the training data including first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively;

[0123] Feature extraction module 2 is configured to extract fbank features from the training data using an improved fbank feature extraction algorithm to obtain fbank features, where the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively;

[0124] Model construction module 3 is configured to use the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively to obtain a first model and a second model, and the first model and the second model are combined through a fusion structure to obtain a third model;

[0125] The model training module 4 is configured to use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

[0126] Example 2

[0127] refer to Figure 6 Embodiment 2 of the present application further proposes a cross-channel voiceprint recognition method, which uses a cross-channel voiceprint recognition model trained according to the above-mentioned cross-channel voiceprint recognition model training method, including the following steps:

[0128] T1, obtain the registered voice, adjust the sampling rate of the registered voice to the preset frequency, and use the improved fbank feature extraction algorithm to extract the fbank features corresponding to the registered voice. Input the fbank features corresponding to the registered voice into the cross-channel voiceprint recognition model to obtain the first feature vector.

[0129] Specifically, during the voiceprint registration stage, the sampling rate of the input registration voice needs to be regularized to 16000 Hz, and then the improved fbank feature extraction algorithm is used to extract the fbank features corresponding to the registration voice. The extracted fbank features corresponding to the registration voice are input into the cross-channel voiceprint recognition model to obtain the first feature vector, which is an N-dimensional feature vector.

[0130] T2, obtain the verification voice, adjust the sampling rate of the verification voice to the preset frequency, and use the improved fbank feature extraction algorithm to extract the fbank features corresponding to the verification voice. Input the fbank features corresponding to the verification voice into the cross-channel voiceprint recognition model to obtain the second feature vector.

[0131] Specifically, in the voiceprint verification stage, the sampling rate of the input verification voice also needs to be regularized to 16000 Hz, and then the improved fbank feature extraction algorithm is used to extract the fbank features corresponding to the verification voice. The extracted fbank features corresponding to the verification voice are input into the cross-channel voiceprint recognition model to obtain the second feature vector, which is an N-dimensional feature vector.

[0132] T3: perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

[0133] Specifically, the similarity comparison uses the cosine similarity algorithm, that is, the first eigenvector and the second eigenvector are cosine-scored. When the score is higher than the threshold, the registration voice and the verification voice are considered to be from the same person. The cosine scoring formula is as follows:

[0134]

[0135] Among them, θ is the angle between the first eigenvector A and the second eigenvector B, A and B are the first eigenvector and the second eigenvector, A i is the i-th vector of the first eigenvector A, B i is the i-th vector of the second eigenvector B, ||A|| is the modulus of the first eigenvector A, and ||B|| is the modulus of the second eigenvector B.

[0136] Further references Figure 7 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a cross-channel voiceprint recognition module device. Figure 6 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0137] The embodiment of the present application provides a cross-channel voiceprint recognition device, which uses a cross-channel voiceprint recognition model trained according to the above-mentioned cross-channel voiceprint recognition model training method, including:

[0138] The registration module 5 is configured to obtain the registration voice, adjust the sampling rate of the registration voice to a preset frequency, and use an improved fbank feature extraction algorithm to extract fbank features corresponding to the registration voice, and input the fbank features corresponding to the registration voice into the cross-channel voiceprint recognition model to obtain a first feature vector;

[0139] Verification module 6 is configured to obtain a verification voice, adjust the sampling rate of the verification voice to a preset frequency, and use an improved fbank feature extraction algorithm to extract fbank features corresponding to the verification voice, and input the fbank features corresponding to the verification voice into the cross-channel voiceprint recognition model to obtain a second feature vector;

[0140] The comparison module 7 is configured to perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

[0141] Reference below Figure 8 , which shows an electronic device (eg Figure 1 A schematic structural diagram of a computer device 800 (a server or terminal device as shown). Figure 8 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0142] like Figure 8 As shown, the computer device 800 includes a central processing unit (CPU) 801 and a graphics processing unit (GPU) 802, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 803 or the program loaded from the storage part 809 to the random access memory (RAM) 804. In the RAM 804, various programs and data required for the operation of the device 800 are also stored. The CPU 801, GPU 802, ROM 803 and RAM 804 are connected to each other through a bus 805. An input / output (I / O) interface 806 is also connected to the bus 805.

[0143] The following components are connected to the I / O interface 806: an input section 807 including a keyboard, a mouse, and the like; an output section 808 including a display such as a liquid crystal display (LCD), a speaker, and the like; a storage section 809 including a hard disk and the like; and a communication section 810 including a network interface card such as a LAN card or a modem. The communication section 810 performs communication processing via a network such as the Internet. A drive 811 may also be connected to the I / O interface 806 as needed. A removable medium 812, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, is installed in the drive 811 as needed, so that a computer program read therefrom can be installed into the storage section 809 as needed.

[0144] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 810, and / or installed from the removable medium 812. When the computer program is executed by the central processing unit (CPU) 801 and the graphics processing unit (GPU) 802, the above-mentioned functions defined in the method of the present application are executed.

[0145] It should be noted that the computer-readable medium described in this application may be a computer-readable signal medium or a computer-readable medium, or any combination thereof. Computer-readable media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer-readable media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution device, apparatus, or component. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution apparatus, device, or device. Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, or any suitable combination thereof.

[0146] Computer program code for performing the operations of the present application can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions and operations of the devices, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of code, and the module, program segment or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based device that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0148] The modules involved in the embodiments described in this application may be implemented in software or hardware, and may also be set in a processor.

[0149] As another aspect, the present application further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: collects a data set, the data set includes a first data set and a second data set based on the first channel and a third data set and a fourth data set based on the second channel, performs data cleaning and data enhancement processing on the data set to obtain training data, the training data includes first training data, second training data, third training data and fourth training data corresponding to the first data set, the second data set, the third data set and the fourth data set respectively; uses an improved fbank feature extraction algorithm to extract fbank features from the training data to obtain fbank features, the fbank features include first fbank features, second fbank features, third fbank features and fourth fbank features corresponding to the first training data, the second training data, the third training data and the fourth training data respectively; uses the second fbank features and the fourth fbank features to train a voiceprint recognition model respectively to obtain a first model and a second model, and the first model and the second model are constructed by combining a fusion structure to obtain a third model; uses the first fbank features to train the third model to obtain a fourth model, and uses the third fbank features to train the fourth model to obtain a cross-channel voiceprint recognition model.

[0150] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A training method for a cross-channel voiceprint recognition model, characterized in that: The following steps are involved: S1, collecting data sets, the data sets including a first data set and a second data set based on a first channel, and a third data set and a fourth data set based on the second channel, performing data cleaning and data enhancement processing on the data sets to obtain training data, the training data including first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively; S2, extracting fbank features from the training data using an improved fbank feature extraction algorithm to obtain fbank features, wherein the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively; S3, use the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively to obtain a first model and a second model, and the first model and the second model are constructed by combining the fusion structure to obtain a third model, and the fusion structure includes a first adder, a first pooling layer, a first convolution layer, a first normalization layer, a first activation function layer, a second convolution layer, a second pooling layer, a third convolution layer, a second normalization layer, a second activation function layer, a fourth convolution layer, a second adder, a third activation function layer, a first multiplier, a second multiplier and a third adder, wherein the first pooling layer, the first convolution layer, the first normalization layer, the first activation function layer, and the second convolution layer are connected in sequence, and the second pooling layer, the third convolution layer, the second normalization layer, the second activation function layer, and the fourth convolution layer are connected in sequence, and the output of the first model and the output of the second model are combined using the first adder. The first addition result is input into the first pooling layer and the second pooling layer respectively, and the first feature and the second feature are extracted by the first convolution layer, the first normalization layer, the first activation function layer, the second convolution layer and the third convolution layer, the second normalization layer, the second activation function layer and the fourth convolution layer respectively. The first feature and the second feature are added by the second adder to obtain a second addition result, and the second addition result is input into the third activation function layer. The output result of the third activation function layer is multiplied with the output of the first model and the output of the second model by the first multiplier and the second multiplier respectively to obtain a first multiplication result and a second multiplication result. The output result of the third activation function layer, the first multiplication result and the second multiplication result are added by the third adder to obtain the output of the third model; S4: Use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

2. The training method of the cross-channel voiceprint recognition model according to claim 1, characterized in that: Collecting a data set in step S1 specifically includes: collecting a first speech from the first channel; Adjusting the sampling rate of the first speech to a preset frequency to obtain the first data set; Extracting a portion of the first speech from the first data set as the second data set; collecting a second voice from the second channel; Adjusting the sampling rate of the second speech to a preset frequency to obtain the third data set; A portion of the second speech is extracted from the third data set as the fourth data set.

3. The training method for a cross-channel voiceprint recognition model according to claim 1, characterized in that: The data cleaning method in step S1 includes: removing silent segments in the speech by using a windowing method; the data enhancement method includes: adding noise, changing the speed or adding reverberation to the cleaned speech.

4. The training method for a cross-channel voiceprint recognition model according to claim 1, characterized in that: The improved fbank feature extraction algorithm in step S2 specifically includes: Pre-emphasize, frame, and window the speech signal in the training data; Performing Fourier transform on the pre-emphasized, framed, and windowed speech signal to obtain the spectrum of each frame of the speech signal, and taking the modulus square of the spectrum of the speech signal to obtain the power spectrum of the speech signal; Performing Mel filtering on the power spectrum to extract the Mel filtering output corresponding to the training data; Merge the Mel filter output corresponding to the second training data and the Mel filter output corresponding to the fourth training data, and calculate the mean and standard deviation; The Mel filter output corresponding to the training data is dynamically normalized according to the mean and standard deviation to obtain a dynamic normalized output: Wherein, M(t) is the Mel filter output corresponding to the training data, d(t) is the dynamic regularization output, μ and σ are the mean and standard deviation respectively; The logarithm of the dynamic warping output is taken to obtain the fbank feature.

5. The training method of the cross-channel voiceprint recognition model according to claim 1, characterized in that: The voiceprint recognition model includes Res2Net50. The loss function used in the training process of the cross-channel voiceprint recognition model is the cross entropy loss function, and the formula is as follows: Where L is the loss function, The model predicts the output, y is the label, and the value is 0 or 1.

6. The method for training a cross-channel voiceprint recognition model according to claim 1, characterized in that: The first activation function layer and the second activation function layer use RELU, and the third activation function layer uses sigmoid.

7. A training device for a cross-channel voiceprint recognition model, characterized in that: include: a training data acquisition module configured to collect data sets, the data sets including a first data set and a second data set based on a first channel, and a third data set and a fourth data set based on the second channel, and perform data cleaning and data enhancement processing on the data sets to obtain training data, the training data including first training data, second training data, third training data, and fourth training data corresponding to the first data set, the second data set, the third data set, and the fourth data set, respectively; a feature extraction module configured to extract fbank features from the training data using an improved fbank feature extraction algorithm to obtain fbank features, wherein the fbank features include a first fbank feature, a second fbank feature, a third fbank feature, and a fourth fbank feature corresponding to the first training data, the second training data, the third training data, and the fourth training data, respectively; The model construction module is configured to use the second fbank feature and the fourth fbank feature to train the voiceprint recognition model respectively to obtain a first model and a second model. The first model and the second model are combined through a fusion structure to obtain a third model. The fusion structure includes a first adder, a first pooling layer, a first convolution layer, a first normalization layer, a first activation function layer, a second convolution layer, a second pooling layer, a third convolution layer, a second normalization layer, a second activation function layer, a fourth convolution layer, a second adder, a third activation function layer, a first multiplier, a second multiplier and a third adder, wherein the first pooling layer, the first convolution layer, the first normalization layer, the first activation function layer and the second convolution layer are connected in sequence, the second pooling layer, the third convolution layer, the second normalization layer, the second activation function layer and the fourth convolution layer are connected in sequence, and the output of the first model and the output of the second model are combined using the The first adder performs addition to obtain a first addition result, and the first addition result is respectively input into the first pooling layer and the second pooling layer, and respectively extracted through the first convolution layer, the first normalization layer, the first activation function layer, the second convolution layer and the third convolution layer, the second normalization layer, the second activation function layer, and the fourth convolution layer to obtain the first feature and the second feature. The first feature and the second feature are added by the second adder to obtain a second addition result, and the second addition result is input into the third activation function layer. The output result of the third activation function layer is multiplied by the output of the first model and the output of the second model by the first multiplier and the second multiplier to obtain a first multiplication result and a second multiplication result. The output result of the third activation function layer, the first multiplication result, and the second multiplication result are added by the third adder to obtain the output of the third model; The model training module is configured to use the first fbank feature to train the third model to obtain a fourth model, and use the third fbank feature to train the fourth model to obtain a cross-channel voiceprint recognition model.

8. A cross-channel voiceprint recognition method, characterized in that: The cross-channel voiceprint recognition model trained by the training method of the cross-channel voiceprint recognition model according to any one of claims 1 to 6 comprises the following steps: T1, obtaining a registration voice, adjusting the sampling rate of the registration voice to a preset frequency, and extracting the fbank features corresponding to the registration voice using an improved fbank feature extraction algorithm, inputting the fbank features corresponding to the registration voice into the cross-channel voiceprint recognition model to obtain a first feature vector; T2, obtaining a verification voice, adjusting the sampling rate of the verification voice to a preset frequency, and extracting fbank features corresponding to the verification voice using an improved fbank feature extraction algorithm. The fbank features corresponding to the verification voice are input into the cross-channel voiceprint recognition model to obtain a second feature vector; T3: Perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

9. A cross-channel voiceprint recognition device, characterized in that: The cross-channel voiceprint recognition model trained by the training method of the cross-channel voiceprint recognition model according to any one of claims 1 to 6 comprises: a registration module configured to obtain a registration voice, adjust a sampling rate of the registration voice to a preset frequency, extract fbank features corresponding to the registration voice using an improved fbank feature extraction algorithm, and input the fbank features corresponding to the registration voice into the cross-channel voiceprint recognition model to obtain a first feature vector; a verification module configured to obtain a verification voice, adjust a sampling rate of the verification voice to a preset frequency, extract fbank features corresponding to the verification voice using an improved fbank feature extraction algorithm, and input the fbank features corresponding to the verification voice into the cross-channel voiceprint recognition model to obtain a second feature vector; The comparison module is configured to perform a similarity comparison on the first feature vector and the second feature vector to obtain a comparison result, and determine whether the verification voice and the registration voice belong to the same person based on the comparison result.

10. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images