Cross-channel content-independent speaker recognition method and system based on adversarial learning

Abstract

A cross-channel content-independent speaker recognition method and system based on adversarial learning, in the offline stage, by collecting speech samples of multiple speakers under multiple different channels to generate a data set, and training a channel classification model; and using the trained channel migration model to perform channel normalization processing on the input speech samples of the speaker to be tested under different channels, and using the trained single-channel speaker recognition unit to perform speaker recognition on the audio data collected by the user and containing the audio belonging to a specific speaker to be found, and find the audio belonging to the specific speaker. The present invention uses speech samples collected from multiple communication channels as training data, trains a channel migration model for speaker recognition, and migrates the speech channel features under different channels, so as to achieve the migration of speech samples under other communication channels to a unified speech channel, thereby completing the speaker recognition task and obtaining a higher recognition accuracy.

Description

Technical Field

[0001] The present invention relates to a technology in the field of speech recognition, specifically a cross-channel content-independent speaker recognition method and system based on adversarial learning, which can be used to recognize speech samples of a specific speaker in other channels when a speech sample of the speaker is collected in a specific communication channel. Background Art

[0002] With the rapid development of the Internet of Things and artificial intelligence technologies, efficient and accurate biometrics can make people's lives more convenient, and a large number of cumbersome identity authentication and permission authentication processes can be simplified and made seamless. Speaker recognition is a specific technical task in biometrics that identifies people based on human speech characteristics. Content-independent speaker recognition does not require the specific content of the speaker to be collected, and only needs to collect any speech content to accurately identify the speaker's identity.

[0003] However, due to the diversification of current acquisition equipment and audio signal encoding methods, there are large differences in the audio information obtained by different channels, which poses a major challenge to existing deep learning-based audio feature extraction and recognition methods. For example, the same audio of the same person collected using a high-performance microphone in public and using an Android phone and an Apple phone has different characteristic components in different frequency bands, which makes it impossible for existing recognition methods to perform cross-channel speaker recognition, that is, the recognition method trained based on speech samples collected using a microphone in public cannot be directly applied to the speaker recognition task of mobile phone recordings. In the actual application of speaker recognition, it is often only possible to collect speech samples of unknown speakers in a single channel, which makes the method unable to accurately identify the speaker's speech samples in other channels, limiting the application scenarios of the method. Summary of the invention

[0004] In view of the defects of the prior art that the training samples are limited and the classification accuracy is low, the present invention proposes a cross-channel content-independent speaker recognition method and system based on adversarial learning. Based on the speech samples collected from multiple communication channels as training data, the channel migration model of speaker recognition is trained, and the speech channel features under different channels are migrated, so as to achieve the migration of speech samples under other communication channels to a unified speech channel, thereby completing the speaker recognition task and obtaining a higher recognition accuracy.

[0005] The present invention is achieved through the following technical solutions:

[0006] The present invention relates to a cross-channel content-independent speaker recognition method based on adversarial learning. In an offline stage, after a data set is generated by collecting speech samples of multiple speakers under multiple different channels, a single-channel speaker recognition unit of a specific target channel is trained based on speech samples under a single channel in the data set, and a channel classification model is trained based on speech samples of multiple different channels in the data set; then a channel migration model is trained based on speech samples of the channel contained in the data set; in an online stage, the trained channel migration model is used to perform channel normalization processing on speech samples of speakers to be tested under different channels, and the trained single-channel speaker recognition unit is used to perform speaker recognition on audio data collected by a user and containing audio belonging to a specific speaker to be found, so as to find the audio belonging to the specific speaker.

[0007] The channel migration model adopts a total loss function obtained by weighted fusion of restoration loss, speaker recognition loss and channel discrimination loss, and is trained based on a stochastic gradient descent method.

[0008] The method specifically comprises the following steps:

[0009] Step 1: Collect the speaker's speech samples in the target channel to be identified, and obtain the collection of different channels C = {C1, C2, ..., C n}, the collection of speech samples in each channel Wherein: k represents the kth channel category.

[0010] Step 2: Use the Mel Frequency Cepstral Coefficient (MFCC) method to extract a single channel C k The time-frequency feature graph of all speech samples in, that is, each speech sample A r Corresponding to a time-frequency feature map F r , and accordingly obtain the feature sample set F of the corresponding speech sample collection, thereby removing the interference of the content on the subsequent recognition method to a certain extent and achieving content-independent speaker recognition.

[0011] Step 3: Collect the channel C k The corresponding feature sample set is recorded as use A randomly initialized convolutional neural network is trained with supervised learning. Specifically, the network parameters of the randomly initialized convolutional neural network are tuned using the stochastic gradient descent propagation method until it can achieve a single channel C k The high-precision recognition and classification of the speaker to which the speech sample belongs, that is, the speaker recognition model under the trained single channel is obtained

[0012] Step 4: Use the feature samples F corresponding to the different channel sets C obtained in step 2 to perform supervised learning training on another randomly initialized convolutional neural network. Specifically, use the stochastic gradient descent propagation method to tune the network parameters of the randomly initialized convolutional neural network until it can achieve high-precision recognition and classification of the communication channel to which the voice sample belongs, that is, obtain the channel classification model Model of the trained voice sample. channel .

[0013] Step 5: The training feature sample F of the random communication channel input Input the randomly initialized AutoEncoder model to get the output feature F output After that, the mean square error (MSE) is used to calculate the training feature sample F input And the output feature F output The difference is used as the restoration loss Loss1 to adjust the parameters of the randomly initialized autoencoder model so that the features obtained after the autoencoder are as similar as possible to the input features.

[0014] Step 6: The output feature F obtained in step 5 output Input the trained speaker recognition model obtained in step 3 Get the probability distribution P of speaker classification based on the model s After peaker, based on the speaker category labels given by the training samples, the cross entropy function is used to calculate the speaker recognition loss Loss2, which is used to adjust the parameters of the randomly initialized autoencoder model obtained in step 5, so that the features obtained after the autoencoder retain the feature information of the speaker identity as much as possible.

[0015] Step 7: The output feature F obtained in step 5 output Input the trained channel classification model obtained in step 4 channel , and obtain the channel classification probability distribution P of the input speech based on the model c After hannel, the input speech samples of any channel are all processed using C k The channel is used as the target label of the sample, and the cross entropy function is used to calculate the channel discrimination loss Loss3, which is used to randomly initialize the autoencoder model for parameter adjustment, so that the features after the autoencoder can counter the channel discrimination model, thereby realizing the input of non-C k The channel speech sample contains other channel feature information normalized to C k Channel information.

[0016] Step 8: Weighted fusion of restoration loss Loss1, speaker recognition loss Loss2 and channel discrimination loss Loss3 is used to train the autoencoder model. AE The total loss function Loss = α*Loss1+β*Loss2+γ*Loss3, where: hyperparameters α, β, and γ are determined according to the training sample set, and their setting requirements are to make the sizes of the three parts in the formula as close to the same order of magnitude as possible.

[0017] Step 9: Based on the stochastic gradient descent method, the total loss function obtained in step 8 is fed back to the parameters in the autoencoder model to train and obtain the optimal channel migration model. AE , used for recognition in the online stage.

[0018] Preferably, the present invention uses Android system mobile phone microphone collection, IOS system mobile phone signal collection, and open high-fidelity microphone collection to verify effectiveness.

[0019] The present invention relates to a cross-channel content-independent speaker recognition system based on adversarial learning for implementing the above method, comprising: a feature extraction unit, a channel recognition unit, a first single-channel speaker recognition unit, a first channel migration unit and a multi-channel speaker recognition unit, wherein: the feature extraction unit extracts a speech sample of a speaker to be recognized Perform feature extraction of Mel frequency cepstral coefficients to obtain features The channel identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample; the first single channel speaker identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample collected in the single channel. The speaker identity is identified to obtain the speaker identity result under a single channel; the first channel migration unit uses the built-in migration model to identify the speaker identity according to the feature Obtain channel-shifted features The multi-channel speaker recognition unit is based on the features Get Features The final speaker identity is obtained after recognition.

[0020] The multi-channel speaker recognition unit comprises a second channel migration unit and a second single-channel speaker recognition unit.

[0021] The speaker's speech sample There is no other speech sample of the speaker in the channel, but in channel C k The speaker's identity information can be identified.

[0022] Technical Effects

[0023] Compared with the shortcomings of the prior art that it cannot directly perform recognition or has very poor recognition accuracy, the present invention constrains the effectiveness of the channel migration model from three different levels: channel information, speaker identity information, and restoration similarity contained in the speech feature samples. The speech samples that have undergone channel migration can be directly applied to the speaker recognition model of the existing channel, thereby achieving higher channel migration accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A schematic diagram of a single channel speaker recognition unit;

[0025] Figure 2 Schematic diagram of channel classification model;

[0026] Figure 3 Schematic diagram of channel migration model;

[0027] Figure 4 The following is a flow chart of an embodiment. DETAILED DESCRIPTION

[0028] like Figure 4 As shown, this embodiment relates to a cross-channel content-independent speaker recognition method based on adversarial learning, which specifically includes:

[0029] Step 1: Collect the speaker's speech samples in the target channel to be identified, define the collection of different channels as C, and define the collection of speech samples in each channel as Where k represents the kth channel category, that is: C = {C1, C2, ..., C n},

[0030] Step 2: Use the Mel Frequency Cepstral Coefficient (MFCC) method to analyze the single channel C k All speech samples in are feature extracted to obtain their time-frequency feature graph, that is, each speech sample A r There is a corresponding time-frequency feature map F r , and the corresponding feature sample set F is obtained.

[0031] The length and width of the time-frequency domain feature map are set according to the audio length and sampling frequency, and too long speech samples need to be sliced.

[0032] In this embodiment, a 64*16 MFCC feature map is generated for subsequent convolutional neural network processing.

[0033] Step 3: Use channel C k The training feature samples in Based on the randomly initialized convolutional neural network, supervised learning is used to tune the model's randomly initialized parameters using the stochastic gradient descent propagation method to meet the requirements of single channel C k High-precision recognition and classification of the speaker to whom the speech sample belongs, that is, obtaining a speaker recognition model under a single channel

[0034] The randomly initialized convolutional neural network uses two convolutional layers, uses the ReLU function for activation, performs the maximum pooling function for pooling, and finally uses the fully connected layer for classification, which can ultimately achieve a recognition accuracy of more than 95%.

[0035] Step 4: Using the feature samples F in different channel sets C, on the basis of another randomly initialized convolutional neural network, using supervised learning, the random gradient descent propagation method is used to tune the model's randomly initialized parameters to meet the requirements of achieving high-precision recognition and classification of the communication channel to which the voice samples belong, that is, to obtain the channel classification model Model of the voice samples. channel .

[0036] The other randomly initialized convolutional neural network uses two convolutional layers, uses the ReLU function for activation, performs the maximum pooling function for pooling, and finally uses the fully connected layer for classification, which can ultimately achieve a recognition accuracy of more than 95%.

[0037] like Figure 2 As shown, the channel classification model includes: two extraction units consisting of a convolution layer, an activation layer and a pooling layer, and a fully connected layer, wherein: the convolution layer in the first extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a feature tensor of 32*8*16 shape; the convolution layer in the second extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a 32*8*16 shape feature tensor; The convolution layer uses 32 3×3 convolution kernels to convolve the input map with a step size of 1 pixel and fill with 1 pixel. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to pool and obtain a feature tensor of shape 16*4*32; the fully connected layer serializes the tensor into a feature vector of shape 1*1*2048, which is then input into the 2048*n fully connected layer and activated using the softmax function, where n is the number of speaker categories to be classified.

[0038] Step 5: Based on the randomly initialized AutoEncoder model, the training feature sample F of the random communication channel is inputInput the model to obtain the output feature F output , and then use the mean square error (MSE) to calculate the difference between the two, which is used as the restoration loss Loss1. The purpose of this step is to require that the features obtained after the autoencoder are as similar as possible to the input features.

[0039] The randomly initialized autoencoder model includes two symmetrical convolution upsampling functions and a ReLu function, and is finally activated by a Sigmoid function to obtain an autoencoding result of the input feature image.

[0040] Step 6: The output feature F obtained in step 5 output Input speaker recognition model Get the probability distribution P of speaker classification based on the model s peaker, and then based on the speaker category label given by the training sample, the cross entropy function is used to calculate the speaker recognition loss Loss2. The purpose of this step is to require that the features obtained after the autoencoder retain the feature information of the speaker identity information as much as possible.

[0041] Step 7: The output feature F obtained in step 5 output Input channel classification model channel , and obtain the channel classification probability distribution P of the input speech based on the model c hannel, then no matter which channel the input speech sample belongs to, the target label is C k The channel is used as the label of the sample, and then the cross entropy function is used to calculate the channel discrimination loss Loss3. The purpose of this step is to require the features of the autoencoder to be able to resist the channel discrimination model, so as to achieve the input of non-C k The channel speech sample contains other channel feature information normalized to C k Channel information.

[0042] Step 8: Based on the weighted summation method, the Loss1, Loss2, and Loss3 obtained in steps 5, 6, and 7 are combined into the training autoencoder model Model. AE The required total loss function Loss = α*Loss1+β*Loss2+γ*Loss3, where α, β, and γ are hyperparameters that need to be determined based on the training sample set. The setting requirement is to make the sizes of the three parts in the formula as close to the same order of magnitude as possible. In this implementation case, the three hyperparameters are set to 1, 0.05, and 0.01, respectively.

[0043] Step 9: Based on the stochastic gradient descent method, the total loss function obtained in step 8 is fed back to the parameters in the autoencoder model to train and obtain the optimal channel migration model. AF , used for recognition in the online stage.

[0044] like Figure 4 As shown, a cross-channel content-independent speaker recognition system based on adversarial learning involved in this embodiment includes: a feature extraction unit, a channel recognition unit, a first single-channel speaker recognition unit, a first channel migration unit and a multi-channel speaker recognition unit, wherein: the feature extraction unit extracts the speaker speech sample to be recognized Perform feature extraction of Mel frequency cepstral coefficients to obtain features The channel identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample; the first single channel speaker identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample collected in the single channel. The speaker identity is identified to obtain the speaker identity result under a single channel; the first channel migration unit uses the built-in migration model according to the feature Input channel migration model AE , obtain the features after channel migration The multi-channel speaker recognition unit is based on the features Features obtained after input channel migration model The final speaker identity is obtained after recognition.

[0045] The feature extraction unit includes: a fast Fourier transform unit, a Mel filter bank unit and a discrete cosine transform unit, wherein: the fast Fourier transform unit performs fast Fourier transform processing according to the speech audio information to obtain the audio spectrum information result, the Mel filter bank unit performs filtering processing according to the audio spectrum information to obtain the audio spectrum intensity coefficient results at different frequencies, and the discrete cosine transform unit performs decorrelation and dimensionality reduction processing according to the grouped audio spectrum intensity coefficient information, and retains the results of the specified spectrum grouping to obtain the MFCC feature spectrum result of the input audio signal.

[0046] The multi-channel speaker recognition unit comprises a second channel migration unit and a second single-channel speaker recognition unit.

[0047] like Figure 1As shown, the first and second single-channel speaker recognition units each include: two extraction units consisting of a convolution layer, an activation layer and a pooling layer, and a fully connected layer, wherein: the convolution layer in the first extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a feature tensor of 32*8*16 shape; the second extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a 32*8*16 shape feature tensor; The convolution layer in the unit uses 32 3×3 convolution kernels, and convolves the input map with a step size of 1 pixel and fills it with 1 pixel. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function for pooling to obtain a feature tensor of shape 16*4*32; the fully connected layer serializes the tensor into a feature vector of shape 1*1*2048, and then inputs it into the 2048*n fully connected layer and activates it using the softmax function, where n is the number of speaker categories to be classified.

[0048] like Figure 3 As shown, the first and second channel migration units are both equipped with a migration model, namely, a channel migration model; the model is an autoencoder structure, including: an encoder consisting of two convolutional layers and one activation layer, a decoder consisting of two deconvolutional layers and two activation layers, wherein: the first convolutional layer of the encoder uses 16 3×3 convolutional kernels, and uses a step size of 2 pixels to fill 1 pixel to convolve the input map. After the shape of the input map is 64*16*1, the first ReLu function is used for activation, and the second convolutional layer uses 32 3×3 convolutional kernels, and uses a step size of The input map is convolved by padding 2 pixels to 1 pixel to obtain a coded tensor of 16*4*32 shape; the first deconvolution layer of the decoder uses 16 3×3 convolution kernels, and uses a stride of 2 pixels to pad 1 pixel and an external padding of 1 to deconvolve the input map. That is, after upsampling, the first activation layer is activated using the ReLu function, and the second deconvolution layer uses a 3×3 convolution kernel, and uses a stride of 2 pixels to pad 1 pixel and an external padding of 1 to deconvolve the input map. Finally, the feature map of 64*16*1 shape is obtained by decoding and restoration.

[0049] After specific practical experiments, in the Ubuntu system, the simulation verification program was written in Python3.8, and GeForce RTX3080 was used as the specific environment setting for training hardware. Based on the AISHELL-ASR0009-OS1 open source Chinese speech database data set training, the above device / method was started / run, and the experimental results that can be obtained are shown in the following table:

[0050] Single speaker recognition The present invention Android 95.97％ 91.71％ iOS 43.81％ 82.65％ MIC 32.04％ 81.65％

[0051] The data set used in this embodiment provides audio samples of 400 different speakers under three channels: a high-fidelity microphone (44.1kHz, 16-bit); an Android phone (16kHz, 16-bit); and an iOS phone (16kHz, 16-bit). The first column shows the accuracy of the data after being directly classified by a single speaker recognition model obtained by training with data collected by the Android system without channel migration according to the present invention. It can be seen that the recognition accuracy of the model for other channels is low. The second column shows that after channel migration based on the present invention, although the recognition accuracy of the samples collected by the Android system decreases slightly, the recognition accuracy of other channels increases significantly.

[0052] Compared with the prior art, after the channel migration of the present invention, the recognition accuracy of audio data from different channel sources in the original speaker recognition model is greatly improved, from 30%-40% to more than 80%, and the performance improvement exceeds 100%.

[0053] The above-mentioned specific implementation can be partially adjusted in different ways by those skilled in the art without departing from the principle and purpose of the present invention. The protection scope of the present invention shall be based on the claims and shall not be limited by the above-mentioned specific implementation. Each implementation scheme within its scope shall be subject to the constraints of the present invention.

Claims

1. A cross-channel content-independent speaker recognition method based on adversarial learning, characterized in that: In the offline stage, after generating a data set by collecting speech samples of multiple speakers in multiple different channels, ① the single channel speaker recognition unit of a specific target channel is trained based on the speech samples in a single channel in the data set ② the channel classification model is trained based on the speech samples in multiple different channels in the data set; Then, the channel migration model is trained based on the speech samples of the channels included in the data set; In the online stage, the trained channel migration model is used to perform channel normalization processing on the input speech samples of the speaker to be tested under different channels, and the trained single-channel speaker recognition unit is used to perform speaker recognition on the audio data collected by the user and containing the audio belonging to the specific speaker to be found, and the audio belonging to the specific speaker is found; The single channel speaker recognition unit comprises: two extraction units respectively consisting of a convolutional layer, an activation layer and a pooling layer, and a fully connected layer; The channel migration model is an autoencoder structure, including: an encoder consisting of two convolutional layers and one activation layer, and a decoder consisting of two deconvolutional layers and two activation layers; The cross-channel content-independent speaker recognition method specifically includes: Step 1: Collect the speaker's speech samples in the target channel to be identified, and obtain the collection of different channels C = {C1, C2, ..., C n }, the collection of speech samples in each channel Where: k represents the kth channel category; Step 2: Use the Mel Frequency Cepstral Coefficient (MFCC) method to extract a single channel C k The time-frequency feature graph of all speech samples in, that is, each speech sample A r Corresponding to a time-frequency feature map F r , and accordingly obtain the feature sample set F corresponding to the speech sample collection; Step 3: Collect the channel C k The corresponding feature sample set is recorded as use A randomly initialized convolutional neural network is trained with supervised learning. Specifically, the network parameters of the randomly initialized convolutional neural network are tuned using the stochastic gradient descent propagation method until it can achieve a single channel C k The high-precision recognition and classification of the speaker to which the speech sample belongs, that is, the speaker recognition model under the trained single channel is obtained Step 4: Use the feature samples F corresponding to the different channel sets C obtained in step 2 to perform supervised learning training on another randomly initialized convolutional neural network. Specifically, use the stochastic gradient descent propagation method to tune the network parameters of the randomly initialized convolutional neural network until it can achieve high-precision recognition and classification of the communication channel to which the voice sample belongs, that is, obtain the channel classification model Model of the trained voice sample. channel ; Step 5: The training feature sample F of the random communication channel input Input the randomly initialized AutoEncoder model to get the output feature F output After that, the mean square error (MSE) is used to calculate the training feature sample F input And the output feature F output The difference is used as the restoration loss Loss1 to adjust the parameters of the randomly initialized autoencoder model; Step 6: The output feature F obtained in step 5 output Input the trained speaker recognition model obtained in step 3 Get the probability distribution P of speaker classification based on the model s After peaker, based on the speaker category labels given by the training samples, the cross entropy function is used to calculate the speaker recognition loss Loss2, which is used to adjust the parameters of the randomly initialized autoencoder model obtained in step 5, so that the features obtained after the autoencoder retain the feature information of the speaker identity information as much as possible; Step 7: The output feature F obtained in step 5 output Input the trained classification model obtained in step 4 channel , we get the channel classification probability distribution P of the input speech based on the model c After hannel, the input speech samples of any channel are all processed using C k The channel is used as the target label of the sample, and the cross entropy function is used to calculate the channel discrimination loss Loss3, which is used to randomly initialize the autoencoder model for parameter adjustment, so that the features after the autoencoder can counter the channel discrimination model, thereby realizing the input of non-C k The channel speech sample contains other channel feature information normalized to C k Channel information; Step 8: Weighted fusion of restoration loss Loss1, speaker recognition loss Loss2 and channel discrimination loss Loss3 is used to train the autoencoder model Model AE The total loss function Loss = α*Loss1+β*Loss2+γ*Loss3, where: hyperparameters α, β, and γ are determined based on the training sample set; Step 9: Based on the stochastic gradient descent method, the total loss function obtained in step 8 is fed back to the parameters in the autoencoder model to train and obtain the optimal channel migration model. AE , used for recognition in the online stage.

2. The cross-channel content-independent speaker recognition method based on adversarial learning according to claim 1 is characterized in that: The channel migration model adopts a total loss function obtained by weighted fusion of restoration loss, speaker recognition loss and channel discrimination loss, and is trained based on a stochastic gradient descent method.

3. The cross-channel content-independent speaker recognition method based on adversarial learning according to claim 1 is characterized in that: The channel classification model includes: two extraction units consisting of a convolution layer, an activation layer and a pooling layer, and a fully connected layer, wherein: the convolution layer in the first extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a feature tensor of 32*8*16 shape; the convolution layer in the second extraction unit The layer uses 32 3×3 convolution kernels, and convolves the input map with a step size of 1 pixel and fills it with 1 pixel. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to pool and obtain a feature tensor of shape 16*4*32; the fully connected layer serializes the tensor into a feature vector of shape 1*1*2048, which is then input into the 2048*n fully connected layer and activated with the softmax function, where n is the number of speaker categories to be classified.

4. A cross-channel content-independent speaker recognition system based on adversarial learning that implements any of the methods described in claims 1 to 3, characterized in that: include: A feature extraction unit, a channel identification unit, a first single channel speaker identification unit, a first channel migration unit and a multi-channel speaker identification unit, wherein: the feature extraction unit extracts a speech sample of a speaker to be identified Perform feature extraction of Mel frequency cepstral coefficients to obtain features The channel identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample; the first single channel speaker identification unit can identify the channel used by the input voice sample according to the characteristics of the input voice sample collected in the single channel. The speaker identity is identified to obtain the speaker identity result under a single channel; the first channel migration unit uses the built-in migration model to identify the speaker identity according to the feature Obtain channel-shifted features The multi-channel speaker recognition unit is based on the features Get Features The final speaker identity is obtained after identification; The speaker's speech sample There is no other speech sample of the speaker in the channel, but in channel C k The speaker's identity information can be identified.

5. The cross-channel content-independent speaker recognition system based on adversarial learning according to claim 4 is characterized in that: The feature extraction unit includes: a fast Fourier transform unit, a Mel filter bank unit and a discrete cosine transform unit, wherein: the fast Fourier transform unit performs fast Fourier transform processing according to the speech audio information to obtain the audio spectrum information result, the Mel filter bank unit performs filtering processing according to the audio spectrum information to obtain the audio spectrum intensity coefficient results at different frequencies, and the discrete cosine transform unit performs decorrelation and dimensionality reduction processing according to the grouped audio spectrum intensity coefficient information, and retains the results of the specified spectrum grouping to obtain the MFCC feature spectrum result of the input audio signal.

6. The cross-channel content-independent speaker recognition system based on adversarial learning according to claim 4 is characterized in that: The multi-channel speaker recognition unit comprises a second channel migration unit and a second single channel speaker recognition unit, wherein: The first and second single-channel speaker recognition units each include: two extraction units respectively composed of a convolution layer, an activation layer and a pooling layer, and a fully connected layer, wherein: the convolution layer in the first extraction unit uses 16 3×3 convolution kernels, and uses a step size of 1 pixel to fill 1 pixel to convolve the input map. The shape of the input map is 64*16*1. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to perform pooling to obtain a feature tensor of 32*8*16 shape; the second extraction unit The convolution layer in the element uses 32 3×3 convolution kernels, and convolves the input map with a step size of 1 pixel and fills it with 1 pixel. After being activated by the ReLu function of the activation layer, the pooling layer uses a 2×2 maximum pooling function to pool and obtain a feature tensor of shape 16*4*32; the fully connected layer serializes the tensor into a feature vector of shape 1*1*2048, and then inputs it into the 2048*n fully connected layer and activates it with the softmax function, where n is the number of speaker categories to be classified.

7. The cross-channel content-independent speaker recognition system based on adversarial learning according to claim 6 is characterized in that: The first channel migration unit and the second channel migration unit both have built-in migration models, namely, channel migration models; The model is an autoencoder structure, including: an encoder consisting of two convolutional layers and an activation layer, a decoder consisting of two deconvolutional layers and two activation layers, where: the first convolutional layer of the encoder uses 16 3×3 convolution kernels, and uses a step size of 2 pixels to fill the input map with 1 pixel to convolve the input map. After the shape of the input map is 64*16*1, the first ReLu function is used for activation, and the second convolutional layer uses 32 3×3 convolution kernels, and uses a step size of 2 pixels to fill the input map with 1 pixel. Convolution is performed to obtain a coded tensor of 16*4*32 shape; the first deconvolution layer of the decoder uses 16 3×3 convolution kernels, and uses a step size of 2 pixels to fill 1 pixel and an external padding of 1 to deconvolve the input map. That is, after upsampling, the first activation layer is activated using the ReLu function, and the second deconvolution layer uses a 3×3 convolution kernel, and uses a step size of 2 pixels to fill 1 pixel and an external padding of 1 to deconvolve the input map. Finally, decoding and restoration obtain a feature map of 64*16*1 shape.

Images