A Speaker Recognition Method Based on Channel Attention in a Communication Scenario

Abstract

The present application relates to a speaker recognition method based on channel attention in a communication scenario. The method includes: constructing a speaker recognition model, which includes a feature extraction backbone network and a speaker classification network connected in sequence; embedding a channel attention mechanism based on cross-network layer feature aggregation into the feature extraction backbone network in the form of multiple channel attention network modules, and the channel attention network module includes a learnable dictionary encoding unit and an information aggregation unit; optimizing and training the speaker recognition model embedded with the channel attention network module, and using the trained speaker recognition model to perform the speaker recognition task in the communication scenario. This method can accurately perceive the importance of each channel feature in the network by representing hidden layer feature information at multiple levels, so as to perform feature selection and differential modeling more efficiently, which has important value for improving the discriminability of speaker representations and the accuracy of speaker recognition.

Description

Technical Field

[0001] The present application relates to the technical field of speaker recognition, and in particular to a speaker recognition method based on channel attention in a communication scenario. Background Art

[0002] Speaker recognition is the task of determining whether a given speech belongs to a specific speaker. It has wide applications in scenarios such as criminal investigation and surveillance, suspect voice retrieval, and security access control in high-sensitivity locations. With the rise of deep learning in recent years, the most advanced speaker recognition systems are mainly implemented through neural networks. The key is to extract highly discriminative speaker representation vectors from speech by designing efficient neural networks. In the speaker representation extraction process, mainstream methods typically use speech spectrogram features as network input and use convolution algorithms to capture voiceprint clues in the time and frequency domains. This involves extracting speaker features in multiple channel dimensions. However, in scenarios such as communications, due to factors such as device and channel differences, the features of different channel dimensions of the neural network show different importance as the input samples change dynamically. Treating these differentiated features in the same way will result in insufficient speaker representation discrimination, limiting the promotion of speaker recognition technology in communication scenarios.

[0003] Currently, attention mechanisms are primarily used in speaker recognition to address these issues, including compressed-excitation attention and global context vector attention. The compressed-excitation attention mechanism offers a simple algorithm, easy deployment, and excellent channel feature selection capabilities. However, it overly simplifies the representation of features in intermediate layers of the network and lacks representation of multi-level information. When training data contains significant amounts of irrelevant noise, the model is prone to overfitting, resulting in poor generalization performance. The global context vector attention mechanism offers high computational efficiency and pays greater attention to low-frequency features in the time-frequency domain that contain richer speaker cues. However, in the presence of strong noise, this method is prone to introducing high-energy noise into the global context acquisition process, leading to biased estimates of the importance of different channel features, affecting attention allocation, and reducing model robustness. Furthermore, these two methods fail to fully consider the utilization of multi-level information within neural networks, including multiple perspectives within a single layer and multiple semantics across multiple layers. The effectiveness of attention needs to be further improved, resulting in limited improvements in speaker representation discrimination and speaker recognition accuracy. Summary of the Invention

[0004] Based on this, it is necessary to address the above technical issues and provide a speaker recognition method based on channel attention in communication scenarios. This method can accurately perceive the importance of each channel feature in the network by representing hidden feature information at multiple levels, thereby performing feature selection and differentiated modeling more efficiently. It is of great value in improving the discriminability of speaker representation and the accuracy of speaker recognition.

[0005] A speaker recognition method based on channel attention in a communication scenario, the method comprising:

[0006] Build a speaker recognition model, which consists of a representation extraction backbone network and a speaker classification network connected in sequence;

[0007] The channel attention mechanism based on cross-layer feature aggregation is embedded in the representation extraction backbone network in the form of multiple channel attention network modules. The channel attention network module includes a learnable dictionary encoding unit and an information aggregation unit. The learnable dictionary encoding unit is designed to obtain a comprehensive representation of features within a specific network layer in the model, and is used to encode the acoustic features of the previous network layer input to obtain an encoding vector. The information aggregation unit is designed to obtain a global information representation of cross-layer features in the model, and is used to aggregate the encoding vectors of multiple network layers to obtain a global channel information representation of the current network layer. The acoustic features of the previous network layer input are calibrated based on the scaling coefficients and translation coefficients calculated from the global channel information representation, and the calibrated acoustic features are used as the input of the next network layer.

[0008] Obtain a speaker speech dataset and input it into a speaker recognition model embedded with a channel attention network module for optimization training. The model parameters are optimized and updated by the speaker classification network in conjunction with loss calculation until a trained speaker recognition model is obtained.

[0009] The trained speaker recognition model is used to perform speaker recognition tasks in communication scenarios.

[0010] In one embodiment, the representation extraction backbone network in the model includes two processes: frame-level speaker representation extraction and segment-level speaker representation extraction; wherein, the frame-level speaker representation extraction includes four residual stages, and each residual stage consists of four residual network blocks, and the channel attention network module is embedded after the last residual network block of each residual stage; the segment-level speaker representation extraction consists of a statistical pooling layer and two fully connected networks connected in sequence; the speaker classification network is a one-layer AM-Softmax network.

[0011] In one embodiment, all networks in the model are built based on the TensorFlow deep learning framework and optimized using the Adam optimizer, and the model is trained using the AM-softmax loss function.

[0012] In one embodiment, the model training set consists of the VoxCeleb 1 / 2 dataset and uses the data augmentation algorithm provided by the Kaldi toolkit. The test set includes the SITW dataset and the VOiCES dataset.

[0013] In one embodiment, the learnable dictionary encoding unit encodes the acoustic features input by the previous network layer to obtain an encoded vector, including:

[0014] Assume that the acoustic features input by the previous network layer are ; where R represents the real number space, and the superscripts T 、 F and C represent the time dimension, frequency dimension, and channel dimension respectively;

[0015] The learnable dictionary encoding unit first performs channel dimensionality reduction on through the convolution operation of to obtain the dimension-reduced acoustic features , and the expression is:

[0016] ;

[0017] where and represent the weight matrix and bias vector of respectively, and N represents the total number of preset dictionary components;

[0018] Then, merge the frequency dimensions of and into the channel dimension to obtain the acoustic feature vector of T frames and T the dimension-reduced acoustic feature vector of frames; where is the number of frames, and ;

[0019] At the same time, define three sets of learnable parameter sets, which are the mean vector , the projection vector , and the weight coefficient , and initialize them as random vectors, all-1 vectors, and 0 respectively; where represents the dictionary component serial number;

[0020] According to the three sets of learnable parameter sets, perform learnable dictionary encoding on to obtain the encoded vector finally output by the learnable dictionary encoding unit, which is expressed as:

[0021] ;

[0022] ;

[0023] ;

[0024] Among them, the encoding vector Depend on N dictionary component vectors composition, express Assigned to The weight of each dictionary component; 、 and Respectively represent i mean vectors, projection vectors and weight coefficients; for The transpose of for The transpose of .

[0025] In one embodiment, the information aggregation unit aggregates the coding vectors of multiple network layers to obtain a global channel information representation of the current network layer, including:

[0026] The definition memory unit aggregates the The feature representation of the layer network is calculated as follows:

[0027] ;

[0028] in, Before Aggregate features of layer networks, Indicates the The encoding vector output by the layer network, Before Aggregate features of layer networks, Represents the nonlinear projection process, and the calculation formula is as follows:

[0029] ;

[0030] in, 、 and b are both learnable parameters, representing the first layer weight matrix, the second layer weight matrix and the bias vector of the nonlinear projection process, respectively. for The transpose of x is the input vector, is the activation function;

[0031] For Layer network, aggregated global channel information representation The expression is:

[0032] ;

[0033] ;

[0034] in, Represents the feature dimension splicing operation, express and The splicing result, and Respectively The weight matrix and bias vector, for The transpose of .

[0035] In one embodiment, the information aggregation unit calibrates the acoustic features inputted by the previous network layer according to the scaling coefficient and the translation coefficient calculated by the global channel information representation, and uses the calibrated acoustic features as the input of the next network layer, including:

[0036] According to Global channel information representation obtained by layer network aggregation , calculate the Scaling factor of the layer network and translation coefficients , respectively expressed as:

[0037] ;

[0038] ;

[0039] in, represents the sigmoid activation function, and Respectively The weight matrix and bias vector, and Respectively The weight matrix and bias vector, and Respectively and Bias;

[0040] Using zoom factors and translation coefficients For Acoustic features of the layer network input Perform feature calibration and convert the calibrated acoustic features into As the first Input of the layer network, calibrated acoustic features It is expressed as:

[0041] .

[0042] In one embodiment, the acoustic features are 64-dimensional logarithmic Mel filter bank energy features. After being processed by the Kaldi toolkit, the acoustic features are randomly cropped to a length of 2 to 4 seconds, and 64 speech segments with the same number of frames are grouped into a mini-batch.

[0043] The above-mentioned speaker recognition method based on channel attention in a communication scenario has the following beneficial effects:

[0044] 1. The channel attention mechanism based on cross-network layer feature aggregation is embedded in the speaker recognition model in the form of multiple channel attention network modules. Based on the learnable dictionary encoding unit in this network module, a comprehensive representation of the features within a specific network layer can be extracted, so that the speaker representation vector has the ability to represent the input feature map from multiple perspectives. In addition, based on the information aggregation unit in this network, features across network layers can be integrated to obtain a richer and more comprehensive feature representation. This can enhance the model's ability to accurately perceive the importance of different channel features and improve the model's discriminability of speaker representation in communication scenarios.

[0045] 2. In the information aggregation unit, by adaptively calculating weighting and bias coefficients through more accurate feature representation and integrating scaling and translation operations in the feature calibration mechanism, attention can be applied from different levels, making the attention mechanism a more efficient feature selection process, which can effectively improve the accuracy of speaker recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 1 is a flow chart of a speaker recognition method based on channel attention in a communication scenario according to an embodiment;

[0047] Figure 2 Schematic diagram of the overall architecture of a channel attention network module in one embodiment;

[0048] Figure 3 Schematic diagram of the structure of a learnable dictionary encoding unit in one embodiment;

[0049] Figure 4 Schematic diagram of the structure of an information aggregation unit in one embodiment;

[0050] Figure 5 Schematic diagram of the embedding deployment of the channel attention network module in a speaker recognition model in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] In one embodiment, Figure 1 As shown, a speaker recognition method based on channel attention in a communication scenario is provided, comprising the following steps:

[0053] Step S1: construct a speaker recognition model, which consists of a representation extraction backbone network and a speaker classification network connected in sequence.

[0054] In step S2, the channel attention mechanism based on cross-network layer feature aggregation is embedded in the representation extraction backbone network in the form of multiple channel attention network modules; the channel attention network module includes a learnable dictionary encoding unit and an information aggregation unit; wherein, the learnable dictionary encoding unit is intended to obtain a comprehensive representation of the features within a specific network layer in the model, and is used to encode the acoustic features of the previous network layer input to obtain a coding vector; the information aggregation unit is intended to obtain a global information representation of the cross-network layer features in the model, and is used to aggregate the coding vectors of multiple network layers to obtain a global channel information representation of the current network layer, and based on the scaling coefficient and translation coefficient calculated from the global channel information representation, the acoustic features of the previous network layer input are feature calibrated, and the calibrated acoustic features are used as the input of the next network layer.

[0055] Specifically, the overall architecture of the channel attention network module is as follows: Figure 2 As shown in the figure, this network module can provide a modular and efficient channel attention method under the condition of controlling the limited growth of model parameters and computational complexity. It can be embedded in the network layer of any speaker recognition model in the form of multiple network modules to achieve efficient use of important features in the network, thereby enhancing the distinguishing ability of the extracted speaker representation and improving the recognition performance of the model. Figure 2 As shown, for any neural network hidden layer 1 in the speaker recognition model, its output First, input the learnable dictionary encoding unit to obtain the encoding vector , and according to the previous Aggregate features of layer networks , and then and Input the information aggregation unit to calculate the scaling coefficient and translation coefficient respectively, and The feature calibration is performed and then used as the input of the neural network hidden layer 2. After the existing speaker recognition model is modified in the above way, a more accurate speaker representation vector can be extracted for the speaker recognition task.

[0056] The structure of the learnable dictionary encoding unit is as follows Figure 3 As shown in , the specific process of encoding the acoustic features input by the previous network layer to obtain the encoding vector includes:

[0057] Assume that the acoustic features of the previous network layer input are ;in, R represents the real number space, the superscript T 、 F and C Represent the time dimension, frequency dimension and channel dimension respectively;

[0058] The learnable dictionary encoding unit first passes The convolution operation of Perform channel dimensionality reduction to obtain acoustic features after dimensionality reduction , the expression is:

[0059] ;

[0060] in, and Respectively The weight matrix and bias vector, N Indicates the total number of preset dictionary components;

[0061] Then, and The frequency dimension is merged into the channel dimension, and we get T Acoustic feature vector of the frame as well as T Acoustic feature vector after frame dimensionality reduction ;in, is the number of frames, and ;

[0062] At the same time, three sets of learnable parameter sets are defined, namely, the mean vector , projection vector and weight coefficients , and are initialized to random vector, all 1 vector and 0 respectively; among them, Indicates the dictionary component number;

[0063] According to three sets of learnable parameter sets Perform learnable dictionary encoding to obtain the encoding vector finally output by the learnable dictionary encoding unit , expressed as:

[0064] ;

[0065] ;

[0066] ;

[0067] Among them, the encoding vector Depend on N dictionary component vectors Composed of denotes the weight assigned to the th dictionary component; , and respectively denote the i th mean vector, projection vector, and weight coefficient; is transpose of is transpose of N dictionary components are introduced and used as the central vectors to calculate weighted statistical information, enabling the speaker representation vector to represent the input feature map from multiple perspectives.

[0068] The structure of the information aggregation unit is shown in Figure 4 as follows. The specific process of its feature aggregation and feature calibration includes:

[0069] First, define that the memory unit aggregates the feature representations from the previous layers of the network, and its calculation formula is as follows:

[0070] [[ID=,38]] ;

[0071] where denotes the aggregated feature of the previous layers of the network, denotes the encoded vector output by the th layer of the network, denotes the aggregated feature of the previous layers of the network, denotes the non - linear projection process, and its calculation formula is as follows:

[0072] ;

[0073] where , and b are all learnable parameters, respectively representing the first - layer weight matrix, second - layer weight matrix, and bias vector of the non - linear projection process, is transpose of x is the input vector, is the activation function;

[0074] For the th layer of the network, the expression of the globally aggregated channel information is:

[0075] ;

[0076] ;

[0077] in, Represents the feature dimension splicing operation, express and The splicing result, and Respectively The weight matrix and bias vector, for In the above feature aggregation process, the memory unit recursively calculates the global feature representation of each shallow network, thereby obtaining complementary multi-level semantic features across network layers. Compared with the feature representation extracted from a single-layer network, it is more robust and richer in information.

[0078] The feature calibration mechanism aims to select attention for each channel feature based on its importance. The core of the mechanism is how to combine the global channel information representation obtained by the information aggregation unit to calculate the feature scaling coefficient and translation coefficient. Global channel information representation obtained by layer network aggregation , calculate the Scaling factor of the layer network and translation coefficients , respectively expressed as:

[0079] ;

[0080] ;

[0081] in, represents the sigmoid activation function, and Respectively The weight matrix and bias vector, and Respectively The weight matrix and bias vector, and Respectively and Bias;

[0082] Using zoom factors and translation coefficients For the first Acoustic features of the layer network input Perform feature calibration and convert the calibrated acoustic features into As the first Input of the layer network, calibrated acoustic features It is expressed as:

[0083] 。

[0084] During the above feature calibration process, by weighting in the channel dimension, higher weights can be assigned to important features, and biases can be assigned more significant translation amounts to important features, so as to perform multi-faceted feature calibration operations on more important features in the channel dimension, thereby enhancing the effectiveness of attention.

[0085] Specifically, this application will embed a channel attention mechanism based on cross-network layer feature aggregation in the form of multiple channel attention network modules into the Figure 5 speaker recognition model shown below. The feature extraction backbone network in this model includes two processes: frame-level speaker feature extraction and segment-level speaker feature extraction. Among them, frame-level speaker feature extraction includes four residual stages, and each residual stage consists of four residual network blocks. The channel attention network module is embedded after the last residual network block of each residual stage. Segment-level speaker feature extraction consists of a statistically pooling layer and two fully connected networks connected in sequence. The speaker classification network is a layer of AM-Softmax (Softmax with an additional margin) network. AM-Softmax can enhance the discriminability of features by introducing a fixed margin parameter to force an increase in the distance between classes.

[0086] Furthermore, in this embodiment, the acoustic features are 64-dimensional log Mel filter bank energy features, and after being processed by the Kaldi toolkit, the acoustic features are randomly cropped to a length of 2 to 4 seconds, and 64 speech segments with the same number of frames are grouped into a mini-batch.

[0087] Step S3: Obtain a speaker voice dataset and input it into the speaker recognition model embedded with the channel attention network module for optimization training. The model parameters are calculated and optimized by the speaker classification network until a trained speaker recognition model is obtained.

[0088] Specifically, all networks in the speaker recognition model in this embodiment are constructed based on the TensorFlow deep learning framework, optimized using the Adam optimizer, and the model is trained using the AM-softmax loss function. The hyperparameters s and mThe learning rate is set to 30 and 0.15, respectively, and the learning rate is gradually reduced from 1e-3 to 1e-4. The model training set consists of the VoxCeleb1 / 2 datasets, using the Kaldi toolkit's built-in data augmentation algorithm. The test set includes the SITW and VOiCES datasets. The VoxCeleb1 / 2 datasets are large-scale speaker recognition datasets containing speech data from celebrity interview videos. The SITW dataset, also known as a real-world speaker dataset, is used to test speaker recognition performance in complex, real-world environments. The VOiCES dataset, also known as a noisy speech dataset, is a test dataset specifically designed for noisy environments.

[0089] Step S4: Using the trained speaker recognition model to perform a speaker recognition task in a communication scenario.

[0090] The above-mentioned speaker recognition method based on channel attention in a communication scenario, by embedding a channel attention mechanism based on cross-network layer feature aggregation in the form of a channel attention network module in the speaker model, can extract a comprehensive representation of the features in a specific network layer based on the learnable dictionary encoding unit in the network module, so that the speaker representation vector has the ability to represent the input feature map from multiple perspectives, and based on the information aggregation unit in the network, it can fuse features across network layers to obtain a richer and more comprehensive feature representation, thereby improving the accuracy of the speaker representation extracted by the model in the communication scenario, and can achieve more accurate speaker recognition.

[0091] Furthermore, to verify the effectiveness of the proposed method, comparative experiments were conducted. Four systems were constructed for comparison: a baseline system and a system using this application. Two comparison systems using only attention methods were also included: Comparison System 1 (Compression-Incentive Attention Mechanism) and Comparison System 2 (Global Context Vector Attention Mechanism). The embedding locations and parameter settings of these two plug-and-play systems in the speaker recognition model were consistent with those of the proposed method. Unless otherwise noted, the experimental settings for all systems were consistent with those of the baseline system.

[0092] In all systems, the parameter N Set to 4. The backend scoring uses the PLDA (Probabilistic Linear Discriminant Analysis) model, whose training and calculation processes are all implemented by Kaldi. Before the calculation, the speaker representation must first be length normalized, centered, and LDA (Linear Discriminant Analysis) dimension reduction. The final experimental results are evaluated using EER (Equal Error Rate) and minDCF (Minimum Normalized Detection Cost Function). The lower the value of these two indicators, the better the performance. The prior target probability is used when calculating minDCF. Set to 0.01.

[0093] The performance comparison results of the four types of systems on the SITW and VOiCES public evaluation datasets are shown in Tables 1 and 2, respectively. As shown in Table 1, on the SITW dataset, which contains a wide range of open recording scenes, the system using this application shows significant advantages in both EER and minDCF indicators, demonstrating that the method proposed in this application has excellent scene generalization. As shown in Table 2, on the VOiCES dataset, which contains strong reverberant noise, the system using this application also achieves significant performance breakthroughs in various indicators of the development and test sets compared to the comparison system, especially on the test set, verifying the higher robustness of the method proposed in this application.

[0094] Table 1 Performance comparison results of four types of systems on the SITW public evaluation dataset

[0095]

[0096] Table 2 Performance comparison results of four types of systems on the VOiCES public evaluation dataset

[0097]

[0098] Table 3 compares the parameter and computational complexity of the four systems using 100 frames of feature input. As can be seen from Table 3, while the system using this application has a higher parameter and computational complexity, the overall increase in computing resource requirements is manageable, with limited impact on actual application deployment. Furthermore, because the proposed method exhibits significant performance advantages in multiple scenarios and under strong noise interference, it is more scalable and universally applicable.

[0099] Table 3 Comparison of parameters and computational complexity of four types of systems

[0100]

[0101] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A speaker recognition method based on channel attention in a communication scenario, characterized in that The method includes: Constructing a speaker recognition model, which consists of a feature extraction backbone network and a speaker classification network connected in sequence; Embedding a channel attention mechanism based on cross-network layer feature aggregation into the feature extraction backbone network in the form of multiple channel attention network modules; the channel attention network module includes a learnable dictionary encoding unit and an information aggregation unit; wherein, the learnable dictionary encoding unit is designed to obtain a comprehensive representation of the features within a specific network layer in the model, and is used to encode the acoustic features input by the previous network layer to obtain an encoded vector; the information aggregation unit is designed to obtain a global information representation of the cross-network layer features in the model, and is used to aggregate the encoded vectors of multiple network layers to obtain the global channel information representation of the current network layer, and based on the scaling coefficient and translation coefficient calculated according to the global channel information representation, perform feature calibration on the acoustic features input by the previous network layer, and use the calibrated acoustic features as the input of the next network layer; Obtaining a speaker speech dataset and inputting it into the speaker recognition model embedded with channel attention network modules for optimization training, and the model parameters are calculated by the speaker classification network to calculate the loss and optimize and update until a trained speaker recognition model is obtained; Using the trained speaker recognition model to perform speaker recognition tasks in communication scenarios.

2. The method according to claim 1, characterized in that, The feature extraction backbone network in the model includes two processes: frame-level speaker feature extraction and segment-level speaker feature extraction; among them, the frame-level speaker feature extraction includes four residual stages, and each residual stage consists of four residual network blocks, and the channel attention network module is embedded after the last residual network block of each residual stage; the segment-level speaker feature extraction consists of a statistical pooling layer and two fully connected networks connected in sequence; the speaker classification network is a one-layer AM-Softmax network.

3. The method according to claim 2, wherein All networks in the model are constructed based on the TensorFlow deep learning framework, optimized using the Adam optimizer, and the model is trained using the AM-softmax loss function.

4. The method according to claim 3, wherein The model training set consists of the VoxCeleb 1 / 2 dataset, and the data augmentation algorithm provided by the Kaldi toolkit is used, and the test set includes the SITW dataset and the VOiCES dataset.

5. The method according to claim 1, wherein The learnable dictionary encoding unit encodes the acoustic features input by the previous network layer to obtain an encoded vector, including: Suppose the acoustic features input by the previous network layer are ; where R represents the real number space, and the superscripts T , F and C represent the time dimension, frequency dimension, and channel dimension respectively; The learnable dictionary coding unit first passes through the convolution operation of to perform channel dimensionality reduction on the acoustic features to obtain the dimensionally reduced acoustic features , and the expression is: ； Among them, and respectively represent the weight matrix and bias vector of N indicating the total number of preset dictionary components; Then, merge the frequency dimensions of and into the channel dimension to obtain T the acoustic feature vectors of the frames, as well as T the acoustic feature vectors after dimensionality reduction of the frames; where is the number of frames, and ; Meanwhile, three sets of learnable parameter sets are defined, namely the mean vector , the projection vector , and the weight coefficient , and they are respectively initialized as random vectors, all-ones vectors, and 0; where represents the dictionary component serial number; Perform learnable dictionary encoding on three sets of learnable parameter sets to obtain the encoded vector finally output by the learnable dictionary encoding unit , which is expressed as: ； ； ； Among them, the encoded vector is composed of N dictionary component vectors . denotes the weight assigned to the th dictionary component; , and respectively denote the i th mean vector, projection vector, and weight coefficient; is the transpose of , and is the transpose of .​ 6. The method according to claim 1, wherein The information aggregation unit aggregates the encoded vectors of multiple network layers to obtain the global channel information representation of the current network layer, including: The defined memory unit aggregates the feature representations from the previous layer network, and its calculation formula is as follows: ； Among them, represents the aggregated feature of the previous layer network, represents the encoded vector output by the th layer network, represents the non - linear projection process, and the calculation formula is as follows: ； Among them, , and b are all learnable parameters, representing the first-layer weight matrix, the second-layer weight matrix, and the bias vector of the non-linear projection process respectively, is the transpose of; x is the input vector, is the activation function; For the layer network, the expression of the aggregated global channel information representation is: ； ； Among them, represents the feature dimension concatenation operation, represents and the concatenation result of, and respectively represent the weight matrix and bias vector of, is the transpose of.

7. The method according to claim 6, characterized in that, The information aggregation unit performs feature calibration on the acoustic features input by the previous network layer based on the scaling coefficient and translation coefficient calculated according to the global channel information representation, and uses the calibrated acoustic features as the input of the next network layer, including: According to the global channel information representation obtained by aggregating the -th layer network, calculate the scaling factor and translation factor of the -th layer network, which are respectively expressed as: ； ； Among them, represents the sigmoid activation function, and respectively represent the weight matrix and bias vector of and respectively represent the weight matrix and bias vector of and respectively represent and the biases of Using a scaling factor and a translation factor to perform feature calibration on the acoustic features input to the nth layer network, and using the calibrated acoustic features as the input to the nth layer network. The calibrated acoustic features are expressed as: 。 8. The method according to claim 1 or 5 or 7, characterized in that, The acoustic features are 64-dimensional log Mel filter bank energy features, and after being processed by the Kaldi toolkit, the acoustic features are randomly cropped to a length of 2 to 4 seconds, and 64 speech segments with the same number of frames are grouped into a mini-batch.

Images