Short voice-based voiceprint clustering method guided by speaker recognition pre-training model

Abstract

The invention discloses a voiceprint clustering method guided by a speaker recognition pre-training model based on short voices, and the method comprises the following steps: obtaining an original voice signal, and randomly combining a plurality of enhancement strategies to achieve data enhancement; performing voice segmentation on the voice signal after data enhancement based on a uniform segmentation mode; extracting voiceprint features of the segmented voice based on an attention mechanism of global time-frequency domain context modeling; and obtaining a clustering result based on K-means clustering and spectral clustering, matching the clustering result with a real speaker tag, performing reverse transmission based on angle-dependent AAM-Softmax loss, and outputting a voiceprint clustering result. According to the method, the influence of environmental interference on feature extraction can be overcome, feature dimensions which are more effective for identity identification can be screened, a clustering output effect which is superior to that of a mainstream algorithm can be obtained with a relatively low parameter quantity, and the robustness under a noise interference condition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of speech processing, and in particular to a voiceprint clustering method guided by a speaker recognition pre-training model based on short speech. Background Art

[0002] As one of the most challenging recognition tasks in the field of speech processing, speaker clustering aims to cluster speech segments of the same speaker into the same category according to audio signal features, solve the problem of "when is which speaker speaking", and is often used in scenarios such as meeting record analysis and overlapping human voice separation. It separates individual speech streams from the input multi-speaker speech segment, extracts voiceprint features of each speech stream respectively, clusters the speech parts of different speakers into clusters according to the feature vector distribution, and finally obtains the time-sharing recording result of multi-speaker speech, and outputs the identity of the speaker speaking in each time period.

[0003] At present, although most phased clustering algorithms can identify the speaker prediction at each moment according to the process, the recognition accuracy is limited by the shortcoming of certain stages. Usually, in the voiceprint recognition module, due to the too short duration of the separated sliced speech segments, the amount of information extracted by subsequent voiceprint feature extraction is insufficient, directly leading to the decline of clustering performance. The speech recorded in complex multi-speaker scenarios is also often affected by noise interference, which affects the generation of feature vectors of identities. Different noises and recording device differences may affect the stability of feature extraction and clustering. Therefore, proposing a more accurate voiceprint feature extraction method is one of the research focuses on improving clustering effect at the present stage.

[0004] However, most of the existing voiceprint feature extraction methods rely on training with large datasets and are not applicable to the short-time sliced speech generated in segments in the clustering task. The extracted feature vectors are difficult to maintain identity stability in subsequent clustering analysis and are prone to performance degradation in interference scenarios. The voiceprint recognition model for clustering tasks usually faces problems such as too high dimensions and information redundancy, which affect the accuracy of clustering. Therefore, the research on making the model adapt to interference differences and obtaining deeper features needs to be further explored. Summary of the Invention

[0005] In order to overcome the defects and deficiencies of the existing technologies, the present invention provides a voiceprint clustering method guided by a pre-trained model for speaker recognition based on short speech, to make up for the defects in the voiceprint feature extraction process of traditional phased clustering algorithms, that is, the lack of information due to the too short separated sliced speech segments, resulting in the decline of recognition performance. The present invention combines a deeply mined speech feature extraction algorithm into a speaker feature modeling method based on Ecapa-TDNN, uses various reverberations to enhance the original speech training, overcomes the influence of environmental interference on feature extraction, combines a time-frequency attention module and 1D-DCT weight calculation, can screen out feature dimensions that are more effective for identity identification, and is more suitable for processing the separated short-time speech slices in the clustering task, and can achieve a clustering output effect superior to mainstream algorithms with a lower number of parameters. The data augmentation training method also improves the robustness under noise interference conditions.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] The present invention provides a voiceprint clustering method guided by a pre-trained model for speaker recognition based on short speech, including the following steps:

[0008] Obtain the original speech signal, and randomly combine various enhancement strategies to achieve data augmentation;

[0009] Segment the speech signal after data augmentation based on a uniform segmentation method;

[0010] Extract the voiceprint features of the segmented speech based on the attention mechanism of global time-frequency domain context modeling;

[0011] Obtain the clustering result based on K-means clustering and spectral clustering, match the clustering result with the true speaker label, and perform backpropagation based on the angle-dependent AAM-Softmax loss to output the voiceprint clustering result.

[0012] As a preferred technical solution, the enhancement strategies include:

[0013] Waveform Dropout: Randomly replace a certain segment of the waveform of the original speech signal with no signal, that is, randomly set it to zero, randomly set zero for some frequency bands, and add a random no-signal spectral segment to the spectrum through a band-stop filter;

[0014] Rate perturbation: Randomly resample at different sampling rates in different segments;

[0015] Reverberation: Select different spatial sound effects as reverberation noise pulses;

[0016] Additive noise: Randomly add a noise sequence to the original speech signal.

[0017] As a preferred technical solution, an attention mechanism based on global time-frequency domain context modeling is used to extract the voiceprint features of the segmented speech, specifically including:

[0018] Preprocess the segmented speech to obtain fixed-dimension MFCCs features with mean normalization. Each input group is batch-normalized, framed through a short-time sliding window to generate an input signal, and trained for multiple epochs in an augmented corpus.

[0019] The original frequency domain dimension is expanded into a high-dimensional hidden vector with a set number of channels through convolution calculation.

[0020] Introduce time-frequency attention in the global pooling stage, calculate the weight dependencies in the time domain and frequency domain respectively, and generate one-dimensional attention mapping graphs in their respective domains based on time dimension attention and frequency dimension attention.

[0021] Generate a frequency dimension attention mapping and a time frame dimension attention mapping, and obtain a time-frequency domain joint attention mapping spectrum based on vector matrix multiplication.

[0022] Introduce DCT channel weight calculation after per-channel convolution of time-frequency attention, and implement global time-frequency domain context modeling through the calculation of multiple DCT component weights.

[0023] As a preferred technical solution, the original frequency domain dimension is expanded into a high-dimensional hidden vector with a set number of channels through convolution calculation, specifically including:

[0024] The original frequency domain dimension expands the feature dimension to C through a dimensionality-increasing convolutional layer, a ReLU activation function layer, and a batch normalization layer, and obtains a three-layer hidden layer feature vector with dimensions [B, T, C] through three-layer compression excitation SE-Res2Block encoding. The output concatenates the output vectors of the three-layer SE-Res2Block encoding module, and the output dimension is [B, T, C×3].

[0025] As a preferred technical solution, one-dimensional attention mapping graphs in their respective domains are generated based on time dimension attention and frequency dimension attention, specifically expressed as:

[0026]

[0027]

[0028] where T att and F att respectively represent the one-dimensional attention mapping graphs in the time dimension and frequency dimension, μ ·i (j) represents the parameter of the j-th dimension of the i-th vector in the · direction, F dim represents the number of frequency channels, and T represents the number of frame blocks.

[0029] As a preferred technical solution, a joint time-frequency domain attention mapping spectrum is obtained based on vector matrix multiplication, which is expressed as:

[0030]

[0031] where TF att (i,j) represents the element in the i-th row and j-th column of the time-frequency attention mapping spectrum, represents vector multiplication calculation, T att and F att respectively represent the one-dimensional attention mapping graphs in the time dimension and the frequency dimension.

[0032] As a preferred technical solution, DCT channel weight calculation is introduced after per-channel convolution of time-frequency attention, which specifically includes:

[0033] Calculate the DCT coefficients, which are expressed as:

[0034]

[0035] where X c [n] represents the data of the n-th dimension of the input vector, Y c [μ] represents the DCT coefficient of the corresponding channel, μ = 0, 1, …, N - 1, and E(μ) is the normalization coefficient;

[0036] Select G of these DCT coefficients to form a new vector matrix φ c , which is expressed as:

[0037]

[0038] where, is the feature matrix of the c-th channel, C s represents the corresponding domain vector dimension number, DCT i represents the index of the output DCT component vector;

[0039] Select the channel with the maximum response by row maximization operation as the global maximum optimization result g c = maxφ c (:), and perform batch normalization and Softmax calculations to obtain the final channel weight C att .

[0040] As a preferred technical solution, a clustering result is obtained based on K-means clustering and spectral clustering, which specifically includes:

[0041] When processing the in-batch speaker feature vector samples, each speaker feature vector sample is labeled according to K randomly selected initial category centers and assigned to the nearest category center, and each category center point is updated again after classification;

[0042] For the number of clustering clusters of the input feature vector sample points, calculate the Laplacian matrix of the sample data, calculate the first K eigenvalues of the Laplacian matrix, and the eigenvectors corresponding to each eigenvalue will form a new matrix by column vectors. Take the new sample points to form clustering clusters in the K-means clustering algorithm, and output the clustering results.

[0043] As a preferred technical solution, update the center point of each category again after classification, which is specifically expressed as:

[0044]

[0045] Among them, N j represents the sample size included in the j-th speaker cluster. The optimization goal is to make the distance of each sample relative to the center of its respective category approach 0. Y i represents the speaker feature vector sample, and a j represents the category center.

[0046] As a preferred technical solution, calculate the Laplacian matrix of the sample data, which is constructed by the matrix W representing the sample similarity and the diagonal matrix D, and is specifically expressed as:

[0047]

[0048] Among them, W is the similarity matrix composed of s ij , D is the n×n diagonal matrix composed of the sum d of each row element of W i , and L = D - W is obtained, where L represents the Laplacian matrix.

[0049] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0050] (1) The present invention adopts the technical solution of introducing an attention mechanism into the pre-trained network for voiceprint recognition to generate more identity-discriminative voiceprint feature encodings, solves the technical problem of inaccurate extraction of sliced speech features after segmentation in the clustering task, achieves the technical effect of further improving the classification quality in the clustering stage, and effectively enhances the user experience in the noise interference scenario.

[0051] (2) The present invention adopts a data augmentation training method implemented by noise reverberation and spatial reverberation. Without increasing the quantity of the original corpus, it can effectively focus on the identity consistency and stability of speech under different interference conditions, accurately grasp the identity-discriminative features, and provide a solid foundation for improving the robustness of the model.

[0052] (3) The present invention adopts a time-frequency attention mechanism to enhance the overall attention of the network to the time-domain and frequency-domain content. When extracting voiceprint features, it not only focuses on reflecting the global frequency-domain features but also pays attention to the energy distribution in the time domain, deeply excavates the detailed features, obtains more comprehensive information, and the recognition pattern is more in line with the human auditory perception and information processing habits, significantly improving the accuracy of feature extraction.

[0053] (4) The present invention adopts a DCT-based channel weight calculation method to provide interpretability for the multi-channel information processing process. Utilizing the signal compression characteristic of the DCT transform, it recalibrates the inter-domain channel weight information, and further improves the performance of the recognition network without adding a large amount of computational cost. Description of the Drawings

[0054] Figure 1 It is a schematic flow diagram of the voiceprint clustering method guided by the speaker recognition pre-training model based on short speech of the present invention;

[0055] Figure 2 It is a schematic network structure diagram of the voiceprint feature vector extraction module of the present invention;

[0056] Figure 3 It is a schematic network structure diagram of the time-frequency domain attention module of the present invention;

[0057] Figure 4 It is a schematic structure diagram of the DCT channel weight calculation of the present invention. Detailed Embodiment

[0058] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0059] Embodiment

[0060] As Figure 1 shown, this embodiment provides a voiceprint clustering method guided by a speaker recognition pre-training model based on short speech, including the following steps:

[0061] S1: Perform enhanced preprocessing on the acquired data;

[0062] In this embodiment, the data augmentation preprocessing is achieved by relying on the spatial reverberation pulses provided by the RIRs corpus and various environmental noises provided by the MUSAN corpus. Its purpose is to train feature vectors adaptable to various interference scenarios, which are used for multi-noise background enhancement at a sampling rate of 16 kHz. During multiple training cycles, enhanced noises are introduced to generate different contaminated data. In the standard augmentation process, a random combination of multiple augmentation strategies is applied to a single segment of speech, providing six different "views" for the same segment of speech, which helps the model obtain feature vectors with identity consistency and robustness during the identity feature vector extraction stage;

[0063] In this embodiment, the process of data augmentation preprocessing will apply a number of random combinations of the following four augmentation operations to the original speech signal, specifically including:

[0064] (1) Waveform Dropout: Randomly replace a certain segment of the original waveform with no signal, that is, randomly set it to zero. Similar processing is performed in the frequency domain by randomly setting zero in some frequency bands, that is, adding random no-signal spectral segments to the spectrum through a band-stop filter;

[0065] (2) Rate Perturbation: Resample at different sampling rates randomly in different segments. The resampled speech signal is slightly accelerated or decelerated relative to the original signal. This can better learn the representation of identity consistency. Considering the impact of rate changes on the feature extraction of the same speaker, changing the rate by an average of 5% can increase the sample representation range without significantly changing the speaker's features;

[0066] (3) Reverberation: Select different spatial sound effects as reverberation noise pulses to improve the identity consistency of the voiceprint features obtained by the same speaker under different scene noise conditions;

[0067] (4) Additive Noise: Randomly add a noise sequence to the original audio, added according to the set signal-to-noise ratio, usually added simultaneously with reverberation, which can increase the diversity of noise interference and help the model better lock in the features beneficial to identity recognition in the speech segment under noise interference;

[0068] The data augmentation preprocessing enriches the corpus data while helping the model establish robust feature vectors with identity stability.

[0069] S2: Speech segmentation;

[0070] The voice segmentation is carried out in a uniform segmentation manner. Since the voice length has relatively low reference value in speaker identification, it is necessary to ensure that the lengths of the segmented voice segments are as consistent as possible to avoid the influence of different voice segment durations on identity judgment. After effective voice detection, the original voice is evenly segmented into short voice slices. The segmentation step length is set to 10 milliseconds, and the duration of each voice segment is 1.5 seconds to 2 seconds. Each segment is converted into MFCCs with 80 dimensions to extract the voiceprint feature vector. Each voice frame after segmentation corresponds to a sliding window of a fixed length, and each window segment is input into the voiceprint recognition network for identity feature extraction, and the intermediate frame-level feature vectors of the clustering model are obtained respectively.

[0071] S3: Voiceprint feature vector extraction;

[0072] As Figure 2 shown, the voiceprint feature vector extraction module is proposed based on the Ecapa-TDNN network. Considering the problem of unstable short-time voice identity identification in the present invention, it is necessary to further enhance the network modeling ability and remove redundant information of the speaker representation. The advantages of the attention mechanism and DCT compressed signal are combined, and the attention mechanism for global time-frequency domain context modeling is used for feature extraction, which can capture the long-term dependence relationship between different time domains and frequency domains. For the intra-domain channels, 1D-DCT is used to model the context weights respectively to improve the model signal modeling ability, and the similarity between the global context and local features is calculated using the global channel attention information to recalibrate the weight information of the inter-domain channels, providing model interpretability for the multi-channel tensors of the time-frequency domain branch. This lightweight module can be integrated into the voiceprint recognition network to improve performance without increasing a large amount of computational cost;

[0073] In this embodiment, the specific process of voiceprint feature vector extraction is as follows:

[0074] S31: Input features: Using the built-in feature extraction tool in SpeechBrain, the original signal is preprocessed to obtain MFCCs features with fixed dimensions that are mean-normalized. Each input group is batch-normalized, and then framed through a short-time sliding window. Specifically, it is framed through a sliding window with a window length of 3 seconds and a moving step length of 1.5 seconds. The generated input signal dimension is [B, F, T], where the batch size B = 32, the number of feature dimensions F = 80, and the number of frame blocks T is not fixed. The pre-training module for voiceprint feature vector extraction will be trained for 15 epochs in the VoxCeleb1 and 2 corpora with data augmentation. After being enhanced by six types of noise addition methods, the actual batch size B is 192;

[0075] S32: Hidden vector extraction: The original frequency domain dimension is expanded into a high-dimensional hidden vector with a specified number of channels C through convolution calculation. Specifically, the feature dimension is expanded to C through an upsampling convolutional layer, a ReLU activation function layer, and a batch normalization layer. Then, it passes through three layers of squeeze-and-excitation SE-Res2Block encoding to obtain a three-layer hidden layer feature vector with dimensions [B, T, C]. The output concatenates the output vectors of the three-layer SE-Res2Block encoding module, and the actual output dimension is [B, T, C×3]. In this embodiment, the value of C is preferably 512;

[0076] S33: Time-frequency domain attention: The voiceprint feature vector extraction module introduces a time-frequency attention module in the global pooling stage to calculate the weight dependence relationships in the time domain and the frequency domain respectively. As Figure 3 shown, it mainly consists of two parallel attention branches, namely the time dimension attention and the frequency dimension attention. Each branch performs average pooling respectively to generate one-dimensional attention maps T att and F att in their respective domains, guiding the model to focus on the time frames or frequency dimensions that are more important for identity recognition in the corresponding domains:

[0077]

[0078] where, μ ·i (j) represents the parameter of the j-th dimension of the i-th vector in the · direction. The one-dimensional vectors obtained by average pooling respectively pass through an encoding structure composed of a per-channel convolution and a ReLU activation function layer to expand the number of channels to C f 、C t . The following DCT channel weight calculation will specifically elaborate on the channel number processing process. The weighted channel vectors then enter a pointwise convolution to merge the channel data and introduce a Sigmoid activation function to model the non-linearity;

[0079] The time-domain and frequency-domain attention weights after depthwise separable convolution are combined through vector multiplication to generate a two-dimensional frame-level feature attention map, helping the model capture the speech feature distribution of the time-frequency domain fusion. The weights of the two domain branches are multiplied and merged, and finally the time-frequency domain joint attention map spectrum is obtained:

[0080]

[0081] where, TF att (i, j) represents the element in the i-th row and j-th column of the time-frequency attention map spectrum, represents the vector multiplication calculation, and the encoded frame-level feature representation is input into the time-frequency attention module, which contains T time frames and F dim frequency dimensions, that is, the number of frequency channels. The frequency domain branch generates the frequency dimension attention map The time-domain branch generates a time-frame dimension attention map Using the principle of vector matrix multiplication, calculate the two-dimensional time-frequency domain attention map spectrum for the final point-to-point feature vector Calculate the weighted features by element-wise multiplication, and the output is expressed as

[0082] S34: DCT Weight Calculation: After the per-channel convolution of the time-frequency attention module, the voiceprint feature vector extraction module introduces DCT channel weight calculation. Utilizing the signal compression characteristic of DCT, it helps the network refine the effective frequency domain channel information. As a more extensive alternative form of global average pooling, as Figure 4 shown, the voiceprint feature vector extraction module calculates 1D-DCT using the following formula:

[0083]

[0084] where, X c [n] represents the nth-dimensional data of the input vector, and Y c [μ] is the DCT coefficient calculated for this channel, μ = 0, 1, …, N - 1, and E(μ) is the normalization coefficient. N input-dimensional data are used when calculating each DCT coefficient result. To achieve global time-frequency domain context modeling using rich voice channel information, design to calculate weights using multiple DCT components, where Y = [Y1, T2, …, Y c , is the feature matrix of the cth channel, C s represents the corresponding domain vector dimension number, set the DCT component DCT i represents the index of the output DCT component vector, select G DCT coefficients from them to form a new vector matrix φ c , whose dimension is G×1:

[0085]

[0086] Define the global context embedding matrix composed of complete 1D-DCT coefficients, and select the channel with the maximum response through row maximization operation, which will also be used as the global maximum optimization result g c = max φ c (:), after maximum optimization, perform batch normalization and Softmax calculation to obtain the final channel weight C att . The per-domain per-channel output X c of each channel will be regulated by the corresponding channel weight C att to achieve relative in-batch channel enhancement or suppression.

[0087] S4: Clustering Backend;

[0088] The clustering backend mainly combines K-means clustering and spectral clustering. The obtained clustering results will match the true speaker labels, and the angle-dependent AAM-Softmax loss will be calculated for backpropagation, and finally the clustering classification results will be output;

[0089] In this embodiment, the specific process of the clustering backend is as follows:

[0090] S41: K-means clustering: When processing the in-batch speaker feature vector samples T = Y1, Y2,..., Y N , according to the randomly selected initial K class centers a1, a2,..., a K each sample Y i is labeled and assigned to the nearest class center a j . After classification, each class center point is updated again. The center point of the belonging class usually shifts due to the newly added samples;

[0091]

[0092] where N j represents the number of samples included in the j-th speaker cluster. The optimization goal is to make the distance of each sample relative to its respective class center approach 0. The above steps will be continuously iterated until the preset maximum number of iterations is satisfied or the set minimum squared error value is reached. The optimization objective function uses the squared error, which measures the vector similarity through the Euclidean space distance.

[0093] S42: Spectral clustering: Compared with traditional methods, it can complete clustering on a sample space with arbitrary shapes and converge to the global optimal solution, converting the clustering problem into an optimal partitioning problem of a graph. For the input n feature vector sample points X = {x1, x2,..., x n} and the number of clustering clusters K, first calculate the Laplacian matrix L of the sample data, which is constructed from the matrix W representing sample similarity and the diagonal matrix D:

[0094]

[0095] where W is the similarity matrix composed of s ij , and D is the n×n diagonal matrix composed of the sum of each row element d i of W, obtaining L = D - W. Then calculate the first K eigenvalues λ = [λ1, λ2,..., λ K of the Laplacian matrix L, and the eigenvectors a i corresponding to each eigenvalue λ i will be composed into a new matrix U = {a1, a2,..., a K} as column vectors. Take the new sample points Y = {y1, y2,..., yn} In the K-means clustering algorithm, cluster clusters C1, C2, …, C K , and output the final result. represents the i-th row vector of U.

[0096] The above embodiments are preferred embodiments of the present invention. However, the embodiments of the present invention are not limited by the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principle of the present invention shall be equivalent replacement methods and are all included in the protection scope of the present invention.

Claims

1. A voiceprint clustering method guided by a pre-trained model for speaker recognition based on short speech, characterized in that It includes the following steps: Obtain the original voice signal and randomly combine multiple enhancement strategies to achieve data enhancement; Segment the voice signal after data enhancement based on the uniform segmentation method; Extract the voiceprint features of the segmented voice based on the attention mechanism of global time-frequency domain context modeling; Obtain the clustering result based on K-means clustering and spectral clustering, match the clustering result with the true speaker label, perform backpropagation based on the angle-dependent AAM-Softmax loss, and output the voiceprint clustering result.

2. The voiceprint clustering method guided by the speaker recognition pre-training model based on short speech according to claim 1, wherein The enhancement strategies include: Waveform Dropout: Randomly replace a certain segment of the waveform of the original voice signal with no signal, that is, randomly set it to zero, randomly set zero for some frequency bands, and add random no-signal spectral segments to the spectrum through a band-stop filter; Rate perturbation: Randomly resample at different sampling rates in different segments; Reverberation: Select different spatial sound effects as reverberation noise pulses; Additive noise: Randomly add a noise sequence to the original voice signal.

3. The voiceprint clustering method guided by the speaker recognition pre-training model based on short speech according to claim 1, wherein Extract the voiceprint features of the segmented voice based on the attention mechanism of global time-frequency domain context modeling, specifically including: Preprocess the segmented voice to obtain MFCCs features with fixed dimensions after mean normalization. Each input group is batch-normalized, framed through a short-time sliding window to generate an input signal, and trained for multiple epochs in the data-enhanced corpus; Expand the original frequency domain dimension into a high-dimensional hidden vector with a set number of channels through convolution calculation; Introduce time-frequency attention in the global pooling stage, calculate the weight dependencies in the time domain and frequency domain respectively, and generate one-dimensional attention mapping graphs in their respective domains based on the time dimension attention and frequency dimension attention; Generate the frequency dimension attention mapping and the time frame dimension attention mapping, and obtain the time-frequency domain joint attention mapping spectrum based on the vector matrix multiplication; Introduce DCT channel weight calculation after the per-channel convolution of the time-frequency attention, and realize global time-frequency domain context modeling through the weight calculation of multiple DCT components; 4. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 3, wherein Expand the original frequency domain dimension into a high-dimensional hidden vector with a set number of channels through convolution calculation, specifically including: Expand the feature dimension of the original frequency domain into C through an upsampling convolutional layer, a ReLU activation function layer, and a batch normalization layer, and obtain a three-layer hidden layer feature vector with dimensions [B, T, C] after three-layer compression excitation SE-Res2Block encoding. Output the concatenated output vector of the three-layer SE-Res2Block encoding module, and the output dimension is [B, T, C×3].

5. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 3, wherein Generate one-dimensional attention mapping graphs in their respective domains based on the time dimension attention and frequency dimension attention, specifically expressed as: Among them, T att and F att respectively represent the one-dimensional attention mapping diagrams in the time dimension and the frequency dimension, and μ ·i (j) represents the parameter of the j-th dimension of the i-th vector in the · direction, and F dim represents the number of frequency channels, and T represents the number of frame blocks.

6. The voiceprint clustering method guided by the speaker recognition pre-training model based on short speech according to claim 3, wherein Obtain the time-frequency domain joint attention mapping spectrum based on the vector matrix multiplication, expressed as: Among them, TF att (i, j) represents the element in the i-th row and j-th column of the time-frequency attention mapping spectrum, represents vector multiplication calculation, T att and F att respectively represent the one-dimensional attention mapping diagrams in the time dimension and the frequency dimension.

7. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 3, wherein Introduce DCT channel weight calculation after the per-channel convolution of the time-frequency attention, specifically including: Calculate the DCT coefficients, expressed as: where X c [n] represents the n-th dimensional data of the input vector, Y c [μ] represents the DCT coefficient of the corresponding channel, μ = 0, 1, …, N - 1, and E(μ) is the normalization coefficient; Select G of the DCT coefficients to form a new vector matrix φ c , denoted as: Among them, is the feature matrix of the c-th channel, C s represents the corresponding domain vector dimension number, DCT i represents the index of the output DCT component vector; The channel with the maximum response is selected by the row maximization operation as the global maximum optimization result g c = max φ c (:), perform batch normalization and Softmax calculations to obtain the final channel weights C att .

8. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 1, wherein Obtain the clustering result based on K-means clustering and spectral clustering, specifically including: When processing the in-batch speaker feature vector samples, mark each speaker feature vector sample according to the randomly selected initialized K category centers, assign it to the nearest category center, and update each category center point again after classification; For the number of clustering clusters of the input feature vector sample points, calculate the Laplacian matrix of the sample data, calculate the first K eigenvalues of the Laplacian matrix, and the eigenvectors corresponding to each eigenvalue will form a new matrix by column vectors. Take the new sample points to form clustering clusters in the K-means clustering algorithm and output the clustering results.

9. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 8, characterized in that, After classification, update the center point of each category again, specifically expressed as: Among them, N j represents the sample size included in the j-th speaker cluster. The optimization objective is to make the distance of each sample from its respective class center approach 0, and Y i represents the speaker feature vector sample, and a j represents the class center.

10. The voiceprint clustering method guided by the pre-trained speaker recognition model based on short speech according to claim 8, characterized in that, Calculate the Laplacian matrix of the sample data, which is constructed from the matrix W representing sample similarity and the diagonal matrix D, specifically expressed as: Among them, W is the similarity matrix composed of s ij The sum of the elements in each row of W is d i The resulting n×n diagonal matrix, and we get L = D - W, where L represents the Laplacian matrix.

Images