Voiceprint recognition method and system, medium, equipment and program product

Abstract

The invention provides a voiceprint recognition method and system, a medium, equipment and a program product. The method comprises the following steps: acquiring to-be-recognized audio data; audio data to be recognized are input into a pre-trained convolutional neural network encoder, down-sampling is carried out, and low-layer acoustic features are extracted; extracting the low-layer acoustic features through a multi-layer converter to obtain universal features of different layers, inputting the universal features of each layer into a hybrid expert processing layer, and performing feature screening on the universal features to obtain screened universal features; performing feature fusion on each screened general feature in a hierarchical dimension to obtain a fused feature; inputting the fusion feature into an acoustic feature recognition network to obtain an acoustic feature embedded vector; the acoustic feature embedded vector is used for representing voiceprint information corresponding to the to-be-recognized audio data. According to the invention, the recognition stability and generalization performance of the audio data can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio data processing, in particular to a voiceprint recognition method, system, medium, device and program product. BACKGROUND

[0002] In recent years, pre-trained models are gradually applied to various downstream speech tasks. Pre-trained models have strong general representation ability. Real-world speaker recognition data, such as the VoxCeleb dataset, often has significant domain differences, such as inconsistent recording devices, noise environments, accents, gender, and speech rates. A single parameterized model is difficult to consider all domains when learning features, which can lead to insufficient generalization to some domains and affect voiceprint recognition in audio data. SUMMARY

[0003] The purpose of the present application is to provide a voiceprint recognition method, system, computer-readable storage medium, electronic device and computer program product, which can improve the recognition stability and generalization performance of audio data.

[0004] To solve the above technical problems, the present application provides a voiceprint recognition method, and the specific technical solutions are as follows:

[0005] Obtain audio data to be identified;

[0006] Input the audio data to be identified into a pre-trained convolutional neural network encoder to downsample and extract low-level acoustic features;

[0007] Extract different levels of general features from the low-level acoustic features through a multi-layer transformer. Each layer of the general features is input into a hybrid expert processing layer to perform feature screening on the general features, and obtain screened general features;

[0008] Perform feature fusion on each of the screened general features in the level dimension to obtain fusion features;

[0009] Input the fusion features into an acoustic feature recognition network to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to represent voiceprint information corresponding to the audio data to be identified.

[0010] Optionally, the structure of the hybrid expert processing layer comprises a plurality of expert subnets and a gating network; each of the expert subnets comprises a first one-dimensional convolutional layer, a rectified linear unit and a second one-dimensional convolutional layer; the gating network comprises a statistical pooling layer and two layers of gating subnetworks; the statistical pooling layer is configured to calculate a mean value and a standard deviation in a time dimension respectively, and splice the mean value and the standard deviation to obtain a global summary vector; the gating subnetworks are configured to map the global summary vector into a logical score through linear transformation.

[0011] The hybrid expert processing layer further comprises a normalization exponential function configured to normalize the selected expert subnets to obtain selection probabilities of the expert subnets.

[0012] Optionally, the normalization of the selected expert subnets to obtain the selection probabilities of the expert subnets comprises:

[0013] The general feature of each layer is input into the hybrid expert processing layer to obtain selection probabilities of the E expert subnets for the general feature.

[0014] The positions corresponding to K expert subnets with the highest selection probabilities are selected as a mask, and the remaining positions are set to zero; the K expert subnets are normalized to obtain the selection probabilities of the expert subnets.

[0015] Optionally, after obtaining the selection probabilities of the E expert subnets for the general feature, the method further comprises:

[0016] The quotient of each selection probability and a set temperature parameter is calculated; the set temperature parameter is used to smooth the condition distribution.

[0017] Correspondingly, the normalization of the K expert subnets comprises:

[0018] The quotient corresponding to each of the K expert subnets is normalized.

[0019] Optionally, the general feature of each layer is input into the hybrid expert processing layer to perform feature screening on the general feature to obtain screened general features.

[0020] The general feature is input into a cascaded residual module.

[0021] In the cascaded residual module, the first layer is configured to obtain screened general features from the general feature.

[0022] Starting from the second layer, each layer obtains the screened general features processed by the previous processing layer, and performs point-by-point multiplication with a gating value to obtain a shallow layer feature; the shallow layer feature is added to the general feature extracted by a transformer of the layer, and then the hybrid expert processing layer is used to obtain a screened feature of the layer, and the screened feature of the layer is used as an input of a next layer.

[0023] until the last layer outputs the final filtered general feature.

[0024] Optionally, each of the filtered general features is fused in the hierarchy dimension to obtain a fused feature including:

[0025] The final filtered general feature is taken as a query representation, and a key representation and a value representation are obtained by stacking the filtered features corresponding to each layer in the cascade residual module;

[0026] The query representation, the key representation, and the value representation are linearly mapped by using a linear layer respectively to obtain a query feature, a key feature, and a value feature;

[0027] The attention weight is obtained based on the query feature and the key feature;

[0028] The fused feature is obtained according to the attention weight and the value feature.

[0029] The application further provides a voiceprint recognition system, comprising:

[0030] An audio data acquisition module is configured to acquire to-be-recognized audio data;

[0031] A feature sampling module is configured to input the to-be-recognized audio data into a pre-trained convolutional neural network encoder to perform down-sampling and extract low-level acoustic features;

[0032] An expert module processing module is configured to extract different levels of general features from the low-level acoustic features through a multi-layer transformer, and input each of the general features of each layer into a hybrid expert processing layer to perform feature filtering on the general features to obtain filtered general features;

[0033] A feature fusion module is configured to fuse each of the filtered general features in the hierarchy dimension to obtain a fused feature;

[0034] A voiceprint recognition module is configured to input the fused feature into an acoustic feature recognition network to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to represent voiceprint information corresponding to the to-be-recognized audio data.

[0035] The application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the method described above.

[0036] The application further provides an electronic device comprising a memory and a processor, wherein the memory has a computer program stored therein, and the processor implements the steps of the method described above when invoking the computer program in the memory.

[0037] The application also provides a computer program product comprising a computer program which, when executed, implements the steps of the method described above.

[0038] The application provides a voiceprint recognition method, comprising: obtaining to-be-recognized audio data; inputting the to-be-recognized audio data into a pre-trained convolutional neural network encoder to perform downsampling and extract low-level acoustic features; extracting different levels of general features from the low-level acoustic features through a multi-layer transformer; inputting each of the general features at different levels into a hybrid expert processing layer to perform feature screening on the general features, to obtain screened general features; performing feature fusion on the screened general features in a level dimension, to obtain fused features; and inputting the fused features into an acoustic feature recognition network, to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to represent voiceprint information corresponding to the to-be-recognized audio data.

[0039] The application compresses original speech in a time domain dimension through a convolutional neural network encoder, retains low-level acoustic properties such as a fundamental frequency and formants, and effectively eliminates redundant sampling points, so that a subsequent network can focus on key short-time frames under lower computational load, and the efficiency of front-end processing is significantly improved. A hybrid expert processing layer introduces a sparse gating mechanism at an output end of each level, can dynamically activate the most relevant sub-network according to implicit conditions such as a speaker gender, a channel, and a noise environment, and performs fine screening and weighting on features, so as to suppress irrelevant variations and retain robust information shared across scenes. The level general features after screening are fused in a unified dimension, so that fine-grained short-time spectral characteristics and long-time global context are complementary to each other, to form fused representations with discriminability and generalization. Finally, the acoustic feature recognition network maps the fused features into a compact embedding vector, which remains compact clustering under multiple interference factors such as a channel, a language, and an emotion, significantly reduces an equal error rate, and can complete cross-domain voiceprint comparison through simple cosine measurement without retraining, to achieve a lightweight, highly robust, and easily migratable voiceprint recognition effect.

[0040] The application also provides a voiceprint recognition system, a computer-readable storage medium, an electronic device, and a computer program product, which have the above beneficial effects, and details are not repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0042] Figure 1A flowchart of a voiceprint recognition method provided by an embodiment of the present application;

[0043] Figure 2 A structural schematic diagram of a hybrid expert processing layer provided by an embodiment of the present application;

[0044] Figure 3 A cross-attention fusion process schematic diagram provided by an embodiment of the present application;

[0045] Figure 4 A structural schematic diagram of a voiceprint recognition system provided by an embodiment of the present application;

[0046] Figure 5 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in a clear and complete manner with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0048] Reference is made to Figure 1 , Figure 1 A flowchart of a voiceprint recognition method provided by an embodiment of the present application, the method comprising:

[0049] S101: obtaining to-be-recognized audio data;

[0050] S102: inputting the to-be-recognized audio data into a pre-trained convolutional neural network encoder to perform down-sampling and extract low-level acoustic features;

[0051] S103: extracting different levels of general features from the low-level acoustic features through a multi-layer transformer, and inputting the general features of each layer into a hybrid expert processing layer to perform feature screening on the general features, to obtain screened general features;

[0052] S104: performing feature fusion on the screened general features in a level dimension, to obtain fused features;

[0053] S105: inputting the fused features into an acoustic feature recognition network, to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to represent voiceprint information corresponding to the to-be-recognized audio data.

[0054] In step S101, the core task of this step is to convert any form of original sound wave signal into audio data to be identified for subsequent algorithm modules to read, analyze and operate directly. In the specific implementation process, any one or any combination of the following methods can be used to complete the collection and preprocessing, and at the same time, the data quality guarantee and compliance control are realized, so as to lay a foundation of high fidelity, low redundancy and high security for subsequent feature extraction.

[0055] In the first way, the sound source can be collected by a microphone array. The microphone array can be linear topology, or ring, spherical or other irregular topology; the number of array elements can be flexibly selected according to the scene complexity, cost and algorithmic limit. The advantage of array collection is that the beam forming, sound source positioning and adaptive noise reduction algorithm can be used to enhance the sound in the target direction and suppress the interference in the non-target direction, so that the original waveform with high signal-to-noise ratio can be obtained in a severe reverberation or strong background noise environment.

[0056] In the second way, the collection can be completed by a single high-sensitivity microphone. This way is suitable for quiet environment or cost-sensitive equipment, such as personal mobile terminal, smart home voice panel, etc. After single microphone collection, echo cancellation, automatic gain control, dynamic range compression and band pass filtering modules in software level can be cascaded to weaken the circuit noise, environmental noise and device self-excited echo, so that the output waveform meets the requirements of the subsequent encoder on input signal-to-noise ratio and amplitude stability.

[0057] In the third way, the audio can be directly read from the existing media file or cloud storage. The file format can be lossless compression or lossy compression, such as linear pulse code modulation, free lossless audio coding, advanced audio coding, etc. After reading, if the sampling rate, bit depth or channel number does not meet the requirements of the back-end network, resampling, bit depth conversion, channel mixing or separation operation is implemented to make the output data meet the requirements of the subsequent convolutional neural network encoder on input tensor size and numerical distribution in time domain, frequency domain and dynamic range.

[0058] In the fourth way, hardware-level encryption and digital signature modules can be embedded in the collection end to encrypt the original waveform or the segment after preliminary preprocessing by national or international algorithms, and at the same time, time stamp, device serial number and integrity check value are added to prevent data tampering in transmission or storage link, and to ensure the traceability and judicial effectiveness of the subsequent voiceprint comparison result.

[0059] In the fifth way, the audio can be segmented and labeled after collection. The segmentation strategy can be based on energy detection, voice activity detection or deep learning endpoint detection algorithm to cut long-time recording into segments containing only valid speech, thereby reducing the computational load of subsequent network inference; the labeling information can include speaker identity, collection scene, channel type, emotion label, etc., which is used for subsequent model fine-tuning or performance evaluation.

[0060] In step S102, the audio data to be identified is input into the pre-trained convolutional neural network encoder for down-sampling and extraction of low-level acoustic features. This step aims to use the convolutional neural network encoder to perform local perception and hierarchical abstraction on the original waveform or time-frequency graph, compress data redundancy through down-sampling operations, while retaining short-time spectral details, pitch contour and formant structure that are crucial for voiceprint differentiation, forming low-level acoustic features. The implementation can be developed from four dimensions of network structure, input form, pre-training strategy and computing power deployment.

[0061] In terms of input form, one-dimensional sample point sequence of the original waveform can be accepted, or two-dimensional time-frequency representation obtained through short-time Fourier transform, mel filter bank or gamma pass filter bank. If the original waveform is used, a one-dimensional convolution kernel can be used in the first layer to simulate the function of a traditional filter bank, realizing end-to-end feature learning; if a time-frequency graph is used, a two-dimensional convolution kernel can be used in the first layer to jointly extract local time-frequency patterns along the time and frequency dimensions.

[0062] Self-supervised pre-training can be performed on a large-scale general audio corpus, for example, through a contrastive learning, mask prediction or sequence reconstruction task, to enable the encoder to learn to decouple speaker differences, channel differences and content differences; then supervised fine-tuning is performed on the target speaker dataset to make the low-level features more discriminative for voiceprint differentiation. Noise enhancement, reverberation enhancement, speech rate disturbance and spectral masking can be introduced in the pre-training process to improve the robustness of the encoder to real environment mismatches.

[0063] The core of step S103 is to use a multi-layer transformer to model the audio sequence globally and to realize on-demand activation of feature screening through a hybrid expert mechanism, so as to capture long-time dependencies, semantic associations and speaker individuality clues at different abstraction levels, and finally output refined screened general features.

[0064] The multi-layer transformer can use a pure self-attention architecture, or a lightweight convolution can be added at the front end for local preprocessing. The number of self-attention heads and the dimension can be dynamically adjusted according to the computing budget. A larger number of heads can capture multiple sub-space information, and a higher dimension can improve the feature expression capability. Position encoding can use absolute sine, learnable absolute or relative position forms to make the network sensitive to the order of speech.

[0065] The mixed expert processing layer can include a plurality of expert subnets and a gating network. Each of the expert subnets includes a first one-dimensional convolutional layer, a rectified linear unit, and a second one-dimensional convolutional layer; the gating network includes a statistical pooling layer and two layers of gating subnetworks; the statistical pooling layer is configured to calculate a mean value and a standard deviation in a time dimension respectively, and splice the mean value and the standard deviation to obtain a global summary vector; the gating subnetworks are configured to map the global summary vector into a logical score through linear transformation; and the mixed expert processing layer further includes a normalization exponential function configured to normalize the selected expert subnets to obtain selection probabilities of the expert subnets.

[0066] The expert subnetworks can be feedforward fully connected, convolutional, or light recurrent units, each of which is good at processing a specific channel, a specific noise, or a specific speaking style; and the gating network can be a single-layer or multi-layer perceptron, or a small transformer, an input of which is a general feature corresponding to a level, and an output of which is an expert weight vector. The gating network selects only a small number of experts for calculation through a softmax or a sparse activation function, so as to realize conditional calculation in an inference stage and reduce overall computing power.

[0067] The feature screening process can introduce sparse regularization and load balancing loss to prevent the same batch of experts from being activated all the time and cause a winner-takes-all phenomenon; and noise disturbance and random dropout can be introduced in the training stage to force the gating network to explore different expert combinations and improve generalization ability. The screened general features filter out redundant information irrelevant to the voiceprint, and retain long-term prosody, semantic, and emotional clues that are most critical for distinguishing speakers.

[0068] This step can mine long-term dependencies that cannot be captured by traditional convolution on a global context level, and realize sparse calculation of on-demand activation through the mixed expert mechanism, and finally output refined, compact, and highly discriminative screened general features.

[0069] In a feasible implementation, after the general features of each layer are input into the mixed expert processing layer, selection probabilities of the general features by E expert subnets are obtained, then a mask is selected at positions corresponding to K expert subnets with the highest selection probabilities, the remaining positions are set to zero, normalization processing is performed on the K expert subnets, and selection probabilities of the expert subnets are obtained.

[0070] Referring to Figure 2 , Figure 2 A structure diagram of the mixed expert processing layer provided by the embodiments of the present application. Figure 2 The MoE module (Mixture of Experts) is included. and The MoE module is input and output, respectively.

[0071] The mixture expert processing layer adopts a Top-K mixture of experts (Top-K MoE) structure. Only the top K experts are selected to participate in the calculation at each forward calculation. The on-demand activated routing can guarantee the expression ability and significantly reduce the calculation overhead. The mixture expert processing layer includes E expert subnets denoted as , and one gating network denoted as .

[0072] Each expert is composed of the following layers stacked in sequence: a one-dimensional convolution layer (with a convolution kernel size of 1), a rectified linear unit (ReLU), and another one-dimensional convolution layer. The gating network is composed of three parts: a statistical pooling layer: the mean and standard deviation are calculated in the time dimension respectively, and are spliced to obtain a global summary vector; two layers of gating subnetworks: two layers of linear transformations (with a ReLU activation function in between) map the summary vector to logits with a length of E; and a normalization exponential function (Softmax): only the selected experts are normalized in the subsequent process to obtain the selection probability of each expert.

[0073] Top-K selection and temperature adjustment: for each sample, the top K positions with the highest scores are selected from the scores of the E experts to form a mask, and the remaining positions are set to zero. Then, only the K selected experts are subjected to Softmax normalization.

[0074] In a feasible implementation, a temperature parameter is introduced to adjust the sharpness of the distribution, thereby controlling the sparsity of the routing. Specifically, the quotient of each selection probability and a set temperature parameter can be calculated, and the set temperature parameter is used to control the smoothness of the conditional distribution. Thereafter, the K expert subnetworks can be normalized.

[0075] Each expert subnet transforms the input features, and the gating network gives the corresponding selection probability. Finally, the expert outputs are weighted and summed with the selection probability as the weight to obtain the final output feature of the MoE module. This process only performs weighted fusion on the K activated experts, thereby realizing domain adaptive feature screening and transformation.

[0076] In a feasible implementation, a cascaded residual module can also be applied. The screened general feature is transmitted into the cascaded residual module.

[0077] The general feature is transmitted into the cascaded residual module.

[0078] In the cascaded residual module, the first layer is used to obtain a screened general feature from the general feature.

[0079] From the second layer, each layer obtains the screened general features processed by the previous processing layer, and point-by-point multiplies the gating value to obtain a shallow layer feature; the shallow layer feature is added to the general feature extracted by the transformer of the current layer, and then the current layer screened feature is obtained after passing through the mixed expert processing layer, and the current layer screened feature is taken as the input of the next layer; until the last layer outputs the final screened general feature.

[0080] The shallow layer feature usually contains more fine-grained acoustic details, but as the network deepens, this information is easily submerged or lost; at the same time, the dependence of networks of different depths on shallow information is not the same. For this reason, a cascaded residual module is designed in the network: on the one hand, to avoid the complete loss of shallow information at the deep layer, and on the other hand, to let the model adaptively control the strength of the residual information. Specifically, the first layer input is the original input of the current layer Layer input = current layer original input + (gate value Last layer residual ), gate value = , is a learnable scalar for each layer.

[0081] In step S104, the screened general features from the transformer-mixed expert path are aligned and fused across layers and modalities, so that the fused features have local details, global context and speaker-specific information, thereby improving the robustness and discriminability of subsequent voiceprint embedding.

[0082] Herein, the specific form of the feature fusion manner is not limited. The fusion manner can adopt a plurality of forms such as concatenation, addition, weighted summation, gate fusion or attention fusion. The concatenation manner is simple to implement, and different layer features can be directly spliced in the channel dimension, and then compressed to the target dimension through dimension reduction convolution or fully connected layer; the addition manner requires consistent feature dimensions, which can be aligned in dimension through linear projection, and then added at the element level, and the advantage is that the number of parameters is small and the calculation is fast.

[0083] The weighted summation can introduce learnable scalar weights or vector weights, so that the network automatically balances the importance of low-level details and high-level abstractions; the gate fusion can learn from the gating mechanism in the long short-term memory network, first calculate the update gate and the forget gate, and then linearly interpolate the features of the two branches, so as to adaptively adjust the fusion ratio under different frames, different frequency bands or different speakers.

[0084] Attention fusion can be further subdivided into self-attention, cross-attention or hierarchical attention. Self-attention can calculate inter-frame correlation again in the unified feature space after fusion, enhancing context consistency; cross-attention can let low-level features as query vectors and high-level features as key-value vectors, realizing bidirectional interaction of details inquiring global or global guiding details; hierarchical attention can calculate attention weights between different levels of features, making the fusion process have soft selection ability.

[0085] Since there may be differences in time domain resolution, frequency domain resolution or channel dimension between low-level features and high-level features, one-dimensional or two-dimensional convolution can be used for upsampling, downsampling or interpolation to align them in time and frequency dimensions; dynamic convolution or adaptive pooling can also be used to automatically adjust the resolution according to the input length.

[0086] In a feasible implementation, the final filtered general features can be used as query representation, key representation and value representation are obtained by stacking the filtered features corresponding to each layer in the cascaded residual module, and linear layers are used to linearly map the query representation, key representation and value representation respectively to obtain query features, key features and value features.

[0087] Thereafter, attention weights are obtained based on the query features and the key features, and finally the fusion features are obtained based on the attention weights and the value features.

[0088] Referring to Figure 3 , Figure 3 , the cross-attention fusion process provided by the embodiments of the present application is a dynamic selection in the layer dimension for each time frame, so that the model can adaptively balance between the robustness of the shallow layer and the discriminability of the deep layer. The last layer feature is used as Query to guide the fusion process to be more suitable for the downstream speaker discrimination task, and Key and Value are obtained by stacking all layer features . Linear layers are used to linearly map Query, Key and Value respectively to obtain , , , and then matrix multiplication, scaling and Softmax are performed on and to obtain attention weights, and finally the fusion features are obtained by multiplying the attention weights and the value features.

[0089] The fusion features obtained in this step contain long-time prosody and semantic information provided by the transformer path, and have robustness across channels and across noise, providing a rich, complementary and stable representation basis for the final voiceprint embedding vector.

[0090] ​In step S105, the high-dimensional, cross-layer, and cross-path fusion features are further refined into low-dimensional, compact, and measurable acoustic feature embedding vectors, which can accurately, stably, and reproducibly represent the speaker's identity, while meeting multiple requirements for storage, transmission, comparison, and privacy protection.

[0091] There are no restrictions on how to obtain the acoustic feature embedding vectors; multilayer perceptrons, residual networks, attention pooling networks, or prototype networks can be used. Multilayer perceptrons map the fused features to the target embedding dimension through layer-by-layer nonlinear transformations, and the number and width of its layers can be flexibly adjusted according to accuracy and speed requirements. Residual networks alleviate gradient vanishing through skip connections, allowing deep networks to still be effectively trained. Attention pooling networks can calculate weights on the time dimension, compressing variable-length speech into fixed-length vectors while highlighting the most critical segments for speaker differentiation. Prototype networks can maintain a prototype vector for each speaker during the training phase, giving the embedding space geometric interpretability.

[0092] Regarding the loss function, classification cross-entropy, metric learning loss, or a combination of both can be used. Metric learning loss can be further subdivided into triplet loss, improved triplet loss, additive angular margin loss, or hyperspherical angular margin loss. Their common goal is to make the embeddings of the same speaker as close as possible in terms of Euclidean or cosine distance, and to make the embeddings of different speakers as far apart as possible. Through joint optimization, the embedding space can achieve good clustering and discriminative properties.

[0093] The acoustic feature embedding vector output in this step has high discriminative power, high robustness, high compression rate and high security. It can be directly used in various scenarios such as speaker verification, speaker identification, voice anti-spoofing, smart access control, remote financial account opening, and personalized vehicle settings.

[0094] This application's embodiments compress the original speech in the temporal domain using a convolutional neural network encoder. This preserves low-level acoustic properties such as fundamental frequency and formants while effectively eliminating redundant sampling points. This allows subsequent networks to focus on key short frames with lower computational load, significantly improving front-end processing efficiency. The hybrid expert processing layer introduces a sparse gating mechanism at each output level. This mechanism dynamically activates the most relevant sub-networks based on implicit conditions such as speaker gender, channel, and noise environment, performing fine-grained feature selection and weighting. This suppresses irrelevant variations while preserving robust information shared across different scenarios. The selected general features from different levels are then fused in a unified dimension, allowing fine-grained short-term spectral characteristics to complement long-term global context, forming a fused representation that is both discriminative and generalizable. Finally, the acoustic feature recognition network maps the fused features into compact embedding vectors, which maintains compact clustering under various interference factors such as channel, language, and emotion, significantly reducing the error rate. Furthermore, it can complete cross-domain voiceprint comparison through simple cosine metric without retraining, achieving lightweight, robust, and easily transferable voiceprint recognition results.

[0095] Furthermore, a hybrid expert module is introduced after each layer, and dynamic routing is used to achieve adaptive feature transformation of the input, thereby completing implicit feature selection under different domain conditions and further improving recognition performance. By introducing a cascaded residual module to pass shallow key information step by step, key acoustic features are prevented from being weakened in deep representations. On this basis, cross-attention fusion is implemented to dynamically integrate multi-layer features in the hierarchical dimension, taking into account both the robustness of shallow features and the discriminativeness of deep features.

[0096] See Figure 4 , Figure 4 This is a schematic diagram of a voiceprint recognition system provided in an embodiment of this application. The system includes:

[0097] The audio data acquisition module is used to acquire the audio data to be recognized;

[0098] The feature sampling module is used to input the audio data to be identified into a pre-trained convolutional neural network encoder, perform downsampling, and extract low-level acoustic features.

[0099] The expert module processing module is used to extract the low-level acoustic features from the multi-layer transformer to obtain general features at different levels. The general features at each level are input to the hybrid expert processing layer to perform feature filtering on the general features to obtain the filtered general features.

[0100] The feature fusion module is used to perform feature fusion on the selected general features at the hierarchical dimension to obtain fused features;

[0101] The voiceprint recognition module is used to input the fused features into the acoustic feature recognition network to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to characterize the voiceprint information corresponding to the audio data to be recognized.

[0102] Based on the above embodiments, as a preferred embodiment, it further includes:

[0103] The temperature parameter control module is used to calculate the quotient of each selection probability and a set temperature parameter; the set temperature parameter is used to determine the smoothness of the conditional distribution.

[0104] This application also provides an embodiment corresponding to a computer-readable storage medium and a computer program product. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in the above method embodiments.

[0105] It is understood that if the methods in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0106] The computer-readable storage medium provided in this embodiment includes the method mentioned above, and has the same effect.

[0107] This application also provides an electronic device, see [link to document]. Figure 5 The present application provides a structural diagram of an electronic device, as shown in the embodiment. Figure 5 As shown, it may include a processor 1410 and a memory 1420.

[0108] The processor 1410 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 1410 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1410 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 1410 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 1410 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0109] The memory 1420 may include one or more computer-readable storage media, which may be non-transitory. The memory 1420 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 1420 is used to store at least the following computer program 1421, which, after being loaded and executed by the processor 1410, is capable of implementing the relevant steps in the methods executed by the electronic device side as disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 1420 may also include an operating system 1422 and data 1423, etc., and the storage method may be temporary storage or permanent storage. The operating system 1422 may include Windows, Linux, Android, etc.

[0110] In some embodiments, the electronic device may further include a display screen 1430, an input / output interface 1440, a communication interface 1450, a sensor 1460, a power supply 1470, and a communication bus 1480.

[0111] certainly, Figure 5 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiments of this application. In practical applications, the electronic device may include more than [other components]. Figure 5 More or fewer components as shown, or combinations of certain components.

[0112] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. As the system provided in the embodiments corresponds to the method provided in the embodiments, the description is relatively simple; relevant parts can be found in the method section.

[0113] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. It should be noted that those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

[0114] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A voiceprint recognition method, characterized in that, include: Obtain the audio data to be recognized; The audio data to be identified is input into a pre-trained convolutional neural network encoder, downsampled, and low-level acoustic features are extracted. The low-level acoustic features are extracted by a multi-layer transformer to obtain general features at different levels. The general features at each level are then input into a hybrid expert processing layer to perform feature filtering, resulting in filtered general features. The filtered general features are fused at the hierarchical level to obtain fused features; The fused features are input into an acoustic feature recognition network to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to characterize the voiceprint information corresponding to the audio data to be recognized.

2. The voiceprint recognition method according to claim 1, characterized in that, The structure of the hybrid expert processing layer includes several expert subnetworks and a gating network; each expert subnetwork includes a first one-dimensional convolutional layer, a modified linear unit, and a second one-dimensional convolutional layer; the gating network includes a statistical pooling layer and two gating subnetworks; the statistical pooling layer is used to calculate the mean and standard deviation in the time dimension respectively, and concatenate them to obtain a global summary vector; the gating subnetwork is used to map the global summary vector into a logical score through linear transformation; The hybrid expert processing layer also includes a normalized exponential function, which is used to normalize the selected expert subnets to obtain the selection probability of each expert subnet.

3. The voiceprint recognition method according to claim 2, characterized in that, Normalizing the selected expert subnets yields the selection probabilities for each expert subnet, including: After the general features described in each layer are input into the hybrid expert processing layer, the selection probabilities of the general features by E expert subnets are obtained. Select the positions corresponding to the K expert subnets with the highest selection probabilities as masks, set the remaining positions to zero, and normalize the K expert subnets to obtain the selection probabilities of each expert subnet.

4. The voiceprint recognition method according to claim 3, characterized in that, After obtaining the selection probabilities of the general feature for the E expert subnets, the process further includes: Calculate the quotient of each selection probability to a set temperature parameter; the set temperature parameter is used to determine the smoothness of the conditional distribution. Accordingly, the normalization process for the K expert subnets includes: The quotients of the K expert subnets are normalized.

5. The voiceprint recognition method according to claim 1, characterized in that, The common features described in each layer are input to the hybrid expert processing layer to perform feature filtering on the common features, resulting in the following filtered common features: The general features are then passed into the cascaded residual module; In the cascaded residual module, the first layer is used to obtain the filtered general features from the general features; Starting from the second layer, each layer obtains the filtered general features processed by the previous processing layer and multiplies them point by point with the gate value to obtain shallow features. The shallow features are added to the general features extracted by the transformer of this layer, and then passed through the hybrid expert processing layer to obtain the filtered features of this layer. The filtered features of this layer are used as the input of the next layer. The final output, after filtering, reveals general features.

6. The voiceprint recognition method according to claim 5, characterized in that, The filtered general features are fused at the hierarchical level to obtain fused features, including: The final filtered general features are used as the query representation, and the key representation and value representation are obtained by stacking the filtered features corresponding to each layer in the cascaded residual module. By using a linear layer to perform linear mapping on the query representation, the key representation, and the value representation respectively, query features, key features, and value features are obtained. Attention weights are obtained based on the query features and the key features; The fused features are obtained based on the attention weights and the value features.

7. A voiceprint recognition system, characterized in that, include: The audio data acquisition module is used to acquire the audio data to be recognized; The feature sampling module is used to input the audio data to be identified into a pre-trained convolutional neural network encoder, perform downsampling, and extract low-level acoustic features. The expert module processing module is used to extract the low-level acoustic features from the multi-layer transformer to obtain general features at different levels. The general features at each level are input to the hybrid expert processing layer to perform feature filtering on the general features to obtain the filtered general features. The feature fusion module is used to perform feature fusion on the selected general features at the hierarchical dimension to obtain fused features; The voiceprint recognition module is used to input the fused features into the acoustic feature recognition network to obtain an acoustic feature embedding vector; the acoustic feature embedding vector is used to characterize the voiceprint information corresponding to the audio data to be recognized.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program, which, when executed, implements the steps of the method as described in any one of claims 1 to 6.

Images