Channel-aware hybrid voiceprint recognition method, device, equipment and medium

Abstract

The application relates to the field of artificial intelligence and discloses a mixed voiceprint recognition method and device based on channel sensing, equipment and a medium, which comprises the following steps: performing feature compression processing on a voice signal, performing multidimensional statistical analysis on a basic time-frequency feature tensor obtained after the processing, and generating a channel context vector; obtaining dynamic activation weights of a plurality of expert networks according to the channel context vector; inputting the basic time-frequency feature tensor into the plurality of expert networks in parallel to perform frequency domain feature processing, and generating a plurality of expert feature tensors; weighting and fusing the plurality of expert feature tensors according to the dynamic activation weights to obtain a fused feature tensor, extracting a speaker embedding vector, performing voiceprint comparison and recognition, and obtaining a voiceprint recognition result. The application can be applied to business scenes such as finance and medical treatment, can obtain a fused speaker embedding by sensing channel characteristics and adaptively activating expert networks, can maintain stable voiceprint extraction and recognition in a complex and changeable channel environment, and can improve voiceprint recognition accuracy and reliability.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology and can be applied to business areas such as fintech and healthcare, particularly to a hybrid voiceprint recognition method, apparatus, device, and medium based on channel awareness. Background Technology

[0002] Currently, voiceprint recognition technology is widely used in finance, security, and intelligent customer service. In the financial sector, scenarios such as remote bank account opening, insurance telephone follow-ups, and mobile payment voiceprint verification can achieve efficient and seamless user identity authentication through voiceprint recognition. In the healthcare sector, scenarios such as remote consultation platforms, intelligent customer service systems, and electronic medical record access control can also use voiceprint recognition to verify the identities of doctors and patients, ensuring the security of medical data and the compliance of medical services.

[0003] In actual business operations, user voice is often affected by various factors such as differences in terminal devices, network encoding and decoding distortion, background noise, and changes in the channel environment. Most existing voiceprint recognition schemes are based on deep neural networks with fixed structures (such as ResNet, ECAPA-TDNN, CAM++, etc.), and their feature extraction paths are static, making it difficult to adapt to the complex and ever-changing channel environment in actual business operations, thus reducing the accuracy and reliability of voiceprint recognition schemes. Summary of the Invention

[0004] The main objective of this invention is to provide a channel-aware hybrid voiceprint recognition method, apparatus, device, and medium, aiming to solve the technical problem of maintaining accurate and reliable voiceprint recognition in complex channel environments in the prior art.

[0005] The technical solution of the present invention is as follows: The first aspect of this invention provides a channel-aware hybrid voiceprint recognition method, comprising: The system receives the speech signal to be processed, performs preprocessing and feature compression on the speech signal, and then obtains the basic time-frequency feature tensor. Perform multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate the corresponding channel context vector; Gated routing processing is performed based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks. The basic time-frequency feature tensor is input in parallel into multiple frequency domain perception expert networks. Each frequency domain perception expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors. The multiple expert feature tensors are weighted and fused according to the dynamic activation weights of each frequency domain sensing expert network to obtain a fused feature tensor. The speaker embedding vector is extracted based on the fusion feature tensor, and the speaker embedding vector is used for voiceprint comparison and recognition to obtain the voiceprint recognition result.

[0006] A second aspect of the present invention provides a channel-aware hybrid voiceprint recognition device, comprising: The preprocessing compression module is used to receive the speech signal to be processed, and after preprocessing and feature compression of the speech signal, obtain the basic time-frequency feature tensor. The channel description module is used to perform multi-dimensional statistical analysis on the basic time-frequency feature tensor and generate the corresponding channel context vector. The gated routing module is used to perform gated routing processing based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks. The frequency domain sensing module is used to input the basic time-frequency feature tensor into multiple frequency domain sensing expert networks in parallel. Each frequency domain sensing expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors. The weighted fusion module is used to perform weighted fusion of the multiple expert feature tensors according to the dynamic activation weights of each frequency domain perception expert network to obtain a fused feature tensor. The comparison and recognition module is used to extract the speaker embedding vector based on the fusion feature tensor, and perform voiceprint comparison and recognition based on the speaker embedding vector to obtain the voiceprint recognition result.

[0007] A third aspect of the present invention provides a computer device including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the channel-aware hybrid voiceprint recognition method described above.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the above-described channel-aware hybrid voiceprint recognition method.

[0009] Beneficial Effects: This invention discloses a channel-aware hybrid voiceprint recognition method, apparatus, device, and medium. Compared with existing technologies, this invention receives a speech signal to be processed, preprocesses and compresses the speech signal to obtain a basic time-frequency feature tensor; performs multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate a corresponding channel context vector; performs gated routing processing based on the channel context vector to obtain dynamic activation weights of multiple preset frequency-domain aware expert networks; inputs the basic time-frequency feature tensor in parallel into multiple frequency-domain aware expert networks, each of which performs frequency-domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors; performs weighted fusion of the multiple expert feature tensors based on the dynamic activation weights of each frequency-domain aware expert network to obtain a fused feature tensor; extracts a speaker embedding vector based on the fused feature tensor; and performs voiceprint comparison and recognition based on the speaker embedding vector to obtain a voiceprint recognition result. This invention can be applied to business scenarios such as fintech and healthcare. By sensing channel characteristics and adaptively activating frequency domain sensing experts, a dynamically fused speaker embedding is obtained, which enables stable voiceprint extraction and recognition in complex and ever-changing real-world channel environments, thereby improving the accuracy and reliability of voiceprint recognition. Attached Figure Description

[0010] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 A schematic diagram of an application environment for the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 2 A flowchart of a channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 3 This is a flowchart of step S201 in the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 4 This is a flowchart of step S302 in the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 5 This is a flowchart of step S202 in the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 6 This is a flowchart of step S203 in the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 7 This is a flowchart of step S204 in the channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 8 A schematic diagram of a model architecture for a channel-aware hybrid voiceprint recognition method provided in an embodiment of the present invention; Figure 9 A schematic diagram of the functional modules of the channel-aware hybrid voiceprint recognition device provided in an embodiment of the present invention; Figure 10 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. The embodiments of the invention are described below in conjunction with the accompanying drawings.

[0013] The channel-aware hybrid voiceprint recognition method provided in this invention can be applied to, for example... Figure 1 In the application environment, it includes a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0014] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as financial clients, healthcare clients, web browser applications, search applications, instant messaging tools, email clients, and / or social media platform software, etc. (for example only).

[0015] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0016] Server 105 can be a server providing various services, such as a backend server supporting the content browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend server can analyze and process received user requests and other data, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices. Server 105 can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the shortcomings of traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"), such as high management difficulty and weak business scalability. Server 105 can also be a server for a distributed system or a server combined with blockchain.

[0017] It should be noted that the channel-aware hybrid voiceprint recognition method provided in this application embodiment can generally be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103. Correspondingly, the channel-aware hybrid voiceprint recognition device provided in this embodiment can also be disposed in the first terminal device 101, the second terminal device 102, or the third terminal device 103. Alternatively, the channel-aware hybrid voiceprint recognition method provided in this embodiment can generally be executed by the server 105. Correspondingly, the channel-aware hybrid voiceprint recognition device provided in this embodiment can generally be disposed in the server 105.

[0018] It should be understood that the number of terminal devices, networks, and servers listed above is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be used.

[0019] like Figure 2 As shown, the channel-aware hybrid voiceprint recognition method provided in this embodiment of the invention specifically includes the following steps: S201. Receive the speech signal to be processed, and after preprocessing and feature compression of the speech signal, obtain the basic time-frequency feature tensor.

[0020] In this embodiment, the voice signal to be processed is received, which can be the user's voice information collected after establishing a connection with the user through mobile phone / landline calls, network voice communication, etc., and used as the recognition object for voiceprint comparison to realize the verification of the user's identity.

[0021] Upon receiving the speech signal to be processed, preprocessing and feature compression are performed. Preprocessing includes standard speech front-end processing such as pre-emphasis, framing, and windowing of the original speech signal to eliminate some linear distortion introduced by the recording device and smooth signal boundaries. Feature compression is performed by using a deep neural network front-end to reduce and abstract the high-dimensional acoustic features, extracting discriminative time-frequency structure information to obtain a basic time-frequency feature tensor. This basic time-frequency feature tensor retains intermediate feature representations of key time-frequency information. For example, it can be a four-dimensional tensor structure of [batch size, number of channels, number of compressed bandwidths, and compression time steps], providing a unified data interface for subsequent channel perception and expert processing.

[0022] For example, in the financial sector, during insurance call follow-up scenarios, users dial customer service hotlines via landlines or mobile phones. The system collects the voice stream in real time, performs preprocessing and front-end feature compression, and then outputs a basic time-frequency feature tensor after multi-level abstraction, laying the foundation for subsequent channel adaptive processing. Alternatively, in the healthcare sector, in remote consultation scenarios, doctors log in via voice through a mobile app. The system collects the doctor's high-definition voice recording, performs preprocessing and feature compression, and generates a basic time-frequency feature tensor as the data basis for doctor identification.

[0023] This embodiment converts the original speech signal into standard acoustic features and performs front-end compression, which reduces the data dimensionality while preserving speaker discrimination information. This provides efficient and stable input data for subsequent channel perception and expert routing, and improves the adaptability to speech with different sampling rates and bit rates.

[0024] S202. Perform multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate the corresponding channel context vector.

[0025] In this embodiment, the basic time-frequency feature tensor obtained based on feature compression retains rich local time-frequency structure information, which is used as the basis for channel perception to comprehensively characterize the current speech channel state. Specifically, multi-dimensional statistical analysis is performed on the basic time-frequency feature tensor, that is, multi-angle statistical measurement is performed on the basic time-frequency feature tensor along the time dimension to characterize the channel characteristics experienced by the current speech, thereby generating a corresponding channel context vector. This channel context vector is a low-dimensional but information-dense vector representation used to comprehensively reflect the channel state information such as equipment distortion, noise interference, and encoding / decoding distortion experienced by the current speech during transmission and acquisition, so as to serve as the basis for expert routing decisions in the subsequent gating network.

[0026] Specifically, when performing multidimensional statistical analysis, the mean, standard deviation, maximum and minimum values ​​of each frequency band channel over time can be calculated, and these statistics can be concatenated into a one-dimensional vector to form a basic description of the channel energy distribution and fluctuation range. Alternatively, inter-band correlation measures and time stability indicators can be introduced to measure the degree of coordination of amplitude changes in adjacent frequency bands and the uniformity of speech segment energy distribution on the time axis, thereby generating a more informative channel context vector.

[0027] For example, in mobile banking identity verification scenarios in the financial sector, users use their mobile phone microphones to perform voice verification in noisy environments. Multi-dimensional statistical analysis of the basic time-frequency feature tensor is performed to extract features such as frequency band energy distribution and frequency domain correlation, generating a channel context vector to characterize the channel state under the combined effects of the mobile device and complex background noise. Alternatively, in intelligent customer service scenarios in the healthcare sector, when patients conduct voice consultations using different brands of terminal devices, multi-dimensional statistical analysis generates a channel context vector to distinguish the frequency response characteristics of different terminal devices, providing an accurate channel state description for subsequent dynamic selection of experts.

[0028] S203. Perform gated routing processing based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks.

[0029] In this embodiment, multiple frequency domain-aware expert networks are pre-constructed. Each frequency domain-aware expert network is a feature transformation sub-network optimized for a specific frequency domain distortion mode or specific channel conditions, meaning each expert has different feature extraction preferences and compensation capabilities. Before performing feature processing on the basic time-frequency feature tensor through multiple frequency domain-aware expert networks, the generated channel context vector is first input to a gating network for gating routing processing. That is, based on the current channel state of the speech, dynamic activation weights are assigned to multiple pre-defined frequency domain-aware expert networks. These dynamic activation weights are normalized probability distribution vectors, with each element corresponding to the activation degree of an expert. The larger the weight, the more suitable the expert is for processing the current speech.

[0030] Specifically, gated routing can be implemented using a lightweight fully connected network. This network receives the channel context vector, performs a nonlinear transformation in the hidden layer, and outputs an original score equal to the number of experts. This score is then normalized to dynamic activation weights using the Softmax function, ensuring that the sum of all weights is 1. Alternatively, a self-attention enhancement mechanism can be introduced on top of the lightweight fully connected network to adaptively weight the features of each dimension within the channel context vector. This strengthens the high-order statistics most critical for current channel discrimination, and then generates dynamic activation weights, improving the accuracy of routing decisions.

[0031] For example, in the telephone banking scenario in the financial sector, when a user dials via a landline, the channel context vector characterizes the fixed-line channel characteristics of clear low frequencies and attenuated high frequencies. The gated routing process outputs higher dynamic activation weights to experts skilled in high-frequency compensation to correct the spectrum loss caused by fixed-line bandwidth limitations. Alternatively, in the remote consultation scenario in the healthcare sector, when a doctor logs in via voice over the network, the gated routing process identifies the spectrum distortion patterns introduced by network encoding and decoding based on the channel context vector. It dynamically activates experts skilled in suppressing encoding and decoding distortion and assigns them higher weights to improve recognition accuracy in weak network environments.

[0032] This embodiment establishes a dynamic mapping relationship between channel state and expert capabilities through gated routing processing. It can adaptively select the optimal expert combination according to the current channel characteristics of the speech, thereby realizing the dynamic adjustment of feature extraction strategy and better adapting to complex and ever-changing channel environments.

[0033] S204. The basic time-frequency feature tensor is input in parallel into multiple frequency domain perception expert networks. Each frequency domain perception expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors.

[0034] In this embodiment, the number of frequency domain sensing expert networks can be flexibly configured according to actual deployment needs. Each expert is an independent feature transformation sub-network, receiving the same basic time-frequency feature tensor as input. That is, the same basic time-frequency feature tensor is simultaneously fed into multiple independent frequency domain sensing expert networks for frequency domain feature processing. There is no data dependency between the expert networks, thereby fully utilizing the hardware's parallel computing capabilities to improve processing efficiency. Specifically, the frequency domain feature processing involves each expert transforming the input features according to its own internal parameters to extract or enhance discriminative information for specific frequency bands and specific structures, thereby generating corresponding expert feature tensors.

[0035] For example, three frequency domain perception expert networks can be preset, focusing on low-frequency enhancement, high-frequency compensation, and full-frequency equalization respectively. Each expert network contains convolutional layers and attention layers. By configuring different parameters, feature extraction of different frequency domain modes can be achieved. After parallel processing of the basic time-frequency feature tensor, three expert feature tensors are generated for subsequent weighted fusion.

[0036] This embodiment expands single feature extraction into parallel feature processing of multiple expert networks through parallel input and multi-expert collaborative processing. This enables the simultaneous handling of multiple channel conditions, avoids the performance bottleneck of a single network under unknown channel conditions, and thus improves the richness and robustness of voiceprint feature extraction.

[0037] S205. The multiple expert feature tensors are weighted and fused according to the dynamic activation weights of each frequency domain sensing expert network to obtain a fused feature tensor.

[0038] In this embodiment, after obtaining multiple expert feature tensors, the multiple expert feature tensors are weighted and fused according to the dynamic activation weights of each frequency-domain sensing expert network generated by the gated routing, resulting in a fused feature tensor. Specifically, the weighted fusion is performed by linearly weighting and summing each expert feature tensor according to the dynamic activation weights output by the gated network. The resulting fused feature tensor retains the general representation capability of the original front-end features while incorporating expert knowledge optimized for the current channel.

[0039] Specifically, fusion can be achieved by using element-wise weighted summation, where each element of the fused feature tensor is equal to the sum of the products of the corresponding element of each expert feature tensor and its dynamic activation weight; or by performing gated residual connection based on element-wise weighted summation, the weighted fused features are superimposed with the basic time-frequency feature tensor, thereby strengthening the key speaker information in the original features and preventing information loss caused by excessive transformation during expert processing.

[0040] For example, in a telephone banking scenario in the financial sector, the gating network assigns a weight of 0.6 to the high-frequency compensation expert, 0.3 to the full-frequency equalization expert, and 0.1 to the low-frequency enhancement expert. The expert feature tensors output by the three experts are then dynamically weighted and fused according to these weights to obtain a fused feature tensor, effectively compensating for the spectrum loss caused by fixed-line bandwidth limitations. Alternatively, in a remote consultation scenario in the healthcare sector, the gating network assigns a weight of 0.5 to the noise reduction expert, 0.4 to the device equalization expert, and 0.1 to the general expert. This ensures that the weighted fused feature tensor possesses both noise suppression and device adaptability, guaranteeing accurate identification in noisy hospital environments.

[0041] This embodiment achieves adaptive integration of multi-expert knowledge through dynamic weighted fusion processing, enabling the fused feature tensor to accurately reflect the optimal feature representation under the current channel conditions. This avoids the limitations of a single expert and prevents feature ambiguity caused by consistent contributions from all experts, thereby improving the discriminative power of voiceprint recognition.

[0042] S206. Based on the fusion feature tensor, the speaker embedding vector is extracted, and the speaker embedding vector is used for voiceprint comparison and recognition to obtain the voiceprint recognition result.

[0043] In this embodiment, the speaker embedding vector is further extracted from the fused feature tensor obtained by weighted fusion. Specifically, this can be achieved through multiple dense convolutional blocks and statistical pooling layers. The dense convolutional blocks further aggregate time-frequency information, while the statistical pooling layers compress the variable-length feature sequence into a fixed-dimensional vector. Finally, the vector is mapped to the speaker embedding space through a fully connected layer to obtain a fixed-dimensional low-dimensional vector representation, i.e., the speaker embedding vector, which is used to compactly represent the speaker's voiceprint identity features and has high discriminativeness and distinguishability.

[0044] Voiceprint comparison and recognition are performed based on the extracted speaker embedding vector. For example, cosine similarity is used to represent the degree of matching between the speaker embedding vector and the pre-stored voiceprint feature vector. When the cosine similarity is greater than a preset threshold, it is determined to be the same speaker; otherwise, it is determined to be different speakers, thus obtaining the corresponding voiceprint recognition result.

[0045] This embodiment extracts the speaker embedding vector from the fused feature tensor and performs voiceprint comparison and recognition. Based on channel-adaptive feature extraction, it facilitates faster retrieval and comparison of large-scale voiceprint databases, thereby improving the response speed and accuracy of voiceprint recognition in complex channel environments.

[0046] In the above embodiments, the present invention discloses a channel-aware hybrid voiceprint recognition method. The method involves receiving a speech signal to be processed, preprocessing and compressing the speech signal to obtain a basic time-frequency feature tensor; performing multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate a corresponding channel context vector; performing gated routing processing based on the channel context vector to obtain dynamic activation weights for multiple preset frequency-domain aware expert networks; inputting the basic time-frequency feature tensor in parallel into the multiple frequency-domain aware expert networks, with each frequency-domain aware expert network performing frequency-domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors; weighting and fusing the multiple expert feature tensors according to the dynamic activation weights of each frequency-domain aware expert network to obtain a fused feature tensor; extracting a speaker embedding vector based on the fused feature tensor; and performing voiceprint comparison and recognition based on the speaker embedding vector to obtain a voiceprint recognition result. This invention can be applied to business scenarios such as fintech and healthcare. By sensing channel characteristics and adaptively activating frequency domain sensing experts, a dynamically fused speaker embedding is obtained, which enables stable voiceprint extraction and recognition in complex and ever-changing real-world channel environments, thereby improving the accuracy and reliability of voiceprint recognition.

[0047] In one embodiment, such as Figure 3 As shown, step S201 includes: S301. Receive the speech signal to be processed, and convert the speech signal into a fixed-dimensional acoustic feature sequence to form an initial input tensor; S302. Extract and compress the time-frequency information of the initial input tensor to obtain the corresponding basic time-frequency feature tensor.

[0048] In this embodiment, after receiving a segment of speech signal to be processed, it is first preprocessed into a fixed-dimensional acoustic feature sequence. That is, the original waveform signal is converted into a parameterized representation with fixed frame length, frame shift, and feature dimensions, such as FBANK, MFCC, or PLP features, using a standard speech signal processing algorithm. These acoustic feature sequences are then organized in batches to form an initial input tensor. For example, if the system receives a speech signal with a duration of 2 to 10 seconds, it first performs pre-emphasis and framing processing, with a frame length of 25ms and a frame shift of 10ms. 80-dimensional FBANK features are extracted from each frame to form an initial input tensor with a structure of [1, 80, T], where T is the number of time frames.

[0049] Subsequently, the initial input tensor undergoes time-frequency information extraction and compression. A convolutional neural network is used to abstract the initial input tensor layer by layer, reducing the temporal resolution and expanding the receptive field to extract high-level time-frequency structural information, thus obtaining the corresponding basic time-frequency feature tensor. For example, the initial input tensor is fed into a front-end feature compression module, which consists of several stacked residual convolutional blocks. After multi-level downsampling and channel expansion, it outputs a high-dimensional four-dimensional tensor.

[0050] This embodiment achieves a standardized conversion from the original speech waveform to features usable by deep networks by converting the original speech signal into a fixed-dimensional acoustic feature sequence and compressing it into a low-dimensional basic time-frequency feature tensor. While retaining key time-frequency information, it significantly reduces the data dimensionality and improves the efficiency and feature representation capability of voiceprint recognition.

[0051] In one embodiment, such as Figure 4 As shown, step S302 includes: S401. Input the initial input tensor into several stacked residual convolutional blocks, and use the output of the previous residual convolutional block as the input of the next residual convolutional block. S402. The input tensor is subjected to layer-by-layer residual convolution and downsampling processing through the stacked residual convolution blocks to extract and compress the time-frequency information in the input tensor. S403. Use the output of the last residual convolution block as the basic time-frequency feature tensor.

[0052] In this embodiment, when extracting and compressing the time-frequency information of the initial input tensor, it is input into several stacked residual convolutional blocks. The residual convolutional block is a basic neural network unit containing convolutional layers, batch normalization layers, activation functions and skip connections. Several residual convolutional blocks are stacked to form a front-end feature compression module. The output of the previous residual convolutional block is directly used as the input of the next residual convolutional block, forming a sequential data stream.

[0053] The input tensor is processed by layer-by-layer residual convolution and downsampling through several stacked residual convolutional blocks. That is, each layer extracts local time-frequency patterns through convolutional kernels and reduces the temporal resolution through convolution or pooling operations with a stride greater than 1. At the same time, the feature representation capability is improved by increasing the number of channels. After layer-by-layer processing, the output of the last residual convolutional block is used as the basic time-frequency feature tensor.

[0054] In practice, the front-end feature compression module can be configured to contain four stacked residual convolutional blocks (ResBlock-1 to ResBlock-4). ResBlock-1 receives an initial input tensor of shape [batch size, number of channels, number of frequency bands, time step], extracts the primary time-frequency structure through a 3×3 convolutional kernel, and outputs it after 2x downsampling; ResBlock-2 receives the output of ResBlock-1, extracts the intermediate spectral pattern, and downsamples it again by 2x; ResBlock-3 and ResBlock-4 perform layer-by-layer feature abstraction and downsampling in sequence, and the output of the last residual convolutional block (ResBlock-4) is the basic time-frequency feature tensor.

[0055] This embodiment extracts rich hierarchical time-frequency features by downsampling layer by layer through stacked residual convolutional blocks. This not only preserves rich local time-frequency structure information, but also provides sufficiently fine-grained feature representations for subsequent channel modeling, achieving a balance between computational efficiency and feature abstraction capabilities.

[0056] In one embodiment, such as Figure 5 As shown, step S202 includes: S501. Obtain the energy dimension data and time dimension data of the basic time-frequency feature tensor in each frequency band channel; S502. Perform multi-dimensional statistics on the energy dimension data and time dimension data of each frequency band channel to extract peak energy characteristics, total energy characteristics, adjacent frequency band correlation characteristics and time stability characteristics. S503. The peak energy feature, total energy feature, adjacent frequency band correlation feature and time stability feature are spliced ​​together and flattened to form the channel context vector.

[0057] In this embodiment, the basic time-frequency feature tensor output by the front-end feature compression module serves as both the input to the expert network and the basis for channel perception. During channel perception, the tensor is subjected to multi-angle statistical analysis along the time dimension to generate a low-dimensional but information-dense channel context vector. Specifically, during the statistical analysis, not only are the mean and standard deviation of each frequency band channel calculated over time, but also higher-order statistics such as peak energy features, total energy features, adjacent frequency band correlation features, and time stability features are further extracted to more comprehensively and reliably characterize the speech channel state.

[0058] Specifically, the energy and time dimension data of the basic time-frequency feature tensor for each frequency band channel are first obtained. The energy dimension data is the energy statistics calculated along the frequency band dimension, reflecting the energy distribution of different frequency bands. The time dimension data is the statistics calculated along the time dimension, reflecting the temporal variation characteristics of the speech signal. Then, multi-dimensional statistics are performed on the energy dimension data and time dimension data of each frequency band channel to extract peak energy features, total energy features, adjacent frequency band correlation features, and time stability features. The peak energy feature refers to the maximum energy value of the channel in time, used to detect sudden interference or clipping distortion. The total energy feature is the cumulative or average energy of each channel in time, characterizing the overall speech activity level and helping to distinguish between silent segments and effective speech. The adjacent frequency band correlation feature is a statistical measure of the smoothness of amplitude changes in adjacent frequency bands, capturing spectral distortion caused by uneven microphone frequency response or codec filtering effects. The time stability feature is an indicator that evaluates the continuity and stationarity of speech through secondary statistics (such as first calculating the average value of each frame frequency band, and then calculating the variance of these average values ​​in time). The above features are concatenated by channel and then flattened to form a one-dimensional vector, namely the channel context vector.

[0059] This embodiment extracts four complementary high-order statistical features from the energy and time dimensions. The channel context vector formed by splicing these features is information-dense and dimensionally controllable, providing a precise channel state description for the gated network, thereby improving the accuracy and reliability of expert routing decisions.

[0060] In one embodiment, such as Figure 6 As shown, step S203 includes: S601. Input the channel context vector into the gating network, parse the channel context vector through the gating network, and identify the channel distortion mode of the current speech signal based on the parsing result; S602. Based on the channel distortion mode, perform dynamic activation tendency scoring on multiple preset frequency domain sensing experts; S603. Normalize the dynamic activation tendency score to obtain the dynamic activation weight.

[0061] In this embodiment, the gating network is a lightweight neural network containing one or more fully connected layers to establish a nonlinear mapping between the channel context vector and expert activation weights. Specifically, the channel context vector is input into the gating network, which parses the channel context vector and identifies the channel distortion mode of the current speech signal based on the parsing result. That is, the type of channel interference experienced by the current speech signal is determined based on the parsed channel context vector. For example, when the correlation between adjacent bands in the frequency domain is low and the time stability is poor, it is identified as a network encoding / decoding distortion mode; when the low-frequency energy is high and the high-frequency energy is attenuated, it is identified as a fixed-line channel attenuation mode, and so on.

[0062] The gated network then dynamically scores the activation tendency of multiple preset frequency domain sensing experts based on the identified channel distortion patterns, outputting an original adaptation score for each expert. The higher the score, the more suitable the expert is for handling the current channel distortion. Finally, the original scores are converted into a probability distribution using functions such as Softmax, and the output is a weight vector with a dimension equal to the total number of experts. Each element of the weight vector is the dynamic activation weight of the corresponding expert, and the sum of the dynamic activation weights of all experts is 1. This weight vector directly determines the contribution ratio during subsequent expert fusion, so as to realize the dynamic allocation of expert resources.

[0063] This embodiment uses a gated network to parse the channel context vector and identify channel distortion patterns, transforming channel features into interpretable expert scheduling strategies. This enables the corresponding expert network to be adaptively activated based on the explicit channel state, thereby improving the adaptive capability and recognition stability of voiceprint recognition in complex and variable channel environments.

[0064] In one embodiment, such as Figure 7 and Figure 8 As shown, each frequency domain aware expert network includes a frequency domain attention module, a spatial convolution module, a temporal convolution module, and a residual connection module. Step S204 includes: In each of the frequency domain awareness expert networks, the basic time-frequency feature tensor is adaptively weighted in the frequency band dimension by the frequency domain attention module to obtain the frequency domain weighted feature tensor. The spatial convolution module performs one-dimensional convolution processing on the frequency domain weighted feature tensor along the frequency band axis to obtain the spatial convolution feature tensor. The spatial convolution feature tensor is convolved along the time axis by the temporal convolution module to obtain the temporal convolution feature tensor. The temporal convolutional feature tensor is residually concatenated with the basic time-frequency feature tensor to obtain the expert feature tensor of each frequency domain perception expert network.

[0065] In this embodiment, the frequency domain-aware expert network consists of multiple parallel-running independent feature transformation sub-networks. Each expert contains a four-level cascaded structure: a frequency domain attention module, a spatial convolution module, a temporal convolution module, and a residual connection module. Each expert receives the same basic time-frequency feature tensor as input, but due to differences in their internal parameters, they possess different feature extraction preferences.

[0066] In each frequency domain sensing expert network, the basic time-frequency feature tensor of the current input is processed in parallel. Specifically, the basic time-frequency feature tensor of the input is first weighted in the frequency band dimension by the frequency domain attention module to highlight the frequency bands that are more discriminative under the current channel conditions, and the frequency domain weighted feature tensor is obtained. The specific implementation can be a channel attention mechanism (such as SE Block) or a more frequency domain self-attention structure, thereby suppressing the frequency bands that are severely affected by noise or distortion and enhancing the response of robust frequency bands.

[0067] Then, the frequency domain weighted feature tensor is processed by one-dimensional convolution along the frequency band axis through the spatial convolution module to obtain the spatial convolution feature tensor. Here, "spatial" refers to the frequency domain direction. For example, a one-dimensional convolution kernel can be used to slide along the frequency band axis to capture local spectral structures (such as formant morphology). It can also be combined with depthwise separable convolution to reduce computational overhead.

[0068] Then, the spatial convolution feature tensor is convolved along the time axis by the temporal convolution module to model the temporal dynamic characteristics of speech and obtain the temporal convolution feature tensor. Preferably, in order to expand the receptive field, dilated convolution or grouped convolution strategies can be used to effectively capture long-term dependencies.

[0069] Finally, the temporal convolutional feature tensor is residually concatenated with the basic time-frequency feature tensor to obtain the corresponding expert feature tensor. That is, the final output of the expert network does not directly use the convolution result, but rather adds it to the original input, i.e., the basic time-frequency feature tensor, to form a residual structure. This helps with stable gradient propagation and preserves key information from the original features. All expert networks run in parallel, each generating an expert feature tensor of the same dimension as the input, thus obtaining the expert feature tensors for each frequency domain-aware expert network.

[0070] This embodiment employs a four-stage cascaded processing approach, consisting of frequency domain attention, spatial convolution, temporal convolution, and residual connections. Frequency domain attention enables adaptive focusing on the optimal frequency band under the current channel conditions, while spatial and temporal convolutions respectively exploit frequency domain locality and temporal continuity. Finally, residual connections preserve key speaker information from the original features while stabilizing gradient propagation, thereby ensuring the feature extraction capabilities of each expert network under the corresponding channel conditions and guaranteeing the reliability of voiceprint feature extraction.

[0071] In one embodiment, step S206 includes: The fusion feature tensor is subjected to multi-layer convolution and pooling to aggregate the time-frequency information in the fusion feature tensor to obtain the speaker embedding vector. Calculate the similarity between the speaker embedding vector and the current user's pre-stored voiceprint feature vector, compare the similarity with a preset threshold, and output the corresponding voiceprint recognition result.

[0072] In this embodiment, after weighting and fusing the outputs of multiple expert networks based on dynamic activation weights to obtain a fused feature tensor, the fused feature tensor is further subjected to multi-layer convolution and pooling processing. Specifically, the fused feature tensor is subjected to deep feature aggregation through multiple dense convolutional blocks. Each dense block contains multiple convolutions and channel concatenation. The representation capability is improved through feature reuse, and time-frequency information is gradually aggregated and the feature abstraction level is improved. The output of the last dense block is compressed by the pooling layer to finally generate a fixed-length speaker embedding vector. This speaker embedding vector is a fixed-dimensional low-dimensional vector that can compactly represent the speaker's identity.

[0073] In the voiceprint feature comparison stage, the speaker's embedded vector is compared with the current user's pre-stored voiceprint feature vector. The similarity is calculated using methods such as cosine similarity or Euclidean distance. This pre-stored voiceprint feature vector is the baseline voiceprint feature extracted and stored by the user during registration. The similarity is used to determine the similarity between the currently collected voiceprint and the user's pre-stored baseline voiceprint. The calculated similarity is compared with a preset threshold. If it is greater than or equal to the preset threshold, it indicates that the current voiceprint is identified as belonging to the same person, and a voiceprint recognition result is output, confirming the identity. If it is less than the preset threshold, it indicates that the voiceprint may not belong to the same person, and a voiceprint recognition result is output, failing the recognition, and the current service is blocked to ensure service security.

[0074] This embodiment uses multi-layer dense convolution and pooling processing to further extract high-level speaker discrimination information from the channel adaptive fusion features, generating a compact and highly discriminative speaker embedding vector. Furthermore, it uses a similarity and threshold comparison mechanism to perform fast and reliable voiceprint identity verification, ensuring the efficiency and reliability of voiceprint recognition in complex and ever-changing channel environments.

[0075] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0076] Further reference Figure 9 As a response to the above Figure 2The present invention provides an embodiment of a channel-aware hybrid voiceprint recognition device, which, in accordance with the implementation of the method shown, provides an embodiment of a channel-aware hybrid voiceprint recognition device. Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0077] like Figure 9 As shown, the channel-aware hybrid voiceprint recognition device 90 described in this embodiment includes: The preprocessing compression module 901 is used to receive the speech signal to be processed, and after preprocessing and feature compression processing of the speech signal, obtain the basic time-frequency feature tensor. The channel description module 902 is used to perform multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate a corresponding channel context vector. The gated routing module 903 is used to perform gated routing processing based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks. The frequency domain sensing module 904 is used to input the basic time-frequency feature tensor into multiple frequency domain sensing expert networks in parallel, and each frequency domain sensing expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors. The weighted fusion module 905 is used to perform weighted fusion of the multiple expert feature tensors according to the dynamic activation weights of each frequency domain perception expert network to obtain a fused feature tensor. The comparison and recognition module 906 is used to extract the speaker embedding vector based on the fusion feature tensor, and perform voiceprint comparison and recognition based on the speaker embedding vector to obtain the voiceprint recognition result.

[0078] The module referred to in this invention is a series of computer program instruction segments that can perform specific functions. It is more suitable than a program for describing the execution process of channel-aware hybrid voiceprint recognition. For specific implementation methods of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0079] In one embodiment, the preprocessing compression module 901 includes: The preprocessing unit is used to receive the speech signal to be processed, convert the speech signal into a fixed-dimensional acoustic feature sequence, and form the initial input tensor. The front-end compression unit is used to extract and compress the time-frequency information of the initial input tensor to obtain the corresponding basic time-frequency feature tensor.

[0080] In one embodiment, the front-end compression unit includes: The input unit is used to input the initial input tensor into several stacked residual convolutional blocks, with the output of the previous residual convolutional block serving as the input of the next residual convolutional block; The layer-by-layer compression unit is used to perform layer-by-layer residual convolution and downsampling processing on the input tensor through the stacked residual convolution blocks, and to extract and compress the time-frequency information in the input tensor. The output unit is used to take the output of the last residual convolutional block as the underlying time-frequency feature tensor.

[0081] In one embodiment, the channel description module 902 includes: The data acquisition unit is used to acquire the energy dimension data and time dimension data of the basic time-frequency feature tensor in each frequency band channel; The multi-dimensional statistical unit is used to perform multi-dimensional statistics on the energy dimension data and time dimension data of each frequency band channel to extract peak energy characteristics, total energy characteristics, adjacent frequency band correlation characteristics and time stability characteristics. The splicing and flattening unit is used to splice and flatten the peak energy characteristics, total energy characteristics, adjacent frequency band correlation characteristics, and time stability characteristics to form the channel context vector.

[0082] In one embodiment, the gated routing module 903 includes: The pattern recognition unit is used to input the channel context vector into the gating network, parse the channel context vector through the gating network, and identify the channel distortion mode of the current speech signal based on the parsing result; A dynamic activation scoring unit is used to perform dynamic activation tendency scoring on multiple preset frequency domain sensing experts based on the channel distortion mode. The normalization output unit is used to normalize the dynamic activation tendency score to obtain the dynamic activation weight.

[0083] In one embodiment, the frequency domain sensing module 904 includes: The frequency domain attention unit is used to adaptively weight the basic time-frequency feature tensor in the frequency band dimension through the frequency domain attention module in each of the frequency domain perception expert networks to obtain the frequency domain weighted feature tensor. A spatial convolution unit is used to perform one-dimensional convolution processing on the frequency domain weighted feature tensor along the frequency band axis through the spatial convolution module to obtain a spatial convolution feature tensor. The temporal convolution unit is used to perform convolution processing on the spatial convolutional feature tensor along the time axis through the temporal convolution module to obtain the temporal convolutional feature tensor. The residual connection unit is used to perform residual connection between the temporal convolutional feature tensor and the basic time-frequency feature tensor to obtain the expert feature tensor of each frequency domain perception expert network.

[0084] In one embodiment, the comparison and identification module 906 includes: An aggregation and extraction unit is used to perform multi-layer convolution and pooling processing on the fused feature tensor, aggregate the time-frequency information in the fused feature tensor, and obtain the speaker embedding vector. The similarity comparison unit is used to calculate the similarity between the speaker embedding vector and the current user's pre-stored voiceprint feature vector, and output the corresponding voiceprint recognition result after comparing the similarity with a preset threshold.

[0085] In the above embodiments, the present invention discloses a channel-aware hybrid voiceprint recognition device. The device receives a speech signal to be processed, performs preprocessing and feature compression on the speech signal to obtain a basic time-frequency feature tensor; performs multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate a corresponding channel context vector; performs gated routing processing based on the channel context vector to obtain dynamic activation weights of multiple preset frequency-domain aware expert networks; inputs the basic time-frequency feature tensor in parallel into the multiple frequency-domain aware expert networks, and each frequency-domain aware expert network performs frequency-domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors; performs weighted fusion of the multiple expert feature tensors based on the dynamic activation weights of each frequency-domain aware expert network to obtain a fused feature tensor; extracts a speaker embedding vector based on the fused feature tensor; and performs voiceprint comparison and recognition based on the speaker embedding vector to obtain a voiceprint recognition result. This invention can be applied to business scenarios such as fintech and healthcare. By sensing channel characteristics and adaptively activating frequency domain sensing experts, a dynamically fused speaker embedding is obtained, which enables stable voiceprint extraction and recognition in complex and ever-changing real-world channel environments, thereby improving the accuracy and reliability of voiceprint recognition.

[0086] Specific limitations regarding the channel-aware hybrid voiceprint recognition device can be found in the limitations of the channel-aware hybrid voiceprint recognition method described above, and will not be repeated here. Each module in the aforementioned channel-aware hybrid voiceprint recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0087] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0088] Another embodiment of the present invention provides a computer device, such as... Figure 10 As shown, the computer device 100 includes: One or more processors 1001 and memory 1002, Figure 10 The following description uses a processor 1001 as an example. The processor 1001 and the memory 1002 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0089] The processor 1001 is used to perform various control logics of the computer device 100. It can be any conventional processor, microprocessor, state machine, general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components or any combination of these components.

[0090] The memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the channel-aware hybrid voiceprint recognition method in the embodiments of the present invention. The processor 1001 executes various functional applications and data processing of the computer device 100 by running the non-volatile software programs, instructions, and units stored in the memory 1002, thereby implementing the channel-aware hybrid voiceprint recognition method in the above method embodiments.

[0091] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by one or more processors, perform the steps of the channel-aware hybrid voiceprint recognition method in any of the above method embodiments.

[0092] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0093] Based on the above description of the embodiments, those skilled in the art will understand that the methods described in the embodiments can be implemented using software plus necessary general-purpose hardware platforms. Of course, they can also be implemented using hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0094] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The computer program can be stored in a non-volatile, computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a memory, magnetic disk, floppy disk, flash memory, optical storage, etc.

[0095] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0096] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A channel-aware hybrid voiceprint recognition method, characterized in that, include: The system receives the speech signal to be processed, performs preprocessing and feature compression on the speech signal, and then obtains the basic time-frequency feature tensor. Perform multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate the corresponding channel context vector; Gated routing processing is performed based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks. The basic time-frequency feature tensor is input in parallel into multiple frequency domain perception expert networks. Each frequency domain perception expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors. The multiple expert feature tensors are weighted and fused according to the dynamic activation weights of each frequency domain sensing expert network to obtain a fused feature tensor. The speaker embedding vector is extracted based on the fusion feature tensor, and the speaker embedding vector is used for voiceprint comparison and recognition to obtain the voiceprint recognition result.

2. The channel-aware hybrid voiceprint recognition method according to claim 1, characterized in that, The process of receiving the speech signal to be processed, performing preprocessing and feature compression on the speech signal, and obtaining the basic time-frequency feature tensor includes: Receive the speech signal to be processed, and convert the speech signal into a fixed-dimensional acoustic feature sequence to form the initial input tensor; The initial input tensor is subjected to time-frequency information extraction and compression processing to obtain the corresponding basic time-frequency feature tensor.

3. The channel-aware hybrid voiceprint recognition method according to claim 2, characterized in that, The step of extracting and compressing time-frequency information from the initial input tensor to obtain the corresponding basic time-frequency feature tensor includes: The initial input tensor is input into several stacked residual convolutional blocks, and the output of the previous residual convolutional block is used as the input of the next residual convolutional block. The input tensor is subjected to layer-by-layer residual convolution and downsampling processing through the stacked residual convolution blocks to extract and compress the time-frequency information in the input tensor. The output of the last residual convolutional block is used as the basic time-frequency feature tensor.

4. The channel-aware hybrid voiceprint recognition method according to claim 1, characterized in that, The step of performing multi-dimensional statistical analysis on the basic time-frequency feature tensor to generate a corresponding channel context vector includes: Obtain the energy dimension data and time dimension data of the basic time-frequency feature tensor in each frequency band channel; Multi-dimensional statistics are performed on the energy dimension data and time dimension data of each frequency band channel to extract peak energy characteristics, total energy characteristics, adjacent frequency band correlation characteristics and time stability characteristics. The peak energy feature, total energy feature, adjacent frequency band correlation feature, and time stability feature are concatenated and flattened to form the channel context vector.

5. The channel-aware hybrid voiceprint recognition method according to claim 1, characterized in that, The step of performing gated routing processing based on the channel context vector to obtain multiple preset dynamic activation weights for frequency domain-aware expert networks includes: The channel context vector is input into a gating network, which then parses the channel context vector and identifies the channel distortion mode of the current speech signal based on the parsing result. Based on the channel distortion pattern, a dynamic activation tendency score is performed on multiple preset frequency domain sensing experts. The dynamic activation tendency score is normalized to obtain the dynamic activation weight.

6. The channel-aware hybrid voiceprint recognition method according to claim 1, characterized in that, Each frequency-domain awareness expert network includes a frequency-domain attention module, a spatial convolution module, a temporal convolution module, and a residual connection module. Each of these frequency-domain awareness expert networks performs frequency-domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors, including: In each of the frequency domain perception expert networks, the basic time-frequency feature tensor is adaptively weighted in the frequency band dimension by the frequency domain attention module to obtain the frequency domain weighted feature tensor. The spatial convolution module performs one-dimensional convolution processing on the frequency domain weighted feature tensor along the frequency band axis to obtain the spatial convolution feature tensor. The spatial convolution feature tensor is convolved along the time axis by the temporal convolution module to obtain the temporal convolution feature tensor. The time-convolutional feature tensor is residually concatenated with the basic time-frequency feature tensor to obtain the expert feature tensor of each frequency-domain perceptual expert network.

7. The channel-aware hybrid voiceprint recognition method according to claim 1, characterized in that, The speaker embedding vector is extracted based on the fused feature tensor, and voiceprint comparison and recognition are performed based on the speaker embedding vector to obtain the voiceprint recognition result, including: The fusion feature tensor is subjected to multi-layer convolution and pooling to aggregate the time-frequency information in the fusion feature tensor to obtain the speaker embedding vector. The similarity between the speaker embedding vector and the current user's pre-stored voiceprint feature vector is calculated, and the similarity is compared with a preset threshold before the corresponding voiceprint recognition result is output.

8. A channel-aware hybrid voiceprint recognition device, characterized in that, include: The preprocessing compression module is used to receive the speech signal to be processed, and after preprocessing and feature compression of the speech signal, obtain the basic time-frequency feature tensor. The channel description module is used to perform multi-dimensional statistical analysis on the basic time-frequency feature tensor and generate the corresponding channel context vector. The gated routing module is used to perform gated routing processing based on the channel context vector to obtain the dynamic activation weights of multiple preset frequency domain awareness expert networks. The frequency domain sensing module is used to input the basic time-frequency feature tensor into multiple frequency domain sensing expert networks in parallel. Each frequency domain sensing expert network performs frequency domain feature processing on the basic time-frequency feature tensor to generate multiple expert feature tensors. The weighted fusion module is used to perform weighted fusion of the multiple expert feature tensors according to the dynamic activation weights of each frequency domain perception expert network to obtain a fused feature tensor. The comparison and recognition module is used to extract the speaker embedding vector based on the fusion feature tensor, and perform voiceprint comparison and recognition based on the speaker embedding vector to obtain the voiceprint recognition result.

9. A computer device, characterized in that, Includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the channel-aware hybrid voiceprint recognition method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the channel-aware hybrid voiceprint recognition method according to any one of claims 1-7.

Images