Voiceprint verification system based on CANN architecture

Abstract

The invention relates to the technical field of biological feature recognition, in particular to a voiceprint verification system based on a CANN architecture. Through a frequency domain feature enhancement module, a time domain feature enhancement module, a time domain deep and shallow feature combination unit and a time domain and frequency domain feature weighted fusion unit, comprehensive modeling of audio signal global space-time consistency is realized. Multi-dimensional information related to a language structure is provided through establishment of a speech recognition continuity feature library, adversarial training is carried out through frequency domain weighted clustering and an adversarial training unit by utilizing GAN, the robustness of the model to adversarial samples is improved, and finally, through design of a self-adaptive resampling strategy unit and a regularization penalty term subunit, the robustness of the model to adversarial samples is improved. The adaptive capacity of the model to different audio qualities and sampling rates is enhanced, a regularization strategy is introduced to prevent overfitting, and the generalization ability and the performance stability of the model facing a novel counterfeiting means are remarkably improved by combining the capture of the long-time dependency modeling unit to the long-time dependency relationship of the audio signal.

Description

Technical Field

[0001] This invention relates to the field of biometric identification technology, and in particular to a voiceprint verification system based on the CANN architecture. Background Technology

[0002] With the rapid development of artificial intelligence technology, deepfake audio generation technology has made significant progress. Novel algorithms based on autoregressive models, generative adversarial networks, and diffusion models have greatly improved the realism of forged audio, making it close to real speech in terms of spectral characteristics and temporal coherence. However, the application of such technologies has also brought serious security risks, especially in sensitive areas such as financial transactions and judicial evidence collection, where forged audio may lead to problems such as identity impersonation and evidence tampering.

[0003] Currently, mainstream voiceprint verification technologies still rely on traditional methods. Although these methods can identify low-quality forged audio to some extent, their limitations are becoming increasingly apparent: First, the single reliance of existing technologies on frequency or time domain features leads to insufficient global spatiotemporal consistency modeling, making it difficult to cope with high-fidelity forged audio; Second, most solutions focus only on single-modal data and do not fully utilize the potential correlation of multimodal information; Third, advancements in adversarial example techniques allow forged audio to evade detection through stealth optimization, rendering traditional fixed feature extraction methods insufficiently robust; Fourth, existing models are typically trained on specific datasets, resulting in weak generalization ability and a significant performance drop when facing new forgery methods. Summary of the Invention

[0004] The purpose of this invention is to provide a voiceprint verification system based on the CANN architecture, aiming to solve the following technical problems in existing technologies: First, the single reliance on frequency domain or time domain features in existing technologies leads to insufficient global spatiotemporal consistency modeling, making it difficult to cope with high-fidelity forged audio; second, most solutions only focus on single-modal data and do not fully utilize the potential correlation of multimodal information; third, the advancement of adversarial example techniques allows forged audio to evade detection through stealth optimization, and traditional fixed feature extraction methods lack robustness; fourth, existing models are usually trained on specific datasets, have weak generalization ability, and their performance drops significantly when facing new forgery methods.

[0005] To achieve the above objectives, this invention employs a voiceprint verification system based on a CANN architecture, comprising an audio input module, a frequency domain feature enhancement module, a time domain feature enhancement module, a time domain deep and shallow feature combination unit, and a time domain and frequency domain feature weighted fusion unit. The audio input module includes an audio input unit, a resampling unit, a frequency domain information normalization unit, and an adaptive resampling strategy unit. The frequency domain feature enhancement module includes a frequency domain feature extraction unit, a frequency domain compression, expansion, and weighting processing unit, and a frequency domain weighted clustering and adversarial training unit. The time domain feature enhancement module includes a time domain shallow feature extraction unit, a time domain deep feature extraction unit, and a time domain feature fusion and weighting unit. The audio input module is connected to the frequency domain feature enhancement module, the frequency domain feature enhancement module is connected to the time domain feature enhancement module, the time domain feature enhancement module is connected to the time domain deep and shallow feature combination unit, and the time domain deep and shallow feature combination unit is connected to the time domain and frequency domain feature weighted fusion unit.

[0006] The audio input unit is used to receive audio signals from different devices, environments, and sampling rates as the initial input data of the system; the resampling unit is used to uniformly process audio signals with different sampling rates; the frequency domain information normalization unit is used to standardize the audio spectrum after resampling to eliminate the spectrum shift caused by different sampling rates; the adaptive resampling strategy unit is used to automatically adjust the accuracy and method of resampling based on factors such as the quality, signal-to-noise ratio, and sampling rate of the input audio.

[0007] The frequency domain feature extraction unit is used to convert the audio signal from the time domain to the frequency domain using a fast Fourier transform to obtain the spectral features of the audio; the frequency domain compression, expansion and weighting processing unit is used to compress and expand the frequency domain image using a 1D convolutional neural network and effectively connect it with the time domain features, while weighting the features of different frequency bands; the frequency domain weighted clustering and adversarial training unit is used to aggregate and weight the features of different frequency bands through a fully connected layer, perform training, and compare with fake audio to identify unnatural changes;

[0008] The temporal shallow feature extraction unit is used to capture local changes in audio signals using a temporal convolutional neural network; the temporal deep feature extraction unit is used to process long-term dependencies of audio signals using a temporal deep neural network to capture the changing trends of audio signals over long time scales; the temporal feature fusion and weighting unit is used to perform weighted fusion of temporal shallow and deep features and introduce a regularization strategy.

[0009] The unit that combines shallow and deep temporal features is used to combine shallow and deep temporal features to form a comprehensive understanding of audio temporal characteristics;

[0010] The time-domain and frequency-domain feature weighted fusion unit is used to deeply fuse time-domain and frequency-domain features, taking into account the consistency of audio signals in different dimensions.

[0011] The audio input unit includes a device audio subunit, an ambient audio subunit, and a sampling rate audio subunit.

[0012] The device audio subunit is used to receive audio signals from the device;

[0013] The environmental audio subunit is used to receive audio signals from the environment;

[0014] The sampling rate audio subunit is used to receive audio signals from the sampling rate.

[0015] The resampling unit includes an interpolation subunit and a downsampling subunit;

[0016] The interpolation subunit is used to improve the resolution of low sampling rate audio;

[0017] The downsampling subunit reduces the resolution of high-sampling-rate audio.

[0018] The frequency domain weighted clustering and adversarial training unit includes a weighted clustering subunit, an adversarial neural network subunit, and a fake audio comparison subunit.

[0019] The weighted clustering subunit is used to aggregate and weight features of different frequency bands;

[0020] The adversarial neural network subunit is used for training with GAN to learn the frequency domain feature variation patterns of normal audio.

[0021] The fake audio comparison subunit is used to compare the learned pattern with fake audio and identify unnatural changes.

[0022] The temporal feature fusion and weighting unit includes a multilayer perceptron network subunit and a regularization penalty term subunit.

[0023] The multilayer perceptron network subunit is used to weightedly fuse shallow and deep temporal features;

[0024] The regularization penalty term subunit is used to introduce a regularization strategy to prevent the model from overfitting.

[0025] The voiceprint verification system based on the CANN architecture also includes a long-term dependency modeling unit, which is connected to the time-domain feature enhancement module.

[0026] The long-term dependency modeling unit is used to model the long-term dependencies of audio signals through a TDNN network and identify details in fake audio that do not conform to the time sequence.

[0027] The voiceprint verification system based on the CANN architecture further includes a speech recognition continuous feature library and a fake audio output unit. The speech recognition continuous feature library is connected to the time-domain and frequency-domain feature weighted fusion unit, and the fake audio output unit is connected to the speech recognition continuous feature library.

[0028] The continuous feature library for speech recognition is used to extract language structure features using ASR technology from a large-scale real audio corpus to establish a continuous feature library, which serves as the core reference for judging the authenticity of audio.

[0029] The fake audio output unit is used to output a warning or indicator when fake audio is detected.

[0030] This invention discloses a voiceprint verification system based on a CANN architecture. The system deeply extracts and fuses frequency and time domain features through a frequency domain feature enhancement module and a time domain feature enhancement module, respectively. Combined with a time domain deep / shallow feature combination unit and a time and frequency domain feature weighted fusion unit, it achieves comprehensive modeling of the global spatiotemporal consistency of audio signals, effectively addressing the challenge of high-fidelity forged audio. The establishment of the speech recognition continuous feature library provides the system with multi-dimensional information related to language structure, enhancing the system's ability to identify forged audio and demonstrating the utilization of potential correlations in multimodal information. Furthermore, through the frequency domain weighted clustering and adversarial training unit, adversarial training using GANs is employed, improving the model's robustness to adversarial examples and reducing false positive and false negative rates. Finally, the design of the adaptive resampling strategy unit and the regularization penalty term subunit enhances the model's adaptability to different audio qualities and sampling rates. The introduction of a regularization strategy to prevent overfitting, combined with the long-term dependency modeling unit's capture of long-term dependencies in audio signals, significantly improves the model's generalization ability and performance stability when facing new forgery methods. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of the voiceprint verification system based on the CANN architecture of the present invention.

[0033] 1-Audio Input Module, 2-Frequency Domain Feature Enhancement Module, 3-Time Domain Feature Enhancement Module, 4-Time Domain Deep and Shallow Feature Combination Unit, 5-Time Domain and Frequency Domain Feature Weighted Fusion Unit, 6-Audio Input Unit, 7-Resampling Unit, 8-Frequency Domain Information Normalization Unit, 9-Adaptive Resampling Strategy Unit, 10-Frequency Domain Feature Extraction Unit, 11-Frequency Domain Compression, Dilation and Weighted Processing Unit, 12-Frequency Domain Weighted Clustering and Adversarial Training Unit, 13-Time Domain Shallow Feature Extraction Unit, 14-Time Domain Deep Feature Extraction Unit 15 - Temporal Feature Fusion and Weighting Unit 16 - Device Audio Subunit 17 - Environmental Audio Subunit 18 - Sampling Rate Audio Subunit 19 - Interpolation Subunit 20 - Downsampling Subunit 21 - Weighted Clustering Subunit 22 - Adversarial Neural Network Subunit 23 - Fake Audio Comparison Subunit 24 - Multilayer Perceptron Network Subunit 25 - Regularization Penalty Subunit 26 - Long-Term Dependency Modeling Unit 27 - Speech Recognition Continuous Feature Library 28 - Fake Audio Output Unit Detailed Implementation

[0034] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0035] Please see Figure 1 This invention provides a voiceprint verification system based on the CANN architecture, including an audio input module 1, a frequency domain feature enhancement module 2, a time domain feature enhancement module 3, a time domain shallow and deep feature combination unit 4, and a time domain and frequency domain feature weighted fusion unit 5. The audio input module 1 includes an audio input unit 6, a resampling unit 7, a frequency domain information normalization unit 8, and an adaptive resampling strategy unit 9. The frequency domain feature enhancement module 2 includes a frequency domain feature extraction unit 10, a frequency domain compression, expansion, and weighted processing unit 11, and a frequency domain weighted clustering and adversarial training unit 12. The time domain feature enhancement module 3 includes a time domain shallow feature extraction unit 13, a time domain deep feature extraction unit 14, and a time domain feature fusion and weighting unit 15. The audio input module 1 is connected to the frequency domain feature enhancement module 2, the frequency domain feature enhancement module 2 is connected to the time domain feature enhancement module 3, the time domain feature enhancement module 3 is connected to the time domain shallow and deep feature combination unit 4, and the time domain shallow and deep feature combination unit 4 is connected to the time domain and frequency domain feature weighted fusion unit 5.

[0036] The audio input unit 6 is used to receive audio signals from different devices, environments, and sampling rates as the initial input data of the system; the resampling unit 7 is used to uniformly process audio signals with different sampling rates; the frequency domain information normalization unit 8 is used to standardize the audio spectrum after resampling to eliminate the spectrum shift caused by different sampling rates; the adaptive resampling strategy unit 9 is used to automatically adjust the accuracy and method of resampling based on factors such as the quality, signal-to-noise ratio, and sampling rate of the input audio.

[0037] The frequency domain feature extraction unit 10 is used to convert the audio signal from the time domain to the frequency domain using a fast Fourier transform to obtain the spectral features of the audio; the frequency domain compression, expansion and weighting processing unit 11 is used to compress and expand the frequency domain image using a 1D convolutional neural network and effectively connect it with the time domain features, while weighting the features of different frequency bands; the frequency domain weighted clustering and adversarial training unit 12 is used to aggregate and weight the features of different frequency bands through a fully connected layer, perform training, and compare with fake audio to identify unnatural changes;

[0038] The temporal shallow feature extraction unit 13 is used to capture local changes in audio signals using a temporal convolutional neural network; the temporal deep feature extraction unit 14 is used to process the long-term dependencies of audio signals using a temporal deep neural network and capture the changing trends of audio signals over a long time scale; the temporal feature fusion and weighting unit 15 is used to perform weighted fusion of temporal shallow and deep features and introduce a regularization strategy.

[0039] The unit 4, which combines shallow and deep temporal features, is used to combine shallow and deep temporal features to form a comprehensive understanding of audio temporal characteristics.

[0040] The time-domain and frequency-domain feature weighted fusion unit 5 is used to perform deep fusion of time-domain and frequency-domain features, taking into account the consistency of audio signals in different dimensions.

[0041] In this embodiment, a multi-level, multi-dimensional feature extraction and fusion mechanism effectively solves the problem of insufficient global spatiotemporal consistency modeling caused by the single dependence on frequency domain or time domain features in existing technologies. The audio input module 1 ensures standardized input of audio with different sampling rates through resampling and frequency domain normalization, providing a stable foundation for subsequent processing. The frequency domain feature enhancement module 2 uses compression and expansion techniques and adversarial training to accurately capture temporal continuity anomalies in frequency domain features. The time domain feature enhancement module 3 comprehensively analyzes the local fluctuations and long-term dependencies of audio through the combination of shallow and deep feature extraction. Finally, the time domain deep and shallow feature combination unit 4 and the time domain and frequency domain feature weighted fusion unit 5 achieve deep integration of multi-dimensional features, significantly improving the system's ability to detect high-fidelity forged audio.

[0042] Furthermore, the audio input unit 6 includes a device audio subunit 16, an ambient audio subunit 17, and a sampling rate audio subunit 18;

[0043] The device audio subunit 16 is used to receive audio signals from the device;

[0044] The environmental audio subunit 17 is used to receive audio signals from the environment;

[0045] The sampling rate audio subunit 18 is used to receive audio signals from the sampling rate.

[0046] In this embodiment, the device audio subunit 16, the ambient audio subunit 17, and the sampling rate audio subunit 18 enable the system to process audio data from different sources more accurately, providing a rich variety of input samples for subsequent feature extraction and model training, which helps to improve the generalization ability and robustness of the model.

[0047] Furthermore, the resampling unit 7 includes an interpolation subunit 19 and a downsampling subunit 20;

[0048] The interpolation subunit 19 is used to improve the resolution of low sampling rate audio;

[0049] The downsampling subunit 20 reduces the resolution of high sampling rate audio.

[0050] In this embodiment, the interpolation subunit 19 improves the resolution of low-sampling-rate audio, ensuring that the temporal details of the audio signal are not lost; the downsampling subunit 20 reduces the resolution of high-sampling-rate audio, reduces the amount of data processing, and maintains the key features of the audio signal.

[0051] Furthermore, the frequency domain weighted clustering and adversarial training unit 12 includes a weighted clustering subunit 21, an adversarial neural network subunit 22, and a fake audio comparison subunit 23;

[0052] The weighted clustering subunit 21 is used to aggregate and weight features of different frequency bands;

[0053] The adversarial neural network subunit 22 is used for training with GAN to learn the frequency domain feature variation pattern of normal audio.

[0054] The fake audio comparison subunit 23 is used to compare the learned pattern with fake audio and identify unnatural changes.

[0055] In this embodiment, the weighted clustering subunit 21 aggregates and weights the features of different frequency bands, highlighting the information of key frequency bands; the adversarial neural network subunit 22 is trained using GAN to learn the frequency domain feature change patterns of normal audio; the fake audio comparison subunit 23 compares the learned patterns with fake audio to identify unnatural changes. This adversarial training mechanism enables the system to more effectively identify frequency domain anomalies in fake audio and improves the robustness of detection.

[0056] Furthermore, the temporal feature fusion and weighting unit 15 includes a multilayer perceptron network subunit 24 and a regularization penalty term subunit 25;

[0057] The multilayer perceptron network subunit 24 is used to weighted fuse shallow and deep features in the temporal domain;

[0058] The regularization penalty term subunit 25 is used to introduce a regularization strategy to prevent the model from overfitting.

[0059] In this embodiment, the multilayer perceptron network subunit 24 weightedly fuses shallow and deep temporal features, comprehensively considering the local fluctuations and long-term dependencies of the audio signal. The regularization penalty term subunit 25 ensures that the model has good generalization ability when processing complex audio signals. The complexity of audio signals makes deep neural networks prone to overlearning from training data, causing the model to be unable to effectively process unknown samples or noisy signals. By adding a regularization penalty term in the feature fusion stage, the model can better balance its dependence on temporal and frequency domain features, avoiding excessive reliance on a single feature, thereby improving the accuracy and robustness of audio fake detection.

[0060] Furthermore, the voiceprint verification system based on the CANN architecture also includes a long-term dependency modeling unit 26, which is connected to the time-domain feature enhancement module 3;

[0061] The long-term dependency modeling unit 26 is used to model the long-term dependency relationship of audio signals through a TDNN network and identify details in fake audio that do not conform to the time sequence.

[0062] In this embodiment, the long-term dependency modeling unit 26 enables the system to capture the long-term trend of audio signals, rather than just short-term fluctuations. Real audio typically exhibits relatively smooth temporal changes, while forged audio often shows discontinuities or traces of human intervention over long periods. Through long-term dependency modeling, this module can identify details in forged audio that do not conform to temporal order, further improving the accuracy of audio recognition.

[0063] Furthermore, the voiceprint verification system based on the CANN architecture also includes a speech recognition continuous feature library 27 and a fake audio output unit 28. The speech recognition continuous feature library 27 is connected to the time-domain and frequency-domain feature weighted fusion unit 5, and the fake audio output unit 28 is connected to the speech recognition continuous feature library 27.

[0064] The continuous feature library 27 for speech recognition is used to extract language structure features using ASR technology from a large-scale real audio corpus to establish a continuous feature library, which serves as the core reference for judging the authenticity of audio.

[0065] The fake audio output unit 28 is used to output a warning or indicator when fake audio is detected.

[0066] In this embodiment, the establishment of this feature library effectively solves the problem of lack of standard reference samples in the prior art, and provides a reliable basis for comparison. The fake audio output unit 28 outputs a warning or mark when it detects fake audio, so as to remind the user to pay attention to the authenticity of the audio in a timely manner. This design makes the system more practical and reliable in the process of audio fake detection, and meets the dual requirements of security and reliability.

[0067] In this invention, to enable consistent analysis of audio signals with different sampling rates, the system employs a resampling unit 7. Specifically, for audio signals with lower sampling rates (e.g., 16kHz), the system increases their sampling rate using the interpolation subunit 19, ensuring that the signal's temporal resolution matches that of higher sampling rate audio signals. For audio signals with higher sampling rates (e.g., 48kHz), the system converts them to the target sampling rate using the downsampling subunit 20. This process ensures that all types of audio signals can be processed under the same standard, thus avoiding feature extraction bias caused by inconsistent sampling rates. To further improve processing performance, the system also employs a frequency domain information normalization unit 8 on top of the resampling process. After the audio signal is resampled, the system standardizes the audio spectrum to ensure that audio signals with different sampling rates exhibit consistent characteristics in the frequency domain. The processing of the frequency domain information normalization unit 8 can eliminate the spectral offset problem caused by different sampling rates, ensuring that the subsequent frequency domain feature extraction module can efficiently and accurately identify subtle differences in fake audio. The processing methods of the frequency domain information normalization unit 8 typically include spectral shifting, scaling, and frequency band segmentation, with the specific selection depending on the characteristics of the audio signal and subsequent analysis requirements. In addition, the adaptive resampling strategy unit 9 is introduced, which automatically adjusts the accuracy and method of resampling based on factors such as the quality, signal-to-noise ratio, and sampling rate of the input audio, ensuring that audio inputs with different sampling rates can achieve the optimal feature extraction effect. This strategy enhances the system's adaptability to diverse audio inputs and improves the processing performance of complex audio signals (such as audio against a noisy background). Through this series of preprocessing methods, the present invention can efficiently process audio signals with various sampling rates, ensuring the consistency of audio feature extraction and providing a stable foundation for subsequent audio authentication. This technical solution makes the system widely applicable in practical applications, capable of processing audio data from different sources and of different qualities, thus improving overall robustness and accuracy.

[0068] In this invention, the main task of the frequency domain feature enhancement module 2 is to extract the spectral features of the audio from the frequency domain perspective and perform collaborative analysis with the time domain features to ensure the consistency of the audio's frequency domain features in the time domain, especially focusing on the continuity and interrelationship between frequency domain and time domain changes. Frequency domain information is crucial for judging the authenticity of audio, especially in fake audio, where frequency domain features are often unnatural or discontinuous. This module enhances the detection capability of these subtle differences through precise feature extraction and adversarial training.

[0069] In the workflow of the frequency domain feature extraction unit 10 described in this invention, the audio signal is first converted from the time domain to the frequency domain using Fast Fourier Transform (FFT) to obtain the audio spectrum. Through FFT, the frequency distribution of the audio at each time point is revealed. The frequency domain image reveals the spectral characteristics of the audio signal and can effectively distinguish the differences in frequency components between real and fake audio. However, due to factors such as sampling rate, environmental noise, and equipment differences, the frequency domain features often have certain fluctuations in the time domain and require further processing to extract useful information.

[0070] In this invention, to ensure the reasonable representation of frequency domain features in the time domain, this module employs the frequency domain compression, expansion, and weighting processing unit 11, using a 1D convolutional neural network (CNN) to compress and expand the frequency domain image. The key to this process is to efficiently compress and expand the frequency domain image through convolutional layers, so that features of different frequency ranges can be reasonably expanded in the time domain, thereby better connecting with time domain features. Through this method, frequency domain features not only retain spectral information but also effectively combine with the temporal continuity and time domain characteristics of audio, enhancing the ability to capture the temporal stability of frequency components.

[0071] In this invention, to extract valuable information from the frequency domain image, the module further utilizes the weighted clustering subunit 21 method. This method aggregates features from different frequency bands through a fully connected layer and weights each frequency band, focusing on the rationality of high-frequency and low-frequency components. During this process, the adversarial neural network subunit 22 is used for training. With the help of a generative adversarial mechanism, the model can learn the changing patterns of frequency domain features in normal audio. By comparing with the fake audio comparison subunit 23, the model can identify unnatural changes in frequency domain features, such as frequency discontinuities and spectral jitter, thereby effectively identifying fake audio.

[0072] The innovation of the frequency domain feature enhancement module 2 in this invention lies in two aspects: First, it employs frequency domain compression and expansion technology, enabling frequency domain features to be better expanded in the time domain and effectively integrated with time domain features. Second, through an adversarial training mechanism, the model can identify unnatural frequency domain changes in fake audio while learning the frequency domain features of normal audio, thereby improving the detection capability of fake audio. Through this technological innovation, this module can more accurately analyze the relationship between frequency domain information and time domain fluctuations, effectively identifying the abnormal features of fake audio.

[0073] In this invention, the temporal shallow feature extraction unit 13 first uses a temporal convolutional neural network (CNN) to perform shallow feature extraction on the audio signal. CNN can capture local changes in the audio signal, especially short-term frequency fluctuations and abrupt changes in the audio signal. The extraction of shallow features mainly focuses on the detailed changes in the audio signal, such as abrupt changes in volume and frequency fluctuations in the audio. These features are particularly important for the identification of fake audio. Through convolution operations, the shallow network can effectively capture local discontinuities and unnatural fluctuations in the temporal domain as the first step in judging fake audio.

[0074] In this invention, the temporal deep feature extraction unit 14 uses a temporal deep neural network (TDNN) to extract deep features from the audio signal. The TDNN network can handle long-term dependencies in the audio signal and capture the changing trends of the audio signal over a long time scale. The goal of deep feature extraction is to establish a time-dependent model of the audio signal and analyze the continuity and regularity of the audio signal in the time dimension. Through training of the deep network, the model can identify temporal discontinuities in fake audio, such as abrupt changes, overlaps, or unnatural timing issues.

[0075] In this invention, the 15-element temporal feature fusion and weighting unit performs deep fusion of the temporal features with the aforementioned frequency domain features after temporal feature extraction. Through a multilayer perceptron (MLP) network, the frequency and temporal features are weighted and fused to enhance the overall temporal consistency recognition capability of the audio signal. In this process, the model not only focuses on local fluctuations in the temporal signal but also comprehensively considers the changing patterns of the frequency domain features in the temporal domain. A regularization penalty term is added during the fusion process to prevent the model from over-relying on any single feature, thus avoiding overfitting. Through weighted fusion, the model can more accurately identify the authenticity of the audio signal, avoiding the misjudgment problems that may arise from single feature extraction methods.

[0076] In this invention, the innovation of the time-domain feature enhancement module 3 lies in its unique multi-level time-domain feature extraction method and deep integration with frequency-domain features, which improves the ability to accurately identify and distinguish audio signals.

[0077] In this invention, the temporal deep and shallow feature combination unit 4 achieves multi-level modeling of audio signals in the temporal domain by combining shallow and deep feature extraction methods. Shallow feature extraction uses a convolutional neural network (CNN) to capture subtle local changes in the audio signal, such as abrupt changes and frequency fluctuations. These shallow features often manifest as unnatural audio breaks or inconsistent signal fluctuations in fake audio, thus serving as a preliminary basis for judging authenticity. Simultaneously, deep feature extraction employs a temporal deep neural network (TDNN) to further capture the long-term dependencies and overall fluctuation trends of the audio signal, ensuring a comprehensive understanding of the temporal continuity and regularity of the audio. Through this combination of shallow and deep feature extraction, the model can better capture various changes in the audio signal, improving its ability to distinguish between real and fake audio.

[0078] In this invention, the time-domain and frequency-domain feature weighted fusion unit 5 not only extracts features in the time domain but also deeply integrates frequency-domain features with time-domain features through weighted fusion. Frequency-domain features provide information on the frequency components of the audio signal, while time-domain features reveal the temporal fluctuations of the audio signal. Since the authenticity of an audio signal is usually reflected in the natural transition and continuity between its frequency and time domains, relying solely on features from one dimension may be limiting. This invention, through weighted fusion of time-domain and frequency-domain features, enables the model to comprehensively consider the consistency of the audio signal across different dimensions, thereby improving the accuracy of identifying fake audio. During feature fusion, by setting different weights, the system can focus on the time-domain fluctuations and frequency-domain changes that are most critical to determining the authenticity of the audio, improving the model's sensitivity to fake audio.

[0079] In this invention, firstly, the audio input unit 6 receives audio signals from different devices, environments, and sampling rates. Then, the interpolation subunit 19 and the downsampling subunit 20 perform unified processing on the audio signals with different sampling rates to ensure consistent temporal resolution. The frequency domain information normalization unit 8 then standardizes the resampled audio spectrum to eliminate spectral shift. Next, the frequency domain feature extraction unit 10 uses a Fast Fourier Transform to convert the audio signal to the frequency domain. The frequency domain compression, expansion, and weighting processing unit 11 uses a 1D convolutional neural network to compress and expand the frequency domain image and interface it with temporal features. Simultaneously, the frequency domain weighted clustering and adversarial training unit 12 aggregates and weights features from different frequency bands through a fully connected layer and uses a GAN for adversarial training to identify unnatural changes in the frequency domain. At the same time, the temporal shallow feature extraction unit 13 uses a temporal convolutional neural network to capture local changes in the audio signal. The deep feature extraction unit 14 uses a temporal deep neural network to process the long-term dependencies of the audio signal. The temporal feature fusion and weighting unit 15 weights and fuses shallow and deep features and introduces a regularization strategy. The temporal shallow and deep feature combination unit 4 further combines the shallow and deep temporal features to form a comprehensive understanding of the audio temporal characteristics. Then, the temporal and frequency domain feature weighted fusion unit 5 performs deep fusion of temporal and frequency domain features, comprehensively considering the consistency of the audio signal in different dimensions. In addition, the long-term dependency modeling unit 26 models the long-term dependencies of the audio signal through a TDNN network to identify details in fake audio that do not conform to the temporal order. Finally, the speech recognition continuous feature library 27 uses a large-scale real audio corpus to extract language structure features through ASR technology to establish a continuous feature library as the core reference for judging the authenticity of audio. The fake audio output unit 28 outputs a warning or label when fake audio is detected, thereby completing the entire audio authentication process.

[0080] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A voiceprint verification system based on CANN architecture, characterized in that, The system includes an audio input module, a frequency domain feature enhancement module, a time domain feature enhancement module, a time domain shallow and deep feature combination unit, and a time domain and frequency domain feature weighted fusion unit. The audio input module includes an audio input unit, a resampling unit, a frequency domain information normalization unit, and an adaptive resampling strategy unit. The frequency domain feature enhancement module includes a frequency domain feature extraction unit, a frequency domain compression, expansion, and weighted processing unit, and a frequency domain weighted clustering and adversarial training unit. The time domain feature enhancement module includes a time domain shallow feature extraction unit, a time domain deep feature extraction unit, and a time domain feature fusion and weighting unit. The audio input module is connected to the frequency domain feature enhancement module, the frequency domain feature enhancement module is connected to the time domain feature enhancement module, the time domain feature enhancement module is connected to the time domain shallow and deep feature combination unit, and the time domain shallow and deep feature combination unit is connected to the time domain and frequency domain feature weighted fusion unit. The audio input unit is used to receive audio signals from different devices, environments, and sampling rates as the initial input data of the system; the resampling unit is used to uniformly process audio signals with different sampling rates; the frequency domain information normalization unit is used to standardize the audio spectrum after resampling to eliminate the spectrum shift caused by different sampling rates; the adaptive resampling strategy unit is used to automatically adjust the accuracy and method of resampling based on factors such as the quality, signal-to-noise ratio, and sampling rate of the input audio. The frequency domain feature extraction unit is used to convert the audio signal from the time domain to the frequency domain using a fast Fourier transform to obtain the spectral features of the audio; the frequency domain compression, expansion and weighting processing unit is used to compress and expand the frequency domain image using a 1D convolutional neural network and effectively connect it with the time domain features, while weighting the features of different frequency bands; the frequency domain weighted clustering and adversarial training unit is used to aggregate and weight the features of different frequency bands through a fully connected layer, perform training, and compare with fake audio to identify unnatural changes; The temporal shallow feature extraction unit is used to capture local changes in audio signals using a temporal convolutional neural network; the temporal deep feature extraction unit is used to process long-term dependencies of audio signals using a temporal deep neural network to capture the changing trends of audio signals over long time scales; the temporal feature fusion and weighting unit is used to perform weighted fusion of temporal shallow and deep features and introduce a regularization strategy. The unit that combines shallow and deep temporal features is used to combine shallow and deep temporal features to form a comprehensive understanding of audio temporal characteristics; The time-domain and frequency-domain feature weighted fusion unit is used to deeply fuse time-domain and frequency-domain features, taking into account the consistency of audio signals in different dimensions.

2. The voiceprint verification system based on CANN architecture as described in claim 1, characterized in that, The audio input unit includes a device audio subunit, an ambient audio subunit, and a sampling rate audio subunit; The device audio subunit is used to receive audio signals from the device; The environmental audio subunit is used to receive audio signals from the environment; The sampling rate audio subunit is used to receive audio signals from the sampling rate.

3. The voiceprint verification system based on the CANN architecture as described in claim 2, characterized in that, The resampling unit includes an interpolation subunit and a downsampling subunit; The interpolation subunit is used to improve the resolution of low sampling rate audio; The downsampling subunit reduces the resolution of high-sampling-rate audio.

4. The voiceprint verification system based on CANN architecture as described in claim 3, characterized in that, The frequency domain weighted clustering and adversarial training unit includes a weighted clustering subunit, an adversarial neural network subunit, and a fake audio comparison subunit; The weighted clustering subunit is used to aggregate and weight features of different frequency bands; The adversarial neural network subunit is used for training with GAN to learn the frequency domain feature variation patterns of normal audio. The fake audio comparison subunit is used to compare the learned pattern with fake audio and identify unnatural changes.

5. The voiceprint verification system based on CANN architecture as described in claim 4, characterized in that, The temporal feature fusion and weighting unit includes a multilayer perceptron network subunit and a regularization penalty term subunit; The multilayer perceptron network subunit is used to weightedly fuse shallow and deep temporal features; The regularization penalty term subunit is used to introduce a regularization strategy to prevent the model from overfitting.

6. The voiceprint verification system based on the CANN architecture as described in claim 5, characterized in that, The CANN-based voiceprint verification system also includes a long-term dependency modeling unit, which is connected to the temporal feature enhancement module. The long-term dependency modeling unit is used to model the long-term dependencies of audio signals through a TDNN network and identify details in fake audio that do not conform to the time sequence.

7. The voiceprint verification system based on CANN architecture as described in claim 6, characterized in that, The voiceprint verification system based on the CANN architecture also includes a speech recognition continuous feature library and a fake audio output unit. The speech recognition continuous feature library is connected to the time-domain and frequency-domain feature weighted fusion unit, and the fake audio output unit is connected to the speech recognition continuous feature library. The continuous feature library for speech recognition is used to extract language structure features using ASR technology from a large-scale real audio corpus to establish a continuous feature library, which serves as the core reference for judging the authenticity of audio. The fake audio output unit is used to output a warning or indicator when fake audio is detected.

Images