Pseudo-speech real-time detection method and system based on multi-channel fusion attention mechanism

Abstract

The present application relates to the technical field of counterfeit speech detection, and particularly relates to a speech real-time detection method and system based on a multi-channel fusion attention mechanism, different lengths of detection windows are sampled for each channel; MFCC features of each frame in the detection window are extracted as input signals; in each detection window, residual network is used to extract input signal features; the input signal features extracted from different local regions in a single detection window are fused to obtain fusion features; an efficient channel attention module is used to generate channel attention weights Mc, the weighted fusion features are input into a full connection layer, and a classification result is obtained through Softmax binary classification, the present application selects part of data packets from the transmitted speech for rapid detection, rather than obtaining all the transmitted speech, and the real-time requirement is met.

Description

Technical Field

[0001] This invention relates to the field of forged speech detection technology, and in particular to a real-time speech detection method and system based on a multi-channel fusion attention mechanism. Background Technology

[0002] In today's information age, information plays a crucial role as a bridge between humanity and the outside world, greatly enhancing our understanding of developments. However, continuous technological advancements have also made the generation of false information relatively easy. Because these false information pieces are difficult to detect and identify, they cause significant disruption to our lives. Therefore, people have higher demands for the security of personal authentication methods. Biometric technology provides a powerful means of identity authentication. This technology analyzes an individual's physiological or behavioral characteristics, such as facial features, palm prints, irises, fingerprints, and voice, to achieve accurate identification of individuals and has been widely applied in various fields such as e-banking, credit cards, and security.

[0003] In recent years, biometric technology has played an increasingly significant role in daily life. As a core means of identity authentication, biometric technology has gradually become a key force in protecting personal privacy and security. Currently available technologies for identity recognition using biometrics include: facial recognition, iris recognition, fingerprint recognition, and speaker verification. Among these, Automatic Speaker Verification (ASV) is a typical biometric system. This system uses specific algorithms to perform pattern recognition and matching on input speech to determine whether the speaker's voice to be verified belongs to a legitimate user.

[0004] With the rapid development of artificial intelligence and machine learning technologies, ASV systems have been widely applied across various industries. These systems rely on user voice input for authentication; therefore, the core of their advancement lies in preventing criminals from using synthesized or forged voices to infringe on personal privacy and steal sensitive information, thereby ensuring the secure use of personal information and protecting legitimate rights. ASV systems provide us with convenient and personalized services. However, this convenience also brings security risks. Hackers use technologies such as recording, speech synthesis, and speech conversion to generate fake voices for voice spoofing attacks. Compared to static physiological characteristics, voice spoofing detection methods are more demanding and challenging. Since voice is transmitted online in real time, the detection algorithm must also be sufficiently efficient. Voice spoofing detection models need to have a sufficiently fast judgment capability. That is, when a voice signal is input, the process of feature extraction, analysis, and judgment should be completed efficiently in the shortest possible time, and feedback should be provided promptly once a malicious spoofing attack is detected. Therefore, how to balance the real-time performance and accuracy of forged voice detection models has become another urgent problem to be solved.

[0005] In the field of forged speech detection, researchers have developed a diverse set of strategies, mainly categorized into feature-based methods and model-based methods. Feature-based methods focus on extracting unique features from speech signals that can distinguish between genuine and fake speech, covering spectral features, Mel-frequency cepstral coefficients (MFCC), and linear predictive coding (LPC). In contrast, model-based methods rely on machine learning and deep learning techniques. For example, convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been applied to extract temporal and frequency features, significantly improving the detection accuracy of the system. Zhang et al. found that combining CNNs and RNNs can improve the robustness of the system. Gomez-Alanis et al. proposed a lightweight convolutional RNN architecture with gated recurrent units (GRUs), and Monteir et al. proposed a residual network with time pooling, enabling the model to handle inputs of different lengths. Res2Net introduces a squeeze-excitation (SE) module to improve the model's ability to detect the most relevant channel information. Tak et al. applied the RawNet2 network to fake speech detection, training a convolutional neural network model on audio data from different sources to detect the presence of fake information in the sound. As an end-to-end network, RawNet2 directly feeds the raw waveform into SincNet, uses ResNet for further discriminative information learning, and then uses GRU to aggregate the speech-level representations.

[0006] Existing technologies suffer from the following technical problems: Current spoofed speech detection strategies rely on offline computer detection after all speech has been transmitted and stored in a buffer. This requires a very large buffer to store and analyze all the data, making real-time detection impossible. Even if the detection system eventually determines that the speech is spoofed, the spoofing has already occurred, failing to truly achieve the goal of spoofed speech detection. Furthermore, the technologies incur significant communication and computational overhead, placing high demands on system resources and resulting in low efficiency, failing to meet real-time requirements. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a real-time pseudo-speech detection method and system based on a multi-channel fusion attention mechanism. This method improves existing pseudo-speech detection technologies, reduces storage and computation costs, and enhances the efficiency and accuracy of the detection algorithm.

[0008] To achieve the above objectives, the technical solution adopted by this invention is as follows: First, a real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism is provided, comprising the following steps:

[0009] S1 sampling and detection window:

[0010] A detection window of different length is sampled for each channel;

[0011] S2 feature extraction:

[0012] Extract the MFCC features of each frame in the detection window as the input signal; within each detection window, use a residual network to extract the features of the input signal.

[0013] S3 fusion features:

[0014] The input signal features extracted from different local regions within a single detection window are fused to obtain the fused features;

[0015] S4 Feature Classification:

[0016] An efficient channel attention module is used to generate channel attention weights Mc. The weighted fused features are then input into a fully connected layer, and binary classification is performed using Softmax to obtain the classification result.

[0017] Preferably, the detection window W is represented as:

[0018] ,

[0019] Where, p k The last received frame is L, and the detection window length is L.

[0020] Preferably, the input signals are concatenated into a matrix in row-major order, with dimensions [D, T], and represented as follows:

[0021]

[0022] Where D is the dimension of the MFCC feature, and T represents the number of time frames.

[0023] Preferably, the input signal feature extraction process is as follows:

[0024] The residual network consists of two convolutional layers forming the convolutional channels in the residual block. A two-dimensional convolutional Conv2D layer is applied to the skip connections. Batch normalization is added after each layer. After k-max pooling of the high-level and low-level features, they are concatenated to generate the residual block output function H(x), which represents the input signal features.

[0025] ,

[0026] Here, F(x) is the high-level feature of the residual network, and c(x) is the low-level feature after a simple dimensionality transformation.

[0027] Preferably, the channel attention weight Mc is calculated as follows:

[0028] Zero-padding unifies the feature vectors from different channels to the same dimension H×W for the fused feature X∈R. C ×H×W Perform global average pooling (AvgPool), then generate channel attention weights Mc using a one-dimensional convolution operation (Conv1D).

[0029] ,

[0030] Where H and W refer to the height and width of the feature map, R refers to the fused feature, C×H×W refers to the dimension of the fused feature, and C refers to the number of channels.

[0031] On the other hand, the present invention provides a real-time pseudo-speech detection system based on a multi-channel fusion attention mechanism, comprising:

[0032] Sampling and detection window module:

[0033] Used to sample detection windows of different lengths for each channel;

[0034] Feature extraction module:

[0035] The MFCC features of each frame in the detection window are used as the input signal; within each detection window, the features of the input signal are extracted using a residual network.

[0036] Feature fusion module:

[0037] This is used to fuse the input signal features extracted from different local regions within a single detection window to obtain fused features;

[0038] Feature classification module:

[0039] The system uses an efficient channel attention module to generate channel attention weights Mc, inputs the weighted fused features into a fully connected layer, and performs binary classification using Softmax to obtain the classification result.

[0040] The above technical solution has the following advantages or beneficial effects:

[0041] To rapidly extract deep features from speech and achieve real-time detection, this invention selects a subset of data from the speech stream instead of detecting the entire stream. Considering that speech spoofing detection systems face various unknown spoofing attacks in real-world applications, using only a fixed-length sliding window for sampling limits the acoustic information obtained from the original speech, making it difficult to detect global spoofing factors. Therefore, this invention uses detection windows of varying lengths to extract MFCC features from the speech. After sampling within the detection window, a residual network is designed to extract the speech features within that window. This invention incorporates a residual attention learning module into the residual network to improve feature granularity and extract more refined feature information. Greater attention is given to channels with more important acoustic information to obtain more comprehensive information.

[0042] The above overview is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments, and features described above, further aspects, embodiments, and features of the invention will become readily apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0043] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in the invention and should not be construed as limiting the scope of the invention.

[0044] Figure 1 A flowchart of a real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism provided by the present invention;

[0045] Figure 2 The improved residual block structure diagram of a real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism provided by the present invention is shown below.

[0046] Figure 3 The present invention provides a structural block diagram of a real-time pseudo-speech detection system based on a multi-channel fusion attention mechanism. Detailed Implementation

[0047] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0048] This invention first provides a real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism, the process of which is as follows: Figure 1As shown, it includes the following steps:

[0049] S1 sampling and detection window:

[0050] The detection window W of different lengths for sampling each channel is represented as:

[0051] ,

[0052] Where, p k The last received frame is L, which is the detection window length. The detection window length L represents the number of frames contained in the detection window, and a different L is selected for each channel.

[0053] S2 feature extraction:

[0054] Extract the MFCC features from each frame in the detection window, use them as input signals, and concatenate them into a matrix in row-major order. The matrix has dimensions [D, T], and is represented as follows:

[0055]

[0056] Where D is the dimension of the MFCC feature, which is 40 dimensions in this embodiment, and T represents the number of time frames.

[0057] Within each detection window, the residual network is used to extract the features of the input signal;

[0058] Compared to directly learning the original feature maps, residual networks have an additional connection method that passes the input to the next layer as the input of the next layer. The mapping method used is constant mapping (identity mapping). The addition of short connections enables residual networks to learn new knowledge based on the original features.

[0059] This invention improves the residual blocks in the residual network, such as... Figure 2As shown, the residual network consists of two convolutional layers forming the convolutional channels in the residual block. After the convolutional layers output high-level features, a two-dimensional convolutional Conv2D layer is applied to the skip connections to facilitate the fusion of low-level features with the high-level features extracted by the convolutional layers, performing dimensionality transformation and simple feature extraction on the original signal. To enhance the robustness of the model and prevent overfitting, batch normalization is added after each layer. This component is widely used in the field of neural networks and has been proven to effectively improve the training and testing performance of the network. Finally, k-max pooling is used to concatenate the high-level and low-level features. Pooling layers are widely used in neural network models. They can reduce the number of redundant neural network parameters while maintaining the overall distribution of features, effectively preventing model overfitting and improving model efficiency. To maintain a rich feature distribution, k-max pooling is used. k-max pooling is a generalization of the max pooling operator, which returns a subsequence of the k maximum values ​​in the input features, rather than a single maximum value. After k-max pooling of the high-level and low-level features, they are concatenated to generate the residual block output function H(x), i.e., the input signal features:

[0060] ,

[0061] Here, F(x) is the high-level feature of the residual network, and c(x) is the low-level feature after a simple dimensionality transformation.

[0062] S3 fusion features:

[0063] The input signal features extracted from different local regions within a single detection window are fused to obtain fused features. To achieve feature fusion and to further extract the correlation between long-distance frames, a fully connected feedforward neural network needs to be added.

[0064] S4 Feature Classification:

[0065] Given that the features of different channels reflect different characteristics of speech and contribute differently to fake speech detection, the process of generating channel attention weights Mc using an efficient channel attention module is as follows:

[0066] Zero-padding unifies the feature vectors from different channels to the same dimension H×W for the fused feature X∈R. C ×H×W Perform global average pooling (AvgPool), then generate channel attention weights Mc using a one-dimensional convolution operation (Conv1D).

[0067] ,

[0068] Where H and W refer to the height and width of the feature map, R refers to the fused feature, C×H×W refers to the dimension of the fused feature, and C refers to the number of channels.

[0069] The weighted fused features are input into the fully connected layer to reduce the dimension to 2, and the classification result is obtained by binary classification through Softmax.

[0070] This invention also provides a real-time pseudo-speech detection system based on a multi-channel fusion attention mechanism, such as... Figure 3 As shown, it includes:

[0071] Sampling and detection window module:

[0072] Used to sample detection windows of different lengths for each channel;

[0073] Feature extraction module:

[0074] The MFCC features of each frame in the detection window are used as the input signal; within each detection window, the features of the input signal are extracted using a residual network.

[0075] Feature fusion module:

[0076] This is used to fuse the input signal features extracted from different local regions within a single detection window to obtain fused features;

[0077] Feature classification module:

[0078] The system uses an efficient channel attention module to generate channel attention weights Mc, inputs the weighted fused features into a fully connected layer, and performs binary classification using Softmax to obtain the classification result.

[0079] In one possible design, the pseudo-speech real-time detection device includes a processor and a memory. The memory stores a program that supports the pseudo-speech real-time detection device in executing the aforementioned pseudo-speech real-time detection method based on a multi-channel fusion attention mechanism. The processor is configured to execute the program stored in the memory.

[0080] This invention also provides a real-time pseudo-speech detection device based on a multi-channel fusion attention mechanism, comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement any of the methods described above.

[0081] This invention also provides a computer-readable storage medium storing a computer program, characterized in that the program, when executed by a processor, implements any of the methods described above.

[0082] The above technical solution has the following advantages or beneficial effects: In order to quickly extract deep features of speech and meet the requirements of real-time detection, this invention selects a portion of the speech data instead of detecting the entire speech stream. Considering that speech spoofing detection systems may face various unknown spoofing attacks in practical applications, using only a fixed-length sliding window for sampling would limit the acoustic information obtained from the original speech, making it difficult to detect global spoofing factors. Therefore, this invention uses detection windows of different lengths to extract MFCC features of the speech. After the detection window sampling is completed, a residual network is designed to extract the features of the speech within that window. This invention adds a residual attention learning module to the residual network to improve feature granularity and extract more refined feature information, giving greater attention to channels with more important acoustic information to obtain more comprehensive information.

[0083] In the description of this specification, references to terms such as "an embodiment," "a preferred embodiment," "an example," "a specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0084] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0085] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0086] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0087] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it includes one or a combination of the steps of the method embodiments. Since the device implementation corresponds to the method implementation, the description of the device implementation is relatively brief, and the relevant description can be found by referring to the description of the method implementation.

[0088] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0089] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in the present invention, and these should all be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism, characterized in that, Includes the following steps: S1 sampling and detection window: A detection window of different length is sampled for each channel; S2 feature extraction: Extract the MFCC features of each frame in the detection window as the input signal; within each detection window, use a residual network to extract the features of the input signal. S3 fusion features: The input signal features extracted from different local regions within a single detection window are fused to obtain the fused features; S4 Feature Classification: An efficient channel attention module is used to generate channel attention weights Mc. The weighted fused features are then input into a fully connected layer, and binary classification is performed using Softmax to obtain the classification result.

2. The real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism as described in claim 1, characterized in that: The detection window W is represented as: ， where p k is the last received frame, and L is the detection window length.

3. The real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism as described in claim 1, characterized in that: The input signals are concatenated into a matrix in row-major order, with dimensions [D, T], and represented as follows: , Where D is the dimension of the MFCC feature, and T represents the number of time frames.

4. The real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism as described in claim 1, characterized in that: The input signal feature extraction process is as follows: The residual network consists of two convolutional layers forming the convolutional channels in the residual block. A two-dimensional convolutional Conv2D layer is applied to the skip connections. Batch normalization is added after each layer. After k-max pooling of the high-level and low-level features, they are concatenated to generate the residual block output function H(x), which represents the input signal features. ， Here, F(x) is the high-level feature of the residual network, and c(x) is the low-level feature after a simple dimensionality transformation.

5. The real-time pseudo-speech detection method based on a multi-channel fusion attention mechanism as described in claim 1, characterized in that: The calculation process for the channel attention weight Mc is as follows: Zero-padding unifies the feature vectors from different channels to the same dimension H×W for the fused feature X∈R. C×H×W Perform global average pooling (AvgPool), then generate channel attention weights Mc using a one-dimensional convolution operation (Conv1D). ， Where H and W refer to the height and width of the feature map, R refers to the fused feature, C×H×W refers to the dimension of the fused feature, and C refers to the number of channels.

6. A real-time pseudo-speech detection system based on a multi-channel fusion attention mechanism, comprising: Sampling and detection window module: Used to sample detection windows of different lengths for each channel; Feature extraction module: The MFCC features of each frame in the detection window are used as the input signal; within each detection window, the features of the input signal are extracted using a residual network. Feature fusion module: This is used to fuse the input signal features extracted from different local regions within a single detection window to obtain fused features; Feature classification module: The system uses an efficient channel attention module to generate channel attention weights Mc, inputs the weighted fused features into a fully connected layer, and performs binary classification using Softmax to obtain the classification result.

Images