A method and system for intelligent detection of forged speech based on layered time perception

Abstract

The present invention provides a method and system for intelligent detection of forged voice based on hierarchical time perception in the field of audio processing technology, and the method comprises the following steps: step S1, creating a forged voice detection model based on an input module, a feature extraction module, a multi-time scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module; step S2, obtaining a large amount of historical voice data, constructing a data set after annotating each of the historical voice data, and dividing the data set into a training set and a verification set; step S3, training the forged voice detection model through the training set, and verifying the trained forged voice detection model through the verification set; step S4, obtaining voice data to be detected, and inputting the voice data to be detected into the forged voice detection model that has passed the verification to perform forged voice detection. The advantages of the present invention are: greatly improving the accuracy of forged voice detection.

Description

Technical Field

[0001] The present invention relates to the technical field of audio processing, and in particular to a method and system for intelligently detecting forged speech based on layered time perception. Background Art

[0002] Fake voice detection aims to accurately distinguish between real voice and fake voice generated by various AI technologies. With the development of deep learning technology, especially the application of generative adversarial networks (GANs), recurrent neural networks (RNNs) and variational autoencoders (VAEs), the quality of fake voice is getting higher and higher, which brings great challenges to traditional fake voice detection methods. There are two main methods for detecting fake voice:

[0003] Method 1: Based on prior knowledge, Mel-frequency cepstral coefficients (MFCCs), short-time Fourier transform (STFT) results, spectrograms, etc. are extracted to capture the temporal and frequency characteristics of speech signals based on these features, and forged speech is detected based on the temporal and frequency characteristics. However, the performance of this method will drop significantly in the presence of background noise, because the noise will interfere with the feature extraction process.

[0004] Method 2: Based on deep learning models, especially convolutional neural networks (CNNs) and long short-term memory networks (LSTMs), deep learning models are used to learn complex feature representations from a large number of audio samples to distinguish forged speech. However, existing deep learning models usually perform well on standard datasets and can achieve high detection accuracy, but their generalization ability is often insufficient when faced with new types of forged speech; for example, if the forged speech uses new synthesis technology, existing deep learning models may not be able to effectively identify it.

[0005] Therefore, how to provide a method and system for intelligent detection of forged voice based on layered time perception to improve the accuracy of forged voice detection has become a technical problem that needs to be solved urgently. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for intelligent detection of forged speech based on layered time perception, so as to improve the accuracy of forged speech detection.

[0007] In a first aspect, the present invention provides a method for intelligently detecting forged speech based on hierarchical time perception, comprising the following steps:

[0008] Step S1, creating a forged speech detection model based on an input module, a feature extraction module, a multi-time scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module;

[0009] The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module;

[0010] Step S2, obtaining a large amount of historical voice data, annotating each of the historical voice data to construct a data set, and dividing the data set into a training set and a validation set;

[0011] Step S3, training the forged speech detection model using the training set, and verifying the trained forged speech detection model using the verification set;

[0012] Step S4: Acquire the voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

[0013] Furthermore, in step S1, the input module is used to obtain input voice data and output the voice data to the feature extraction module;

[0014] The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module;

[0015] The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0016] The time-aware attention module is used to suppress the noise of the scale feature to obtain a low-noise feature;

[0017] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module;

[0018] The output module is used to output the probability that the speech data is forged speech according to the fusion features.

[0019] Further, in step S1, the multi-time-scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer;

[0020] The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module;

[0021] Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-feature; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on each of the weight vectors and the corresponding compressed scale sub-feature in the channel dimension to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0022] The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence;

[0023] The second global average pooling layer and the second fully connected layer are used to generate time position weights; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature;

[0024] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through jump connection to obtain fused features;

[0025] The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector.

[0026] Furthermore, the step S2 is specifically as follows:

[0027] A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:2.

[0028] Furthermore, the step S3 is specifically as follows:

[0029] The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process, and the learning rate is dynamically adjusted until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

[0030] In a second aspect, the present invention provides a fake speech intelligent detection system based on layered time perception, comprising the following modules:

[0031] A forged speech detection model creation module, used to create a forged speech detection model based on an input module, a feature extraction module, a multi-time scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module;

[0032] The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module;

[0033] A data set construction module is used to obtain a large amount of historical speech data, construct a data set after annotating each of the historical speech data, and divide the data set into a training set and a verification set;

[0034] a forged speech detection model training module, used to train the forged speech detection model using the training set, and to verify the trained forged speech detection model using the verification set;

[0035] The forged voice detection module is used to obtain voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

[0036] Furthermore, in the forged voice detection model creation module, the input module is used to obtain input voice data and output the voice data to the feature extraction module;

[0037] The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module;

[0038] The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0039] The time-aware attention module is used to suppress the noise of the scale feature to obtain a low-noise feature;

[0040] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module;

[0041] The output module is used to output the probability that the speech data is forged speech according to the fusion features.

[0042] Furthermore, in the forged speech detection model creation module, the multi-time scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer;

[0043] The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module;

[0044] Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-feature; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on each of the weight vectors and the corresponding compressed scale sub-feature in the channel dimension to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0045] The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence;

[0046] The second global average pooling layer and the second fully connected layer are used to generate time position weights; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature;

[0047] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through jump connection to obtain fused features;

[0048] The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector.

[0049] Furthermore, the data set construction module is specifically used for:

[0050] A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:2.

[0051] Furthermore, the forged speech detection model training module is specifically used for:

[0052] The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process, and the learning rate is dynamically adjusted until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

[0053] The advantages of the present invention are:

[0054] A forged speech detection model is created through an input module, a feature extraction module, a multi-time-scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module; then a large amount of historical speech data is obtained for annotation to construct a data set, and the data set is divided into a training set and a verification set, the forged speech detection model is trained through the training set, and the trained forged speech detection model is verified through the verification set; finally, the speech data to be detected is obtained, and the speech data to be detected is input into the forged speech detection model that has passed the verification to perform forged speech detection; that is, forged speech detection is performed through the trained forged speech detection model, and the input module of the forged speech detection model is used to obtain the input speech data and output the speech data to the feature extraction module; the feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain the initial speech features, and output the initial speech features to the multi-time-scale convolution module; the multi-time-scale convolution module The block is used to extract scale sub-features of different time scales from the initial speech features, and weightedly fuse the scale sub-features to obtain scale features, and output the scale features to the time-aware attention module; the time-aware attention module is used to suppress noise on the scale features to obtain low-noise features; the hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features of different levels to obtain fused features, and output the fused features to the output module; the output module is used to output the probability that the speech data is forged speech based on the fused features; that is, the scale sub-features of different time scales are captured by the multi-time-scale convolution module and weightedly fused to obtain scale features, so as to better adapt to different types of forged speech and improve the generalization ability; the time-aware attention module is used to perform adaptive noise suppression to reduce the impact of background noise on detection performance, and the hierarchical feature fusion module is combined to fuse features of different levels to further improve the generalization ability, and ultimately greatly improve the accuracy of forged speech detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.

[0056] Figure 1 It is a flow chart of a method for intelligently detecting forged speech based on layered time perception of the present invention.

[0057] Figure 2 It is a structural schematic diagram of a forged speech intelligent detection system based on layered time perception of the present invention.

[0058] Figure 3 It is a structural schematic diagram of the forged speech detection model of the present invention. DETAILED DESCRIPTION

[0059] The technical solution in the embodiment of the present application has the following general idea: forged speech detection is performed through a trained forged speech detection model, the input module of the forged speech detection model is used to obtain input speech data, and output the speech data to the feature extraction module; the feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module; the multi-time scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion of each scale sub-feature to obtain scale features, and output the scale features to the time-aware attention module; the time-aware attention module is used to perform scale feature fusion on the scale features The method comprises the following steps: first, performing noise suppression to obtain low-noise features; second, performing dimensionality reduction and fusion of scale features and low-noise features at different levels to obtain fused features, and outputting the fused features to the output module; the output module is used to output the probability that the speech data is forged speech based on the fused features; that is, the multi-time-scale convolution module is used to capture the scale sub-features of different time scales and weightedly fuse them to obtain scale features, so as to better adapt to different types of forged speech and improve the generalization ability; the time-aware attention module is used to perform adaptive noise suppression to reduce the impact of background noise on detection performance, and the hierarchical feature fusion module is used to fuse features at different levels to further improve the generalization ability and improve the accuracy of forged speech detection.

[0060] Please refer to Figures 1 to 3 As shown, a preferred embodiment of the forged speech intelligent detection method based on layered time perception of the present invention comprises the following steps:

[0061] Step S1, creating a fake speech detection model (Hierarchical Time-Aware Convolutional Network, HTACN) based on an input module, a feature extraction module, a multi-time scale convolution block (MTSCB), a temporal-aware attention module (TAA), a hierarchical feature fusion module (HFF) and an output module;

[0062] The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module;

[0063] Step S2, obtaining a large amount of historical voice data, annotating each of the historical voice data to construct a data set, and dividing the data set into a training set and a validation set;

[0064] Step S3, training the forged speech detection model using the training set, and verifying the trained forged speech detection model using the verification set;

[0065] Step S4: Acquire the voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

[0066] The present invention introduces multi-scale time-aware convolutional layers and hierarchical feature fusion through a time-aware attention module and a hierarchical feature fusion module, which can capture the characteristics of speech signals at different time scales, thereby better adapting to different types of forged speech; the time-aware attention module utilizes the time-aware attention mechanism to adaptively suppress noise and reduce the impact of background noise on detection performance, thereby significantly improving the detection performance of the forged speech detection model in a noisy environment and ensuring its reliability in complex scenarios, and ultimately effectively solving the shortcomings of the prior art in terms of generalization ability and noise robustness.

[0067] In the step S1, the input module is used to obtain input voice data and output the voice data to the feature extraction module;

[0068] The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module; in specific implementation, the feature extraction module uses a larger convolution kernel (such as 7) and a step size to perform preliminary feature extraction;

[0069] The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0070] The time-aware attention module is used to suppress noise on scale features to obtain low-noise features; that is, the time-aware attention module is used to enhance or suppress output at a specific time position, thereby suppressing noise and optimizing information flow;

[0071] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module, that is, to promote cross-layer fusion of features and be responsible for feature extraction at multiple time scales;

[0072] The output module is used to output the probability that the speech data is forged speech according to the fusion features.

[0073] In step S1, the multi-time-scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer;

[0074] The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module;

[0075] Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features, for example, convolution kernels of sizes 3, 5, and 7 are selected; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers to improve the feature expression capability; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-features, which is used to determine the importance of different scale sub-features and realize adaptive multi-time scale information (scale sub-feature) fusion; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on the channel dimension on each of the weight vectors and the corresponding compressed scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module; the first mapping layer is constructed based on the Sigmoid function;

[0076] The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence;

[0077] The second global average pooling layer and the second fully connected layer are used to generate time position weights, which reflect the influence of each time position on the current task; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature; the second mapping layer is constructed based on the Sigmoid function;

[0078] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through skip connections to obtain fused features, which helps to solve the gradient vanishing problem and enhance the model expression ability;

[0079] The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector; the normalization layer is constructed based on the SoftMax function.

[0080] The step S2 is specifically as follows:

[0081] A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:2.

[0082] The step S3 is specifically as follows:

[0083] The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process. The learning rate is dynamically adjusted to accelerate convergence until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

[0084] A preferred embodiment of the forged speech intelligent detection system based on layered time perception of the present invention includes the following modules:

[0085] A forged speech detection model creation module is used to create a forged speech detection model (Hierarchical Time-Aware Convolutional Network, HTACN) based on an input module, a feature extraction module, a multi-time scale convolution block (MTSCB), a temporal-aware attention module (TAA), a hierarchical feature fusion module (HFF) and an output module;

[0086] The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module;

[0087] A data set construction module is used to obtain a large amount of historical speech data, construct a data set after annotating each of the historical speech data, and divide the data set into a training set and a verification set;

[0088] a forged speech detection model training module, used to train the forged speech detection model using the training set, and to verify the trained forged speech detection model using the verification set;

[0089] The forged voice detection module is used to obtain voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

[0090] The present invention introduces multi-scale time-aware convolutional layers and hierarchical feature fusion through a time-aware attention module and a hierarchical feature fusion module, which can capture the characteristics of speech signals at different time scales, thereby better adapting to different types of forged speech; the time-aware attention module utilizes the time-aware attention mechanism to adaptively suppress noise and reduce the impact of background noise on detection performance, thereby significantly improving the detection performance of the forged speech detection model in a noisy environment and ensuring its reliability in complex scenarios, and ultimately effectively solving the shortcomings of the prior art in terms of generalization ability and noise robustness.

[0091] In the forged speech detection model creation module, the input module is used to obtain input speech data and output the speech data to the feature extraction module;

[0092] The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module; in specific implementation, the feature extraction module uses a larger convolution kernel (such as 7) and a step size to perform preliminary feature extraction;

[0093] The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module;

[0094] The time-aware attention module is used to suppress noise on scale features to obtain low-noise features; that is, the time-aware attention module is used to enhance or suppress output at a specific time position, thereby suppressing noise and optimizing information flow;

[0095] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module, that is, to promote cross-layer fusion of features and be responsible for feature extraction at multiple time scales;

[0096] The output module is used to output the probability that the speech data is forged speech according to the fusion features.

[0097] In the forged speech detection model creation module, the multi-time scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer;

[0098] The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module;

[0099] Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features, for example, convolution kernels of sizes 3, 5, and 7 are selected; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers to improve the feature expression capability; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-features, which is used to determine the importance of different scale sub-features and realize adaptive multi-time scale information (scale sub-feature) fusion; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on the channel dimension on each of the weight vectors and the corresponding compressed scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module; the first mapping layer is constructed based on the Sigmoid function;

[0100] The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence;

[0101] The second global average pooling layer and the second fully connected layer are used to generate time position weights, which reflect the influence of each time position on the current task; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature; the second mapping layer is constructed based on the Sigmoid function;

[0102] The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through skip connections to obtain fused features, which helps to solve the gradient vanishing problem and enhance the model expression ability;

[0103] The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector; the normalization layer is constructed based on the SoftMax function.

[0104] The data set construction module is specifically used for:

[0105] A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:2.

[0106] The forged speech detection model training module is specifically used for:

[0107] The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process. The learning rate is dynamically adjusted to accelerate convergence until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

[0108] In summary, the advantages of the present invention are:

[0109] A forged speech detection model is created through an input module, a feature extraction module, a multi-time-scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module; then a large amount of historical speech data is obtained for annotation to construct a data set, and the data set is divided into a training set and a verification set, the forged speech detection model is trained through the training set, and the trained forged speech detection model is verified through the verification set; finally, the speech data to be detected is obtained, and the speech data to be detected is input into the forged speech detection model that has passed the verification to perform forged speech detection; that is, forged speech detection is performed through the trained forged speech detection model, and the input module of the forged speech detection model is used to obtain the input speech data and output the speech data to the feature extraction module; the feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain the initial speech features, and output the initial speech features to the multi-time-scale convolution module; the multi-time-scale convolution module The block is used to extract scale sub-features of different time scales from the initial speech features, and weightedly fuse the scale sub-features to obtain scale features, and output the scale features to the time-aware attention module; the time-aware attention module is used to suppress noise on the scale features to obtain low-noise features; the hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features of different levels to obtain fused features, and output the fused features to the output module; the output module is used to output the probability that the speech data is forged speech based on the fused features; that is, the scale sub-features of different time scales are captured by the multi-time-scale convolution module and weightedly fused to obtain scale features, so as to better adapt to different types of forged speech and improve the generalization ability; the time-aware attention module is used to perform adaptive noise suppression to reduce the impact of background noise on detection performance, and the hierarchical feature fusion module is combined to fuse features of different levels to further improve the generalization ability, and ultimately greatly improve the accuracy of forged speech detection.

[0110] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for intelligent detection of forged speech based on layered time perception, characterized in that: The steps include: Step S1, creating a forged speech detection model based on an input module, a feature extraction module, a multi-time scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module; The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module; Step S2, obtaining a large amount of historical voice data, annotating each of the historical voice data to construct a data set, and dividing the data set into a training set and a validation set; Step S3, training the forged speech detection model using the training set, and verifying the trained forged speech detection model using the verification set; Step S4: Acquire the voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

2. The method for detecting fake speech based on hierarchical time perception as claimed in claim 1, characterized in that: In the step S1, the input module is used to obtain input voice data and output the voice data to the feature extraction module; The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module; The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module; The time-aware attention module is used to suppress the noise of the scale feature to obtain a low-noise feature; The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module; The output module is used to output the probability that the speech data is forged speech according to the fusion features.

3. The method for detecting fake speech based on hierarchical time perception as claimed in claim 1, characterized in that: In step S1, the multi-time-scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer; The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module; Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-feature; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on each of the weight vectors and the corresponding compressed scale sub-feature in the channel dimension to obtain a scale feature, and output the scale feature to the time-aware attention module; The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence; The second global average pooling layer and the second fully connected layer are used to generate time position weights; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature; The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through jump connection to obtain fused features; The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector.

4. The method for detecting fake speech based on hierarchical time perception as claimed in claim 1, characterized in that: The step S2 is specifically as follows: A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:

2.

5. The method for detecting fake speech based on hierarchical time perception as claimed in claim 1, characterized in that: The step S3 is specifically as follows: The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process, and the learning rate is dynamically adjusted until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

6. A fake speech intelligent detection system based on layered time perception, characterized by: Includes the following modules: A forged speech detection model creation module, used to create a forged speech detection model based on an input module, a feature extraction module, a multi-time scale convolution module, a time-aware attention module, a hierarchical feature fusion module and an output module; The input end of the feature extraction module is connected to the output end of the input module, and the output end is connected to the input end of the multi-time scale convolution module; the output end of the multi-time scale convolution module is connected to the input end of the time-aware attention module; the input end of the hierarchical feature fusion module is connected to the output end of the multi-time scale convolution module and the output end of the time-aware attention module, and the output end is connected to the input end of the output module; A data set construction module is used to obtain a large amount of historical speech data, construct a data set after annotating each of the historical speech data, and divide the data set into a training set and a verification set; a forged speech detection model training module, used to train the forged speech detection model using the training set, and to verify the trained forged speech detection model using the verification set; The forged voice detection module is used to obtain voice data to be detected, and input the voice data to be detected into a verified forged voice detection model to perform forged voice detection.

7. The forged speech intelligent detection system based on layered time perception as claimed in claim 6, characterized in that: In the forged speech detection model creation module, the input module is used to obtain input speech data and output the speech data to the feature extraction module; The feature extraction module is used to perform preliminary feature extraction on the speech data, reduce the spatial dimension of the speech data, obtain initial speech features, and output the initial speech features to the multi-time scale convolution module; The multi-time-scale convolution module is used to extract scale sub-features of different time scales from the initial speech features, and perform weighted fusion on each of the scale sub-features to obtain a scale feature, and output the scale feature to the time-aware attention module; The time-aware attention module is used to suppress the noise of the scale feature to obtain a low-noise feature; The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels to obtain fused features, and output the fused features to the output module; The output module is used to output the probability that the speech data is forged speech according to the fusion features.

8. The forged speech intelligent detection system based on layered time perception as claimed in claim 6, characterized in that: In the forged speech detection model creation module, the multi-time scale convolution module includes a plurality of parallel first convolution layers, a plurality of second convolution layers, a first global average pooling layer, a first fully connected layer and a first mapping layer; The input end of each of the second convolutional layers is respectively connected to the output end of a first convolutional layer, and the output end is connected to the input end of the first global average pooling layer; the input end of the first fully connected layer is connected to the output end of the first global average pooling layer, and the output end is connected to the input end of the first mapping layer; the input end of each of the first convolutional layers is connected to the output end of the feature extraction module; the output end of the first mapping layer is connected to the input end of the time-aware attention module; Each of the first convolutional layers is composed of convolution kernels of different sizes, and is used to extract scale sub-features of different time scales from the initial speech features; the number of output channels of each of the first convolutional layers is the same; each of the second convolutional layers is composed of a one-dimensional convolution, and is used to fuse the scale sub-features output by each of the first convolutional layers; the first global average pooling layer is used to compress the fused scale sub-features; the first fully connected layer is used to generate a weight vector of the compressed scale sub-feature; the first mapping layer is used to map each weight vector to [0,1], and perform element-by-element multiplication operation on each of the weight vectors and the corresponding compressed scale sub-feature in the channel dimension to obtain a scale feature, and output the scale feature to the time-aware attention module; The time-aware attention module includes a second global average pooling layer, a second fully connected layer, and a second mapping layer; the second global average pooling layer, the second fully connected layer, and the second mapping layer are connected in sequence; The second global average pooling layer and the second fully connected layer are used to generate time position weights; the second mapping layer is used to map each time position weight to [0,1], and multiply each time position weight by the scale feature to obtain a low noise feature; The hierarchical feature fusion module is used to reduce the dimension and fuse the scale features and low-noise features at different levels through one-dimensional convolution, and combine the shallow features and deep features in each scale feature and low-noise feature through jump connection to obtain fused features; The output module includes a third fully connected layer and a normalization layer, the output end of the third fully connected layer is connected to the input end of the normalization layer; the third fully connected layer is used to compress the fused features into a feature vector of a preset length; the normalization layer is used to output the probability that the speech data is forged speech based on the feature vector.

9. The forged speech intelligent detection system based on layered time perception as claimed in claim 6, characterized in that: The data set construction module is specifically used for: A large amount of historical voice data is obtained, and a data set is constructed after each of the historical voice data is annotated with fake voice or real voice, and the data set is divided into a training set and a verification set based on a ratio of 8:

2.

10. The forged speech intelligent detection system based on layered time perception as claimed in claim 6, characterized in that: The forged speech detection model training module is specifically used for: The forged speech detection model is trained using the training set. During the training process, half-precision floating-point numbers are used to accelerate the training process, and the learning rate is dynamically adjusted until a preset convergence condition is met. The trained forged speech detection model is then verified using the verification set. If the verification passes, the training is terminated; if the verification fails, the training set is expanded to continue the training.

Images