A forged speech detection method based on PixelHop feature dimensionality reduction

Abstract

The present invention discloses a method for detecting forged speech based on PixelHop feature dimensionality reduction, comprising the following steps: S1, collecting real-time speech to be detected; S2, inputting the real-time speech into a trained forged speech detection model; S3, outputting a detection result through the trained forged speech detection model, thereby completing forged speech detection. The present invention inputs a multi-feature fusion image into a PixelHop unit sequence, connects the neighborhood attributes of a target pixel in the multi-feature fusion image with its adjacent pixels to obtain a neighborhood alliance, aggregates and reduces the dimensionality of the neighborhood alliance using a label-assisted regression method to obtain a feature vector of the PixelHop unit, and performs feature concatenation on the feature vectors of the PixelHop units to obtain a vector for training a classifier. This method enables the establishment of a lightweight, high-performance model, addressing the poor generalization problem of existing forged speech detection models.

Description

Technical Field

[0001] The present invention belongs to the field of speech forgery detection and recognition, and in particular relates to a forged speech detection method based on PixelHop feature dimensionality reduction. Background Art

[0002] The fundamental approach to voice forgery detection is to identify differences between forged and authentic speech features for classification. Conventional voice forgery detection techniques rely on artificial or neural networks to extract features and classify them using classifiers. However, as deepfakes continuously learn from massive amounts of voice data, relying on a single or a few voice features is insufficient for accurate voice forgery detection.

[0003] At present, most research work is mainly focused on the study of forged speech detection technology for ASV systems, and the research on Deepfake speech detection is also attracting widespread attention from scholars. However, existing research has not yet completely solved the problem of forged speech detection. Most existing forged speech detection models are one-to-one models, which cannot be easily and effectively transferred to speech synthesis tasks of other unknown people; in addition, if it is to be implemented in real application scenarios, issues such as conversion rate, model size, and the impact of noise environment need to be considered. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a forged speech detection method based on PixelHop feature dimensionality reduction, which solves the problem of poor generalization of the existing forged speech detection model.

[0005] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is: a method for detecting forged speech based on PixelHop feature dimensionality reduction, comprising the following steps:

[0006] S1. Collect real-time speech to be detected;

[0007] S2. Input the real-time speech into the trained forged speech detection model;

[0008] S3. Output the detection results through the trained forged voice detection model to complete the forged voice detection.

[0009] In S2, the method for obtaining the trained forged speech detection model is specifically as follows:

[0010] S21. Collect existing speech synthesis and conversion data;

[0011] S22, performing image processing on the speech data set to obtain a multi-feature fusion image;

[0012] S23, inputting the multi-feature fusion image into the PixelHop unit sequence to obtain a vector for training the classifier;

[0013] S24, inputting the vector into the classifier for training to obtain a trained classifier;

[0014] S25. Optimize the model according to the trained classifier to obtain a trained forged speech detection model.

[0015] Further: in S21, the speech synthesis and conversion data are obtained from the ASVspoof 2019 database.

[0016] Furthermore, the step S22 includes the following steps:

[0017] S221, generating an audio feature spectrogram based on the speech data set;

[0018] S222. Splice the audio feature spectrograms and grayscale the spliced ​​images to obtain a multi-feature fusion image.

[0019] The beneficial effect of the above further scheme is that the generated audio feature spectrogram can partially reflect the audio features from an image perspective. Different feature spectrograms reflect different focuses on the audio features. In order to consider the speech features more comprehensively, a multi-feature fusion image method is adopted to generate a multi-feature fusion image of speech in a feature splicing manner to improve the accuracy of speech authentication.

[0020] Further: in said S221, the generated audio feature spectrogram includes a Mel-frequency cepstral coefficient spectrogram and a linear-frequency cepstral coefficient spectrogram.

[0021] Further: in S23, the PixelHop unit sequence includes several PixelHop units, and a standard 2×2 to 1×1 maximum pooling unit is inserted between two consecutive PixelHop units.

[0022] Further: the S23 is specifically:

[0023] S231, inputting the multi-feature fusion image into a PixelHop unit sequence, and connecting the neighborhood attributes of the target pixel in the multi-feature fusion image with its adjacent pixels through each PixelHop unit to obtain a neighborhood alliance;

[0024] S232, aggregating and reducing the dimension of the neighborhood alliance through a label-assisted regression method to obtain a feature vector of the PixelHop unit;

[0025] S233. Perform feature concatenation on the feature vectors of the PixelHop unit to obtain a vector for training the classifier.

[0026] The beneficial effect of the above further scheme is: the present invention converts the speech deep fake confrontation problem into an image classification problem. After completing the screening, feature splicing and image preprocessing of the multi-feature fusion image, it migrates it by analogy with the DefakeHop method, inputs the spliced ​​feature spectrogram into the PixelHop unit sequence, connects the neighborhood attributes of the target pixel with its adjacent pixels, and generates a long feature vector.

[0027] Furthermore, in S231, the method for obtaining the neighborhood attribute of the target pixel is specifically as follows:

[0028] The PixelHop unit outputs a neighborhood representation of a certain size centered on the target pixel, and the neighborhood representation is continuously expanded and approximated by subspace to obtain a new representation. The new representations are arranged from small to large to obtain the neighborhood attributes of the target pixel.

[0029] The specific method of obtaining the neighborhood alliance is:

[0030] The neighborhood attributes of the selected pixel are calculated through several PixelHop units, where The PixelHop unit will be the target pixel Neighborhood attributes and their adjacent Pixels are connected to obtain a neighborhood alliance;

[0031] in, , is the total number of PixelHop units.

[0032] Furthermore, in S232, the method of performing aggregation and dimensionality reduction by using the label-assisted regression method is specifically as follows:

[0033] The output of the PixelHop unit is passed through the maximum pooling module to take the maximum, minimum and average values ​​of the response in a small non-overlapping area to obtain the output of the PixelHop unit with reduced dimensions. The output of the PixelHop unit with reduced dimensions is aggregated through the feature aggregation space to obtain the feature vector of the PixelHop unit, where the dimension of the feature vector of the PixelHop unit is M .

[0034] Further: the S233 is specifically:

[0035] Connect the feature vectors of the PixelHop unit to obtain A vector of features, which is used to train the classifier.

[0036] Further: the S25 is specifically:

[0037] The trained classifier is used to update the model parameters to obtain a trained forged speech detection model, wherein the model is specifically a long short-term memory network.

[0038] The beneficial effects of the above further solution are: using long short-term memory networks, having the ability to process sequences, and being able to utilize time series features.

[0039] The beneficial effects of the present invention are:

[0040] (1) The present invention provides a forged speech detection method based on PixelHop feature dimensionality reduction. It combines multiple feature spectrograms to perform feature learning with multi-feature fusion. It uses PixelHop unit sequences to extract feature vectors with effective features from various parts of the image. By updating model parameters and selecting classifiers based on the vectors, it is possible to establish a model with the advantages of lightweight and high performance, thus solving the problem of poor generalization of existing forged speech detection models.

[0041] (2) The forged speech detection model constructed by the present invention has the characteristics of small scale, fast training speed, and small number of training samples. The established model directly targets the audio features of the synthesized / converted speech without paying attention to the identity of the speaker. The robustness and generalization of the model are taken into account to cope with the problem of noise in reality and improve the detection accuracy of unknown speech data. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a forged speech detection method based on PixelHop feature dimensionality reduction according to the present invention. DETAILED DESCRIPTION

[0043] The specific embodiments of the present invention are described below to facilitate understanding of the present invention by those skilled in the art. However, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the appended claims, these changes are obvious, and all inventions and creations utilizing the concepts of the present invention are protected.

[0044] like Figure 1 As shown, in one embodiment of the present invention, a method for detecting forged speech based on PixelHop feature dimensionality reduction includes the following steps:

[0045] S1. Collect real-time speech to be detected;

[0046] S2. Input the real-time speech into the trained forged speech detection model;

[0047] S3. Output the detection results through the trained forged voice detection model to complete the forged voice detection.

[0048] In S2, the method for obtaining the trained forged speech detection model is specifically as follows:

[0049] S21. Collect existing speech synthesis and conversion data;

[0050] S22, performing image processing on the speech data set to obtain a multi-feature fusion image;

[0051] S23, inputting the multi-feature fusion image into the PixelHop unit sequence to obtain a vector for training the classifier;

[0052] S24, inputting the vector into the classifier for training to obtain a trained classifier;

[0053] S25. Optimize the model according to the trained classifier to obtain a trained forged speech detection model.

[0054] In S21, the speech synthesis and conversion data are obtained from the ASVspoof 2019 database.

[0055] In this embodiment, the selected database is ASVspoof 2019. The present invention can also obtain speech synthesis and conversion data from other versions of ASVspoof.

[0056] The ASVspoof 2019 database, published by the Speech Technology Research Centre at the University of Edinburgh, covers speech synthesis, speech conversion, and replay attacks. The database was used in the 3rd Automatic Speaker Verification Spoofing and Countermeasures Challenge, focusing on countermeasures against all three major attack types: text-to-speech (TTS), voice-to-conversion (VC), and replay spoofing. This database brings ASV spoofing closer to the field of automatic speaker verification.

[0057] The S22 includes the following sub-steps:

[0058] S221, generating an audio feature spectrogram based on the speech data set;

[0059] S222. Splice the audio feature spectrograms and grayscale the spliced ​​images to obtain a multi-feature fusion image.

[0060] The generated audio feature spectrogram can partially reflect the audio features from an image perspective. Different feature spectrograms reflect different focuses on the audio features. In order to consider the speech features more comprehensively, a multi-feature fusion image method is adopted to generate a multi-feature fusion image of speech by feature splicing to improve the accuracy of speech authentication.

[0061] In the above-mentioned S221, the generated audio feature spectrogram includes a Mel-frequency cepstral coefficient spectrogram and a linear frequency cepstral coefficient spectrogram.

[0062] In this embodiment, the Mel-frequency cepstral coefficient is a spectral feature that conforms to the human auditory characteristics and has a nonlinear relationship with frequency. It is widely used in speech recognition, audio feature analysis, and other aspects. The linear frequency cepstral coefficient is a cepstral feature based on a triangular filter bank. It is similar to the widely used Mel-frequency cepstral coefficient, but the linear frequency cepstral coefficient may have better resolution in the high-frequency region.

[0063] In S23, the PixelHop unit sequence includes several PixelHop units, and a standard 2×2 to 1×1 maximum pooling unit is inserted between two consecutive PixelHop units.

[0064] Since the neighborhoods of adjacent pixels overlap with each other in each PixelHop unit and there is spatial redundancy in the attributes, a standard 2×2 to 1×1 maximum pooling unit is inserted between two consecutive PixelHop units.

[0065] The S23 is specifically:

[0066] S231, inputting the multi-feature fusion image into a PixelHop unit sequence, and connecting the neighborhood attributes of the target pixel in the multi-feature fusion image with its adjacent pixels through each PixelHop unit to obtain a neighborhood alliance;

[0067] S232, aggregating and reducing the dimension of the neighborhood alliance through a label-assisted regression method to obtain a feature vector of the PixelHop unit;

[0068] S233. Perform feature concatenation on the feature vectors of the PixelHop unit to obtain a vector for training the classifier.

[0069] In this embodiment, the present invention converts the speech deep fake confrontation problem into an image classification problem. After completing the screening of multi-feature fusion images, feature splicing and image preprocessing, the migration is performed by analogy with the DefakeHop method, and the spliced ​​feature spectrogram is input into the PixelHop unit sequence. The neighborhood attributes of the target pixel are connected with its adjacent pixels to generate a long feature vector.

[0070] The above method of generating long feature vectors expects to reduce the dimensionality of connected neighborhood attributes while also expecting the neighborhood attributes of the same object class to reside in a smaller subspace in the high-dimensional attribute space. Therefore, label-assisted regression is used for aggregation and dimensionality reduction.

[0071] In S231, the method for obtaining the neighborhood attribute of the target pixel is specifically as follows:

[0072] The PixelHop unit outputs a neighborhood representation of a certain size centered on the target pixel, and the neighborhood representation is continuously expanded and approximated by subspace to obtain a new representation. The new representations are arranged from small to large to obtain the neighborhood attributes of the target pixel.

[0073] The specific method of obtaining the neighborhood alliance is:

[0074] The neighborhood attributes of the selected pixel are calculated through several PixelHop units, where The PixelHop unit will be the target pixel Neighborhood attributes and their adjacent Pixels are connected to obtain a neighborhood alliance;

[0075] in, 1 to integer, is the total number of PixelHop units.

[0076] Each PixelHop unit generates a neighborhood representation corresponding to its stage index and the input neighborhood size. After the PixelHop units are subjected to the label-assisted regression method, the spectral dimension is changed from down to , while the spatial dimension remains unchanged.

[0077] In S232, the method of performing aggregation and dimensionality reduction by using the label-assisted regression method is specifically as follows:

[0078] The output of the PixelHop unit is passed through the maximum pooling module to take the maximum, minimum and average values ​​of the response in a small non-overlapping area to obtain the output of the PixelHop unit with reduced dimensions. The output of the PixelHop unit with reduced dimensions is aggregated through the feature aggregation space to obtain the feature vector of the PixelHop unit, where the dimension of the feature vector of the PixelHop unit is M .

[0079] The spatial size of feature aggregation is denoted as , Represents an optional hyperparameter.

[0080] The S233 is specifically:

[0081] Connect the feature vectors of the PixelHop unit to obtain A vector of features, which is used to train the classifier.

[0082] The S25 is specifically:

[0083] The trained classifier is used to update the model parameters to obtain a trained forged speech detection model, wherein the model is specifically a long short-term memory network.

[0084] In this embodiment, since the audio feature spectrogram has obvious sequence representation, a long short-term memory network is used, which has the ability to process sequences and can utilize time series features.

[0085] The beneficial effects of the present invention are as follows: a forged speech detection method based on PixelHop feature dimensionality reduction provided by the present invention combines a variety of feature spectrograms to perform feature learning of multi-feature fusion, and uses PixelHop unit sequences to extract feature vectors with effective features from various parts of the image. By updating model parameters and selecting classifiers through vectors, a model with lightweight and high-performance advantages can be established, which solves the problem of poor generalization of existing forged speech detection models.

[0086] The forged speech detection model constructed by the present invention has the characteristics of small scale, fast training speed, and small number of training samples. The established model directly targets the audio features of synthesized / converted speech without paying attention to the identity of the speaker. It also takes into account the robustness and generalization of the model to cope with the problem of noise in reality and improve the detection accuracy of unknown speech data.

[0087] In the description of the present invention, it should be understood that the terms "center", "thickness", "upper", "lower", "horizontal", "top", "bottom", "inner", "outer", "radial", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only and cannot be understood as indicating or implying the relative importance or the number of technical features implicitly specified. Therefore, the features defined by "first", "second", and "third" may explicitly or implicitly include one or more of such features.

Claims

1. A forged speech detection method based on PixelHop feature dimensionality reduction, characterized in that: The following steps are involved: S1. Collect real-time speech to be detected; S2. Input the real-time speech into the trained forged speech detection model; S3. Output the detection results through the trained forged speech detection model to complete the forged speech detection; In S2, the method for obtaining the trained forged speech detection model is specifically as follows: S21. Collect existing speech synthesis and conversion data; S22, performing image processing on the speech data set to obtain a multi-feature fusion image; S23, inputting the multi-feature fusion image into the PixelHop unit sequence to obtain a vector for training the classifier; S24, inputting the vector into the classifier for training to obtain a trained classifier; S25. Optimize the model according to the trained classifier to obtain a trained forged speech detection model.

2. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 1, characterized in that: In S21, the speech synthesis and conversion data are obtained from the ASVspoof 2019 database.

3. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 1, characterized in that: The S22 includes the following sub-steps: S221, generating an audio feature spectrogram based on the speech data set; S222. Splice the audio feature spectrograms and grayscale the spliced ​​images to obtain a multi-feature fusion image.

4. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 3, characterized in that: In the above-mentioned S221, the generated audio feature spectrogram includes a Mel-frequency cepstral coefficient spectrogram and a linear frequency cepstral coefficient spectrogram.

5. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 1, characterized in that: In S23, the PixelHop unit sequence includes several PixelHop units, and a standard 2×2 to 1×1 maximum pooling unit is inserted between two consecutive PixelHop units.

6. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 5, characterized in that: The S23 is specifically: S231, inputting the multi-feature fusion image into a PixelHop unit sequence, and connecting the neighborhood attributes of the target pixel in the multi-feature fusion image with its adjacent pixels through each PixelHop unit to obtain a neighborhood alliance; S232, aggregating and reducing the dimension of the neighborhood alliance through a label-assisted regression method to obtain a feature vector of the PixelHop unit; S233. Perform feature concatenation on the feature vectors of the PixelHop unit to obtain a vector for training the classifier.

7. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 6, characterized in that: In S231, the method for obtaining the neighborhood attribute of the target pixel is specifically as follows: The PixelHop unit outputs a neighborhood representation of a certain size centered on the target pixel, and the neighborhood representation is continuously expanded and approximated by subspace to obtain a new representation. The new representations are arranged from small to large to obtain the neighborhood attributes of the target pixel. The specific method of obtaining the neighborhood alliance is: The neighborhood attributes of the selected pixel are calculated through several PixelHop units, where The PixelHop unit will be the target pixel Neighborhood attributes and their adjacent Pixels are connected to obtain a neighborhood alliance; in, , is the total number of PixelHop units.

8. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 7, characterized in that: In S232, the method of performing aggregation and dimensionality reduction by using the label-assisted regression method is specifically as follows: The output of the PixelHop unit is passed through the maximum pooling module to take the maximum, minimum and average values ​​of the response in a small non-overlapping area to obtain the output of the PixelHop unit with reduced dimensions. The output of the PixelHop unit with reduced dimensions is aggregated through the feature aggregation space to obtain the feature vector of the PixelHop unit, where the dimension of the feature vector of the PixelHop unit is M .

9. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 8, characterized in that: The S233 is specifically: Connect the feature vectors of the PixelHop unit to obtain A vector of features, which is used to train the classifier.

10. The forged speech detection method based on PixelHop feature dimensionality reduction according to claim 1, characterized in that: The S25 is specifically: The trained classifier is used to update the model parameters to obtain a trained forged speech detection model, wherein the model is specifically a long short-term memory network.

Images