Method and system for spoofed speech attribution based on robust adversarial defense boundaries

Abstract

This invention discloses a method and system for attributing forged speech based on robust adversarial defense boundaries. The method includes: collecting speech samples; inputting the speech samples into a trained forged speech attribution model, and outputting the probability that the speech samples belong to various speech forgery methods as the recognition result of the forged speech attribution. The forged speech attribution model includes: a random transformation module for randomly combining and transforming the speech samples to obtain their time-frequency features; a feature extraction module for extracting deep features from the time-frequency features; and a feature classification module for classifying the deep features and outputting the probability that the speech samples belong to various speech forgery methods. This invention, when performing forged speech attribution, can accurately determine the category of speech forgery methods under conditions without adversarial perturbation, and can significantly improve the recognition accuracy of speech forgery method categories under conditions with adversarial perturbation.

Description

Technical Field

[0001] This invention belongs to the field of forged speech attribution technology, specifically relating to a forged speech attribution method and system based on robust adversarial defense boundaries. Background Technology

[0002] The emergence and rapid development of deepfakes have increased content security risks on social media. Defense against voice and image deepfakes has become an important research direction in information security. In recent years, defense against voice deepfakes has evolved from detecting the authenticity of voice information to tracing the origin of voice forgery methods. Forged Speech Attribution (FSA) is one of the key technologies for tracing the origin of voice forgery methods and a crucial step in establishing robust and explainable voice anti-spoofing strategies.

[0003] However, most existing FSA (Fault-Searching Aspect) works employ deep learning models and do not consider the impact of adversarial attacks. In real-world scenarios, attackers may add subtle adversarial perturbations to forged speech, creating adversarial examples to mislead the recognition model's judgment of the forgery method category. This is equivalent to disguising the forgery method to avoid tracing its origin. Such adversarial perturbations are imperceptible to human hearing. While improving the adversarial robustness of deep learning models has always been a key issue in the field of artificial intelligence, preliminary experiments show that existing FSA models have weak adversarial robustness and cannot accurately identify forged speech disguised by perturbations. Therefore, there is an urgent need to conduct research on FSA algorithms specifically for forged speech under adversarial attack conditions to improve the adversarial robustness of FSA models. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and system for attributing spoofed speech based on robust adversarial defense boundaries. When performing spoofed speech attribution, it can accurately identify speech spoofing methods under conditions without adversarial perturbations, and significantly improve the recognition accuracy of speech spoofing methods under conditions with adversarial perturbations.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] Firstly, a method for attributing forged speech is provided, comprising: collecting speech samples; inputting the speech samples into a trained forged speech attribution model, and outputting the probability of the speech samples belonging to various speech forgery methods as the recognition result of the forged speech attribution; wherein, the forged speech attribution model comprises: a random transformation module for randomly combining and transforming the speech samples and obtaining their time-frequency features; a feature extraction module for extracting deep features of the speech samples from the time-frequency features; and a feature classification module for classifying the deep features and outputting the probability of the speech samples belonging to various speech forgery methods.

[0007] Furthermore, the training method for the forged speech attribution model includes: inputting speech samples from the training set into a random transformation module to obtain time-frequency features of the speech samples after random combination transformation; using the time-frequency features as input, extracting deep features of the speech samples using a feature extraction module; outputting the deep features to a feature classification module to output predicted logical values ​​of the speech samples belonging to each category; using a decision boundary distance regularization module, with the predicted logical values ​​as input, constructing an upper bound for the adversarial robustness of the model and calculating the decision boundary distance regularization loss; obtaining the predicted label of the sample based on the predicted logical values ​​and calculating the classification loss; using a linear combination of the decision boundary distance regularization loss and the classification loss as a robust adversarial defense boundary loss function to supervise the training of the feature extraction module and the feature classification module, and obtaining the weight parameters of the feature extraction module and the feature classification module.

[0008] Furthermore, the speech samples are randomly combined and transformed to obtain their time-frequency features, including: randomly generating the number of sampled transformations based on a set set of n transformation types. , n represents the number of defined transformation types; for the number of transformations Q obtained from sampling, for the th transformation... Second transformation The type of randomized transformation The probability of using this transformation and the intensity of the transformation Among them, for the type of transformation Different For different transformation types, a random selection is made from n predefined transformation types; the transformation is then applied. probability Following a Bernoulli distribution with parameter p, i.e., using a transformation The probability is p; using transformation intensity Following a uniform distribution of given parameters, Q transformations are applied sequentially to the input speech samples to obtain samples that have undergone random combination transformations. :

[0009] (1-1)

[0010] in, This represents one arrangement of the Q transformations; each transformation It is a ternary random parameter set, consisting of These three random variables are used to describe it; The parameter is The transformation function; using short-time Fourier transform, Mel filtering, and logarithmic operations, the speech samples after random combination transformation are converted into log-Mel spectra, that is, the time-frequency features of the speech samples after random combination transformation. , represented as ,in, This refers to the random transformation module. This refers to the short-time Fourier transform, Mel filtering, and logarithmic operation.

[0011] Furthermore, the n transformation types are defined, including: reverberation effect transformation, ambient noise superposition transformation, background music superposition transformation, and noisy human voice superposition transformation.

[0012] Furthermore, constructing an upper bound on the adversarial robustness of the model includes: First, defining the stochastic neural network model. The boundary of confrontation and defense is

[0013] (1) Among them, For the first Analogous to the first The adversarial defense boundary of a class is defined by the following formula:

[0014] (2)

[0015] in, It is a voice sample The true label; The model is obtained by sampling using the Monte Carlo algorithm when performing adversarial example attacks. The weights; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The randomized version, ; To use the Monte Carlo algorithm to sample random transformation parameters when performing adversarial example attacks; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The input and calculation formula are:

[0016] (3)

[0017] in, To implement adversarial sample attacks, add to On the counter-disturbance, ; To find the optimal perturbation when dealing with counter-perturbations, the calculation formula is as follows:

[0018] (4)

[0019] in, and These represent the models obtained by sampling using the Monte Carlo algorithm when solving for the optimal perturbation. The weights and transformation parameters with respect to the input, Indicates the number of samples. This represents the loss function used by the Transform Expectation Algorithm (TEA). The TEA is used in conjunction with gradient-based attack methods to solve for the optimal perturbation in a stochastic neural network. When using gradient-based attack methods to generate adversarial examples, in each iteration of updating the adversarial examples, the TEA uses the Monte Carlo algorithm to sample multiple input or target model weights and uses the average of the gradients calculated based on the sampled input or model weights as the final gradient.

[0020] Next, based on the adversarial defense boundary described in formula (2), the model is obtained. About categories The robustness condition for adversarial operations is:

[0021] (5)

[0022] That is, when for Arbitrary Monte Carlo sampling transform version , Both are located on the border When it is greater than 0, the model It can still be correctly identified The label, i.e., the model About categories It is robust against;

[0023] Next, use Taylor's formula at the point Expanding the left side of formula (5), we get:

[0024] (6)

[0025] in, Describes the Euclidean norm operator. yes higher-order infinitesimals, Represents the gradient. yes and The angle between them; Formula (6) is obtained through identity transformation:

[0026] (7)

[0027] in, It is one of the factors affecting the robustness of the model against adversarial attacks; because It is a voice sample The true label, therefore. The first term on the left side of formula (7) is greater than 0, and the numerator of the second term on the left side is... It is an infinitesimal term, much smaller than In the design of adversarial examples, Compared to The value is very small; therefore, the second term does not affect the sign of the whole on the left side. The sign of the left side of formula (7) is determined by the sign of the first term on the left side; therefore, the left side of formula (7) is greater than 0.

[0028] because ,distinguish 0 degrees 90 degrees and 90 degrees Analyze the factors affecting the model's robustness against adversity in three scenarios, including 180-degree angles.

[0029] ① When 90 degrees At 180 degrees, At this point, the left side of formula (7) is greater than 0, because Therefore, the left side of formula (7) is greater than If formula (7) holds, then inequality (5) holds; that is, when 90 degrees At 180 degrees, The model is adversarially robust if it can correctly predict the category of adversarial examples.

[0030] ② When 0 degrees At 90 degrees, Strengthen the conditions of formula (7) to:

[0031] (8)

[0032] If formula (8) holds, then formula (7) holds; formula (8) is obtained through identity transformation:

[0033] (9)

[0034] ③ When When, ,Right now and The directions are consistent, including the following special cases: It is a special type of SNN, namely a fixed neural network; when When the neural network is fixed, the attacker targets the same model weight parameters when solving for the optimal perturbation and when using it to launch the attack; the transformation parameters of the input are equivalent to an identity transformation. In this case, the obtained optimal perturbation lies in the gradient direction of the loss function with respect to the input, i.e. and The directions are consistent. At this point, formula (7) is equivalent to formula (8), and formula (9) is obtained after identity transformation.

[0035] For cases ② and ③, if... If formula (9) is true, then formula (7) is true, and similarly, formula (5) is true; define the left side of formula (9) as ,Right now:

[0036] (10)

[0037] because , In form, it is equivalent to a sample With the Class and First Class decision boundary The distance between; Defined as the model in the first... The upper bound of the robustness against adversarial forces on the dimension; The larger the value, the greater the probability that formula (9) holds true, and the higher the model's probability in the 1st month. The stronger the robustness against adversarial forces on a given dimension, the better; therefore... This is one factor affecting the adversarial robustness of the model; considering the adversarial robustness of the model across various dimensions, the upper bound of the model's adversarial robustness is constructed as follows:

[0038] (11)

[0039] It is the minimum of the upper bound of the model's adversarial robustness across all dimensions; therefore, increasing This will help increase the number of each This increases the likelihood that formula (9) holds true, which in turn helps to enhance the adversarial robustness of the model.

[0040] For a fixed neural network, the condition for adversarial robustness is that equation (8) holds, i.e., the left side of equation (8) is greater than 1; while for an SNN, the condition for adversarial robustness is that equation (7) holds, i.e., the left side of equation (7) is greater than 1. Since the left side of formulas (7) and (8) is the same, and Therefore, under the same conditions, the adversarial robustness condition of SNN is easier to meet, that is, SNN has stronger adversarial robustness.

[0041] Further, the decision boundary distance regularization loss is calculated, including: for the speech samples Its real label is The time-frequency characteristics of the samples after random combination transformation are as follows: ,in, They are Width and height; then use the feature extraction module. extract depth features ,in, These are depth features The number of channels, width, and height; using the feature classification module. Output Predicted logistic value vectors belonging to each category ,in, It is the number of categories. Based on the adversarial robustness upper bound of the constructed model, the decision boundary distance regularization loss is calculated. for:

[0042] (12)

[0043] in, It is a threshold parameter. Describe the gradient; minimize Encourage the model to learn satisfy ;in, The formula is:

[0044] (13)

[0045] in, express Belongs to the The predicted logical value of the class. express Belongs to the The predicted logical value of the class. Represents the logical value vector right The gradient.

[0046] Further, the classification loss is calculated, including using the Softmax cross-entropy loss function or the AM-Softmax loss function.

[0047] Consider a batch of samples The quantity is It takes the logical values ​​and labels output by the feature classification module as input.

[0048] When using the Softmax cross-entropy loss function to calculate the classification loss, the formula is:

[0049] (14)

[0050] in, Indicates the first One sample, express Belongs to the The class's logical value, Indicates sample The corresponding one-hot encoded vector, Represents logarithmic operations;

[0051] When using the AM-Softmax loss function to calculate the classification loss, the formula is (14), the only difference being the method for calculating the logical value; the logical value used by the AM-Softmax loss function is:

[0052] (14-1)

[0053] in, These are samples In the deep features and classification module, the first Class classifier weights Intra-class and inter-class similarity, It is the margin applied to intra-class similarity, where, .

[0054] Furthermore, the robust adversarial defense boundary loss function is:

[0055] (15)

[0056] in, It is a parameter that balances the importance of the corresponding loss.

[0057] Furthermore, the deep features of the speech samples are extracted, including: using time-frequency features as input, the deep features of the speech samples are extracted using ResNet-50; ResNet-50 has four convolutional blocks, and the output of the last convolutional block is used as the deep features.

[0058] Secondly, a fake speech attribution system is provided, comprising: a data acquisition module for acquiring speech samples; and an attribution module for inputting the speech samples into a trained fake speech attribution model and outputting the probability of the speech samples belonging to various speech forgery methods as the recognition result of fake speech attribution; wherein the fake speech attribution model comprises: a random transformation module for performing random combination transformations on the speech samples and obtaining their time-frequency features; a feature extraction module for extracting deep features of the speech samples from the time-frequency features; and a feature classification module for classifying the deep features and outputting the probability of the speech samples belonging to various speech forgery methods.

[0059] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the spoofing speech attribution method as described in the first aspect.

[0060] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0061] (1) This invention inputs the time-frequency features of randomly combined and transformed speech samples into a trained fake speech attribution model and outputs the probability of the time-frequency features belonging to various speech forgery methods as the recognition result of fake speech attribution. When performing fake speech attribution, it can accurately identify the category of speech forgery method under the condition of no adversarial perturbation, and can also greatly improve the recognition accuracy of fake speech disguised by adversarial perturbation.

[0062] (2) To simulate the interference that fake speech may be subjected to in the real world, a random transformation module is designed. By combining multiple random transformations, the randomness of the input samples is enhanced, thereby enhancing the adversarial robustness of the model.

[0063] (3) Define the concept of adversarial defense boundary, analyze the adversarial robustness of the model based on this, and design a decision boundary distance regularization module. By maximizing the difference between the correct class logic value and other class logic values ​​at the same time and minimizing the gradient of the difference with respect to the input, the sensitivity of the model output logic value to adversarial perturbation is reduced, and the adversarial robustness is improved.

[0064] (4) Compared with existing forged speech attribution methods, the method based on robust adversarial defense boundary can not only improve the recognition effect of speech forgery under no-disturbance conditions, but also significantly improve the recognition accuracy of forged speech disguised by adversarial perturbation.

[0065] (5) Compared with existing adversarial defense methods, the method based on robust adversarial defense boundary is the first to analyze the adversarial robustness of the model from the perspective of adversarial defense boundary. Attached Figure Description

[0066] Figure 1This is a main flowchart of a forged speech attribution method based on robust adversarial defense boundaries provided in an embodiment of the present invention;

[0067] Figure 2 This is a visualization of features learned on the evaluation set of the 2023 Chinese Fake Audio Detection (CFAD2023) dataset. (a) is a visualization of the features learned by the WCA method, and (b) is a visualization of the features learned by the method of this invention.

[0068] Figure 3 The curves showing the change in the upper bound of the adversarial robustness of the model learned by the method of this invention and other various methods as the training process progresses when trained on the training set of the CFAD2023 dataset.

[0069] Figure 4 This is a flowchart illustrating the training method of the fake speech attribution model in a fake speech attribution method based on robust adversarial defense boundaries provided in an embodiment of the present invention. Detailed Implementation

[0070] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0071] Example 1

[0072] like Figures 1-4 As shown, a forged speech attribution method based on robust adversarial defense boundaries includes: collecting speech samples; inputting the speech samples into a trained forged speech attribution model, and outputting the probability of the speech sample belonging to various speech forgery methods as the recognition result of the forged speech attribution; wherein, the forged speech attribution model includes: a random transformation module for randomly combining and transforming the speech samples and obtaining their time-frequency features; a feature extraction module for extracting deep features of the speech samples from the time-frequency features; and a feature classification module for classifying the deep features and outputting the probability of the speech sample belonging to various speech forgery methods.

[0073] In this invention, the feature extraction module employs a deep convolutional neural network to extract deep features from speech samples. This includes: using time-frequency features as input, extracting deep features from the speech samples using ResNet-50; ResNet-50 has four convolutional blocks, and the output of the last convolutional block is used as the deep feature; and outputting the obtained deep features of the speech samples to the feature classification module. It is worth noting that other deep learning-based feature extraction networks can also be used as a scheme for extracting deep features from speech samples, such as ResNet-18. The feature classification module performs category determination on the deep features of the speech samples; the feature classification module is a fully connected layer followed by a Softmax function, and the number of output nodes of the fully connected layer is the same as the total number of categories of speech forgery methods.

[0074] For the input speech samples Using a random transformation module, a random combination transformation version of the speech samples is obtained. Short-time Fourier transform, Mel filtering, and logarithmic operations are then used to convert the randomly transformed speech samples into log-Mel spectra as time-frequency features. ,in, They are Width and height.

[0075] Alternatively, ResNet-50 can be used as the feature extraction module. ResNet-50 has four convolutional blocks, using the output of the last convolutional block as the depth feature. ,in, These are depth features The number of channels, width, and height.

[0076] A fully connected layer is used as the feature classification module. This module takes deep features as input and outputs a predicted logistic vector of the features belonging to each category. ,in, It is the number of categories. The index corresponding to the maximum value in the probability vector corresponds to the class label of the sample.

[0077] The present invention trains a fake speech attribution model using the following method.

[0078] (1) Input the speech samples in the training set into the random transformation module to obtain the time-frequency features of the speech samples after random combination transformation.

[0079] The speech samples are randomly combined and transformed to obtain their time-frequency features, including:

[0080] Based on the set n transformation types, the number of sample transformations to be simulated is randomly generated. , where n represents the number of transformation types set;

[0081] For the number of transformations Q obtained from sampling, for the th transformation... Second transformation The type of randomized transformation The probability of using this transformation and the intensity of the transformation ;

[0082] Among them, for the type of transformation Different For different transformation types, a random selection is made from n predefined transformation types; the transformation is then applied. probability Following a Bernoulli distribution with parameter p, i.e., using a transformation The probability is p; using transformation intensity Follows a uniform distribution with given parameters;

[0083] Applying Q transformations sequentially to the input speech sample yields a sample with randomized combinations of transformations. :

[0084] (1-1)

[0085] in, This represents one arrangement of the Q transformations; each transformation It is a ternary random parameter set, consisting of These three random variables are used to describe it; The parameter is Transformation function;

[0086] The speech samples, after random combination transformation, are converted into log-Mel spectra using short-time Fourier transform, Mel filtering, and logarithmic operations. These spectra represent the time-frequency characteristics of the speech samples after random combination transformation. , can be represented as ,in, This refers to the random transformation module. This refers to the short-time Fourier transform, Mel filtering, and logarithmic operation.

[0087] In this invention, the n transformation types include at least: reverberation effect transformation, environmental noise superposition transformation, background music superposition transformation, and noisy human voice superposition transformation.

[0088] Reverberation transformation, simulating the propagation of fake speech within a room, is achieved through convolution of the fake speech with the room's impulse response. For the fake speech sample... A room impulse response audio file was randomly selected from the room impulse response dataset and compared with... Perform convolution operations to obtain the reverberated, fake speech.

[0089] Environmental noise superposition transformation is used to simulate a scenario where forged speech is interfered with by environmental noise. For An ambient noise audio segment was randomly extracted from the NOISE subset of the MUSAN dataset. It was then cropped or zero-padding to a length consistent with... Same, and added according to random signal-to-noise ratio The signal-to-noise ratio is randomly selected between 0dB and 15dB.

[0090] Background music is layered and changed to simulate a scenario where fake speech is interfered with by music. For A segment of audio is randomly selected from the MUSIC subset of the MUSAN dataset and adjusted to match the content by cropping or zero-padding. Same length, then add In this process, the signal-to-noise ratio is randomly selected between 5dB and 15dB.

[0091] The system superimposes and alters noisy human voices to simulate the interference of other speakers' real voices on the fabricated speech. For Randomly selected from the SPEECH subset of the MUSAN dataset One voice sample. exist The signal-to-noise ratio (SNR) of each selected speech sample is randomly chosen between 13 dB and 20 dB.

[0092] (2) Using the time-frequency features as input, the feature extraction module is used to extract the deep features of the speech samples; the deep features are output to the feature classification module to output the predicted logical values ​​of the speech samples belonging to each category.

[0093] (3) Using the decision boundary distance regularization module, with the prediction logic value as input, construct the upper bound of the adversarial robustness of the model and calculate the decision boundary distance regularization loss.

[0094] The upper bound of the adversarial robustness of the model includes:

[0095] First, define the stochastic neural network model. The boundary of confrontation and defense is

[0096] (1) Among them, For the first Analogous to the first The adversarial defense boundary of a class is defined by the following formula:

[0097] (2)

[0098] in, It is a voice sample The true label; The model is obtained by sampling using the Monte Carlo algorithm when performing adversarial example attacks. The weights; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The randomized version, ; To use the Monte Carlo algorithm to sample random transformation parameters when performing adversarial example attacks; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The input and calculation formula are:

[0099] (3)

[0100] in, To implement adversarial sample attacks, add to On the counter-disturbance, ; To find the optimal perturbation when dealing with counter-perturbations, the calculation formula is as follows:

[0101] (4)

[0102] in, and These represent the models obtained by sampling using the Monte Carlo algorithm when solving for the optimal perturbation. The weights and transformation parameters with respect to the input, Indicates the number of samples. This represents the loss function used by the Transform Expectation Algorithm (TEA). The TEA can be combined with gradient-based attack methods to solve for optimal perturbations in stochastic neural networks. Specifically, when using gradient-based attack methods to generate adversarial examples, in each iteration of updating the adversarial examples, the TEA uses the Monte Carlo algorithm to sample multiple input or target model weights and uses the average of the gradients calculated based on the sampled input or model weights as the final gradient.

[0103] Next, based on the adversarial defense boundary described in formula (2), the model is obtained. About categories The robustness condition for adversarial operations is:

[0104] (5)

[0105] That is, when for Arbitrary Monte Carlo sampling transform version , Both are located on the border When it is greater than 0, the model It can still be correctly identified The label, i.e., the model About categories It is robust against resistance.

[0106] Next, use Taylor's formula at the point Expanding the left side of formula (5), we get:

[0107] (6)

[0108] in, Describes the Euclidean norm operator. yes higher-order infinitesimals, Represents the gradient. yes and The angle between them; Formula (6) is obtained through identity transformation:

[0109] (7)

[0110] From formula (7), we can obtain that This is one of the factors affecting the robustness of the model against adversarial attacks. Because... The first term on the left side of formula (7) is greater than 0, and the numerator of the second term on the left side is... It is an infinitesimal term, much smaller than In the design of adversarial examples, Compared to The value is very small. Therefore, the second term does not affect the sign of the total on the left side, and the sign of the left side of formula (7) is determined by the sign of the first term on the left side. Therefore, the left side of formula (7) is greater than 0.

[0111] because ,distinguish 0 degrees 90 degrees and 90 degrees We will analyze the factors affecting the robustness of the model against adversity in three scenarios: 180 degrees and 180 degrees.

[0112] ① When 90 degrees At 180 degrees, At this point, the left side of formula (7) is greater than 0, because Therefore, the left side of formula (7) is greater than If formula (7) holds, then inequality (5) holds; that is, when 90 degrees At 180 degrees, A model is adversarially robust if it can correctly predict the category of adversarial examples.

[0113] ② When 0 degrees At 90 degrees, Strengthen the conditions of formula (7) to:

[0114] (8)

[0115] If formula (8) holds, then formula (7) holds; formula (8) is obtained through identity transformation:

[0116] (9)

[0117] ③ When When, ,Right now and The directions are consistent, including the following special cases: It is a special type of SNN, namely a fixed neural network. When When the neural network is fixed, the attacker targets the same model weight parameters when solving for the optimal perturbation and when using it to launch the attack; the transformation parameters on the input are equivalent to an identity transformation. In this case, the obtained optimal perturbation lies in the gradient direction of the loss function with respect to the input, i.e. and The directions are consistent. At this point, formula (7) is equivalent to formula (8), and after identity transformation, formula (9) is obtained.

[0118] For cases ② and ③, if... If formula (9) holds true, then formula (7) holds true, and similarly, formula (5) can be derived to hold true. Let the left side of formula (9) be defined as... ,Right now:

[0119] (10)

[0120] Observe formula (10), since , In form, it is equivalent to a sample With the Class and First Class decision boundary The distance between them. Defined as the model in the first... Upper Bound on Adversarial Robustness (UBAR). The larger the value, the greater the probability that formula (9) holds true, and the higher the model's probability in the 1st month. The greater the robustness against adversarial forces on a given dimension, the stronger the resistance. Therefore, This is one factor affecting the adversarial robustness of the model. Considering the adversarial robustness of the model across various dimensions, the upper bound of the model's adversarial robustness can be constructed as:

[0121] (11)

[0122] It can be seen that, This represents the minimum upper bound of the model's robustness against adversarial forces across all dimensions. Therefore, increasing... This will help increase the number of each This increases the likelihood that formula (9) holds true, which will be more conducive to enhancing the model's adversarial robustness.

[0123] It is worth noting that for fixed neural networks, the condition for adversarial robustness is that equation (8) holds, i.e., the left side of equation (8) is greater than 1; while for SNNs, the condition for adversarial robustness is that equation (7) holds, i.e., the left side of equation (7) is greater than 1. Since the left-hand sides of formulas (7) and (8) are the same, and Therefore, under the same conditions, the adversarial robustness condition of SNN is easier to meet, that is, SNN has stronger adversarial robustness. Therefore, when performing random combination transformation on speech samples and obtaining their time-frequency features, a random transformation module is designed. By performing random combination transformation on the input speech samples, the network becomes an SNN, which enhances the network's adversarial robustness.

[0124] Calculate the decision boundary distance regularization loss, including:

[0125] Voice samples Its real label is For the samples after random combination transformation, the log-Mel spectrum is extracted as the time-frequency feature. ,in, They are Width and height; then use the feature extraction module. extract depth features ,in, These are depth features The number of channels, width, and height; using the feature classification module. Output Predicted logistic value vectors belonging to each category ,in, It is the number of categories. ; Calculate the regularization loss of the decision boundary distance for:

[0126] (12)

[0127] in, It is a threshold parameter. Describe the gradient; minimize Encourage the model to learn satisfy ;in, The formula is:

[0128] (13)

[0129] in, express Belongs to the The predicted logical value of the class. express Belongs to the The predicted logical value of the class. Represents the logical value vector right The gradient.

[0130] By minimizing the decision boundary distance regularization loss, it is beneficial to maximize the difference between the correct class logistic value and other class logistic values, minimize the gradient of this difference with respect to the input, reduce the sensitivity of the model's output logistic value to adversarial perturbations, and improve adversarial robustness.

[0131] (4) Based on the predicted logical value, obtain the predicted label of the sample and calculate the classification loss.

[0132] Calculate the classification loss, including: using the Softmax cross-entropy loss function or the AM-Softmax loss function to calculate the classification loss;

[0133] Consider a batch of samples The quantity is It takes the logical values ​​and labels output by the feature classification module as input.

[0134] When using the Softmax cross-entropy loss function to calculate the classification loss, the formula is:

[0135] (14)

[0136] in, Indicates the first One sample, express Belongs to the The class's logical value, Indicates sample The corresponding one-hot encoded vector, Represents logarithmic operations;

[0137] When using the AM-Softmax loss function to calculate the classification loss, the formula is (14), the only difference being the method for calculating the logical value; the logical value used by the AM-Softmax loss function is:

[0138] (14-1)

[0139] in, These are samples In the deep features and classification module, the first Class classifier weights Intra-class and inter-class similarity, It is the margin applied to intra-class similarity, where, .

[0140] It should be noted that other loss functions that can be used for sample classification can also be used here to calculate the classification loss.

[0141] By minimizing the classification loss, the model's classification performance on samples can be improved.

[0142] (5) The linear combination of decision boundary distance regularization loss and classification loss is used as the robust adversarial defense boundary loss to supervise the training of the feature extraction module and the feature classification module, and the weight parameters of the feature extraction module and the feature classification module are obtained.

[0143] By linearly combining the two loss functions mentioned above, the final robust adversarial defense boundary loss function is obtained as follows:

[0144] (15)

[0145] in, It is a parameter that balances the importance of the corresponding loss.

[0146] The feature extraction and classification modules are trained using a linear combination of the two aforementioned losses for a total of T training rounds. In each round, the weight parameters of the feature extraction and classification modules are updated using the gradient backpropagation algorithm, and the classification accuracy is calculated on the validation set. After training, the model achieving the highest classification accuracy on the validation set is used as the final model. The performance of the final model is evaluated on the evaluation set, and the predicted labels for the samples are output.

[0147] In this embodiment of the invention, Robust Adversarial Defense Based Speech Forgery Attribution (RADB-FSA) employs two modules: Random Transform (RT) and Decision Boundary Distance Regularization (DBDR) to achieve robust adversarial defense. The RT module simulates the interference that forged speech may encounter in real-world scenarios, performing random combinations of transformations on the input speech during both training and inference, leveraging randomness to improve adversarial robustness. The DBDR module introduces a decision boundary distance regularization loss function, encouraging the model to increase its upper bound on adversarial robustness and reducing the model's sensitivity to adversarial perturbations in class prediction. This method not only achieves high forgery method attribution performance under conditions without adversarial perturbations but also significantly improves the model's forgery method attribution performance for forged speech disguised by adversarial perturbations.

[0148] The following section, using a specific implementation of an embodiment of the present invention on the CFAD2023 dataset, introduces a method flow for achieving speech forgery attribution based on a robust adversarial defense boundary. For example... Figure 4 As shown, this embodiment utilizes the CFAD2023 dataset to perform forged speech attribution based on robust adversarial defense boundaries. This may include the following steps:

[0149] S401 divides the speech data into training samples, validation samples, and test samples.

[0150] S402 uses a random transformation module to perform random combination transformations on speech samples, and uses short-time Fourier transform, Mel filtering and logarithmic operations to convert the randomly transformed speech data into a log-Mel spectrum.

[0151] S403 takes the log-Melp input as input to the ResNet-50 feature extraction module and outputs deep features.

[0152] S404 inputs deep features into the classification module and outputs the category logical value.

[0153] S405 calculates the decision boundary distance regularization loss using the class logistic value, input samples, and sample labels. .

[0154] S406 uses the class logistic value and sample label to calculate the classification loss using Softmax cross-entropy loss. .

[0155] S407 combines the aforementioned two losses to obtain the final robust adversarial defense boundary loss. .

[0156] S408, utilizing Training the feature extraction module and the classification module.

[0157] S409 uses a trained feature extraction module and classification module to identify data that has been camouflaged by adversarial perturbation.

[0158] The 2023 Chinese Fake Audio Detection (CFAD2023) consists of three subsets: a training set, a validation set, and an evaluation set. It includes nine forgery methods, defined as F00 to F08, where genuine speech (represented by F00) is considered a special category. The remaining eight categories are speech synthesis methods based on traditional vocoders or neural vocoders, including commonly used vocoders such as Hifi-GAN, WaveNet, and STRAIGHT. All nine categories are included in the training, validation, and evaluation sets. The training set contains 3,200 speech samples per category. The validation set contains 1,200 speech samples per category. The evaluation set contains 3,500 speech samples per category.

[0159] The Adam optimizer was used, with a batch size of 128, and the training was performed for 50 epochs. The initial learning rate was set to 2e-4, and the weights decayed to 1e-5. The learning rate was reduced to 0.2 times its original value every 10 epochs. The weight parameters in formula (5) are... and threshold parameters Set them to 0.1 and 100 respectively.

[0160] The calculations were performed, and the results were compared with typical current defense methods. Table 1 lists the results.

[0161] Table 1. Comparison of RADB with other methods (%)

[0162]

[0163] In Table 1, the leftmost column represents the attack methods used. Specifically, "clean" indicates no attack. Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Basic Iterative Method (BIM), and Jitter are gradient-based white-box adversarial attack methods. CW is an optimization-based white-box adversarial attack method. EOT-FGSM / BIM / PGD / Jitter is an iterative version of FGSM / BIM / PGD / Jitter. EOT-based adversarial attack methods can enhance the attack performance against stochastic neural networks. A stochastic neural network is a neural network model that introduces randomness into the input or the network itself. This network enhances its adversarial robustness by introducing randomness. Here, EOT stands for Expectation Over Transformation (EOT) algorithm. This algorithm can be combined with gradient-based adversarial example attack methods. Specifically, when using the aforementioned gradient-based attack methods (such as PGD) to generate adversarial examples, in each iteration of updating the adversarial examples, EOT uses the Monte Carlo algorithm to sample multiple input or target model weights and uses the average of the gradients calculated based on the sampled input or model weights as the final gradient. One-Pixel and Square are black-box attack methods. One-Pixel refers to a single-pixel attack method, which generates adversarial examples by adding perturbations to a single data point in the input. Square is a query-based attack method that performs an attack by adding perturbations to a local square region.

[0164] Lines beginning with "Vanilla" indicate defense methods; specifically, "Vanilla" means no defense method is used. "AT" indicates an adversarial training-based defense method, which uses adversarial examples generated by PGD for adversarial training. DIO is the method described in the 2024 article "Robust Neural Networks via Orthogonal Decomposition" published in the journal Pattern Recognition; UniG is the method published in the 2023 ACM Intelligent Systems and Technology. The methods described in the Transactions of journals are: Statistical gradients for improving the robustness of deep networks in the real world; WCA is the method described in the 2021 International Conference on Machine Learning paper "Weight covariance alignment for adversarial robust neural networks"; DTDD is the method described in the 2020 International Conference on Machine Learning paper "Defense through diversified directions"; L2P is the method described in the 2020 International Conference on Computer Vision and Pattern Recognition paper "Learn2Perturb: An end-to-end feature perturbation learning method to improve adversarial robustness"; PNI is the method described in the 2019 International Conference on Computer Vision and Pattern Recognition paper "PNI: Trainable stochasticity to improve the robustness of neural networks against adversarial attacks"; IGR is the method described in the 2018 AAAI conference on Artificial Intelligence paper "Improving the adversarial robustness and interpretability of deep neural networks by regularizing input gradients"; and RADB is the method of this invention.

[0165] As shown in Table 1, compared with all baseline defense methods, the method of this invention achieves higher accuracy on both clean and adversarial samples. Without EOT, under the strongest white-box attack (PGD), the method of this invention improves the model's accuracy from 55.78% to 65.13% compared to the most effective baseline method WCA. Under one-pixel black-box attack conditions, the method of this invention improves the model's accuracy from 88.35% to 93.98% compared to the most effective baseline method WCA. When using EOT, under the strongest white-box attack (EOT-PGD), the accuracy of the method of this invention is still 7.55% higher than WCA. These results indicate that the method of this invention not only improves the model's performance on clean samples but also enhances the attribution performance of the model's forgery methods in the presence of adversarial attacks.

[0166] from Figure 2It can be seen that in the embedding feature representations of samples learned by the state-of-the-art WCA method, some categories are close to each other (e.g., F06 and F08), and even overlap (e.g., F00 and F02, F04 and F07). Compared with WCA's representation, the embedding feature representations learned by RADB perform better: speech samples of different categories can be easily distinguished in the feature representation space. This indicates that the features of the forgery method learned by RADB have higher discriminative power between classes and are more compact within classes. This means that, compared with WCA, to confuse the model's judgment of the feature representations learned by RADB and push the feature representation of one category learned by the model into the cluster of another category, attackers generally need to use a larger perturbation. Therefore, RADB has stronger adversarial robustness.

[0167] from Figure 3 As can be seen, compared with other methods, the RADB method significantly improves the upper bound of the model's adversarial robustness.

[0168] Example 2

[0169] Based on the forged speech attribution method based on robust adversarial defense boundaries described in Embodiment 1, this embodiment provides a forged speech attribution system based on robust adversarial defense boundaries, including:

[0170] The data acquisition module is used to collect voice samples;

[0171] The attribution module is used to input the speech sample into the trained fake speech attribution model and output the probability of the speech sample belonging to various speech forgery methods as the recognition result of fake speech attribution.

[0172] The forged speech attribution model includes:

[0173] The random transformation module is used to perform random combination transformations on speech samples and obtain their time-frequency features;

[0174] The feature extraction module is used to extract deep features of speech samples from the time-frequency features;

[0175] The feature classification module is used to classify the deep features and output the probability that the speech sample belongs to each type of speech forgery method.

[0176] Example 3

[0177] Based on the forged speech attribution method based on robust adversarial defense boundary described in Embodiment 1, this embodiment provides a computer-readable storage medium storing a computer program thereon. The computer program, when executed by a processor, implements the forged speech attribution method based on robust adversarial defense boundary as described in Embodiment 1.

[0178] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0180] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0181] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0182] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method for attributing forged speech, characterized in that, include: Collect voice samples; The speech sample is input into the trained fake speech attribution model, and the output is the probability of the speech sample belonging to various speech forgery methods, which is used as the recognition result of fake speech attribution. The forged speech attribution model includes: The random transformation module is used to perform random combination transformations on speech samples and obtain their time-frequency features; The feature extraction module is used to extract deep features of speech samples from the time-frequency features; The feature classification module is used to classify the deep features and output the probability that the speech sample belongs to various speech forgery methods. This includes performing random combination transformations on speech samples and obtaining their time-frequency features, including: Based on the set n transformation types, the number of simulated sampling transformations is randomly generated. , where n represents the number of transformation types set; For the number of transformations Q obtained from sampling, for the th transformation... Second transformation The type of randomized transformation The probability of using this transformation and the intensity of the transformation ; Among them, for the type of transformation Different For different transformation types, a random selection is made from n predefined transformation types; the transformation is then applied. probability Following a Bernoulli distribution with parameter p, i.e., using a transformation The probability is p; using transformation intensity Follows a uniform distribution with given parameters; Applying Q transformations sequentially to the input speech sample yields a sample with randomized combinations of transformations. : （1-1） in, This represents an arrangement of the Q transformations; each transformation It is a ternary random parameter set, consisting of These three random variables are used to describe it; The parameter is Transformation function; The speech samples, after random combination transformation, are converted into log-Mel spectra using short-time Fourier transform, Mel filtering, and logarithmic operations. These spectra represent the time-frequency characteristics of the speech samples after random combination transformation. , represented as ,in, This refers to the random transformation module. This refers to the short-time Fourier transform, Mel filtering, and logarithmic operation.

2. The method for attributing forged speech according to claim 1, characterized in that, The training method for the forged speech attribution model includes: Input the speech samples in the training set into the random transformation module to obtain the time-frequency features of the speech samples after random combination transformation; Using the time-frequency features as input, the feature extraction module extracts the deep features of the speech samples; the deep features are then output to the feature classification module, which outputs the predicted logical values ​​of the speech samples belonging to each category. Using the decision boundary distance regularization module, with the predicted logic value as input, the upper bound of the model's adversarial robustness is constructed, and the decision boundary distance regularization loss is calculated. Based on the predicted logical value, the predicted label of the sample is obtained, and the classification loss is calculated; The training of the feature extraction module and the feature classification module is supervised by a linear combination of decision boundary distance regularization loss and classification loss as the robust adversarial defense boundary loss function, thus obtaining the weight parameters of the feature extraction module and the feature classification module.

3. The method for attributing forged speech according to claim 2, characterized in that, The n transformation types are defined, including: reverberation effect transformation, ambient noise superposition transformation, background music superposition transformation, and noisy human voice superposition transformation.

4. The method for attributing forged speech according to claim 2, characterized in that, The upper bound of the adversarial robustness of the model includes: First, define the stochastic neural network model. The boundary of the confrontation defense is: (1) in, For the first Analogous to the first The adversarial defense boundary of a class is defined by the following formula: (2) in, It is a voice sample The true label; The model is obtained by sampling using the Monte Carlo algorithm when performing adversarial example attacks. The weights; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The randomized version, ; To use the Monte Carlo algorithm to sample random transformation parameters when performing adversarial example attacks; It was obtained by sampling using the Monte Carlo algorithm when performing adversarial sample attacks. The input and calculation formula are: (3) in, To implement adversarial sample attacks, add to On the counter-disturbance, ; To find the optimal perturbation when dealing with counter-perturbations, the calculation formula is as follows: (4) in, and These represent the models obtained by sampling using the Monte Carlo algorithm when solving for the optimal perturbation. The weights and transformation parameters with respect to the input, Indicates the number of samples. This represents the loss function used by the Transform Expectation Algorithm (TEA). The TEA is used in conjunction with gradient-based attack methods to solve for the optimal perturbation in a stochastic neural network. When using gradient-based attack methods to generate adversarial examples, in each iteration of updating the adversarial examples, the TEA uses the Monte Carlo algorithm to sample multiple input or target model weights and uses the average of the gradients calculated based on the sampled input or model weights as the final gradient. Next, based on the adversarial defense boundary described in formula (2), the model is obtained. About categories The robustness condition for adversarial operations is: (5) That is, when for Arbitrary Monte Carlo sampling transform version , Both are located on the border When it is greater than 0, the model It can still be correctly identified The label, i.e., the model About categories It is robust against; Next, use Taylor's formula at the point Expanding the left side of formula (5), we get: (6) in, Describes the Euclidean norm operator. yes higher-order infinitesimals, Represents the gradient. yes and The angle between them; Formula (6) is obtained through identity transformation: (7) in, It is one of the factors affecting the robustness of the model against adversarial attacks; because It is a voice sample The true label, therefore. The first term on the left side of formula (7) is greater than 0, and the numerator of the second term on the left side is... It is an infinitesimal term, much smaller than In the design of adversarial examples, Compared to The value is very small; therefore, the second term does not affect the sign of the whole on the left side. The sign of the left side of formula (7) is determined by the sign of the first term on the left side; therefore, the left side of formula (7) is greater than 0. because ,distinguish 0 degrees 90 degrees and 90 degrees Analyze the factors affecting the model's robustness against adversity in three scenarios, including 180-degree angles. ① When 90 degrees At 180 degrees, At this point, the left side of formula (7) is greater than 0, because Therefore, the left side of formula (7) is greater than If formula (7) holds, then inequality (5) holds; that is, when 90 degrees At 180 degrees, The model is adversarially robust if it can correctly predict the category of adversarial examples. ② When 0 degrees At 90 degrees, Strengthen the conditions of formula (7) to: （8） If formula (8) holds, then formula (7) holds; formula (8) is obtained through identity transformation: （9） ③ When When, ,Right now and The directions are consistent, including the following special cases: It is a special type of SNN, namely a fixed neural network; when When the neural network is fixed, the attacker targets the same model weight parameters when solving for the optimal perturbation and when using it to launch the attack; the transformation parameters of the input are equivalent to an identity transformation. In this case, the obtained optimal perturbation lies in the gradient direction of the loss function with respect to the input, i.e. and The directions are consistent. At this point, formula (7) is equivalent to formula (8), and formula (9) is obtained after identity transformation. For cases ② and ③, if... If formula (9) is true, then formula (7) is true, and similarly, formula (5) is true; define the left side of formula (9) as ,Right now: （10） because , In form, it is equivalent to a sample With the Class and First Class decision boundary The distance between; Defined as the model in the first... The upper bound of the robustness against adversarial forces on the dimension; The larger the value, the greater the probability that formula (9) holds true, and the higher the model's probability in the 1st month. The stronger the robustness against adversarial forces on a given dimension, the better; therefore... This is one factor affecting the adversarial robustness of the model; considering the adversarial robustness of the model across various dimensions, the upper bound of the model's adversarial robustness is constructed as follows: （11） It is the minimum of the upper bound of the model's adversarial robustness across all dimensions; therefore, increasing This will help increase the number of each This increases the likelihood that formula (9) holds true, which in turn helps to enhance the adversarial robustness of the model. For a fixed neural network, the condition for adversarial robustness is that equation (8) holds, i.e., the left side of equation (8) is greater than 1; while for an SNN, the condition for adversarial robustness is that equation (7) holds, i.e., the left side of equation (7) is greater than 1. Since the left side of formulas (7) and (8) is the same, and Therefore, under the same conditions, the adversarial robustness condition of SNN is easier to meet, that is, SNN has stronger adversarial robustness.

5. The method for attributing forged speech according to claim 2, characterized in that, Calculate the decision boundary distance regularization loss, including: For the speech samples Its real label is The time-frequency characteristics of the samples after random combination transformation are as follows: ,in, They are Width and height; then use the feature extraction module. extract depth features ,in, These are depth features The number of channels, width, and height; using the feature classification module. Output Predicted logistic value vectors belonging to each category ,in, It is the number of categories. Based on the adversarial robustness upper bound of the constructed model, the decision boundary distance regularization loss is calculated. for: （12） in, It is a threshold parameter. Describe the gradient; minimize Encourage the model to learn satisfy ;in, The formula is: （13） in, express Belongs to the The predicted logical value of the class. express Belongs to the The predicted logical value of the class. Represents the logical value vector right The gradient.

6. The forged speech attribution method according to claim 4, characterized in that, Calculate the classification loss, including: using the Softmax cross-entropy loss function or the AM-Softmax loss function to calculate the classification loss; Consider a batch of samples The quantity is It takes the logical values ​​and labels output by the feature classification module as input. When using the Softmax cross-entropy loss function to calculate the classification loss, the formula is: （14） in, Indicates the first One sample, express Belongs to the The class's logical value, Indicates sample The corresponding one-hot encoded vector, Represents logarithmic operations; When using the AM-Softmax loss function to calculate the classification loss, the formula is (14), the only difference being the method for calculating the logical value; the logical value used by the AM-Softmax loss function is: （14-1） in, These are samples In the deep features and classification module, the first Class classifier weights Intra-class and inter-class similarity, It is the margin applied to intra-class similarity, where, .

7. The method for attributing forged speech according to claim 5, characterized in that, The robust adversarial defense boundary loss function is: （15） in, It is a parameter that balances the importance of the corresponding loss.

8. The method for attributing forged speech according to claim 6, characterized in that, Extracting deep features from speech samples, including: Using time-frequency features as input, ResNet-50 is used to extract deep features from speech samples; ResNet-50 has four convolutional blocks, and the output of the last convolutional block is used as the deep features.

9. A spoofed speech attribution system, characterized in that, include: The data acquisition module is used to collect voice samples; The attribution module is used to input the speech sample into the trained fake speech attribution model and output the probability of the speech sample belonging to various speech forgery methods as the recognition result of fake speech attribution. The forged speech attribution model includes: The random transformation module is used to perform random combination transformations on speech samples and obtain their time-frequency features; The feature extraction module is used to extract deep features of speech samples from the time-frequency features; The feature classification module is used to classify the deep features and output the probability that the speech sample belongs to various speech forgery methods. This includes performing random combination transformations on speech samples and obtaining their time-frequency features, including: Based on the set n transformation types, the number of simulated sampling transformations is randomly generated. , where n represents the number of transformation types set; For the number of transformations Q obtained from sampling, for the th transformation... Second transformation The type of randomized transformation The probability of using this transformation and the intensity of the transformation ; Among them, for the type of transformation Different For different transformation types, a random selection is made from n predefined transformation types; the transformation is then applied. probability Following a Bernoulli distribution with parameter p, i.e., using a transformation The probability is p; using transformation intensity Follows a uniform distribution with given parameters; Applying Q transformations sequentially to the input speech sample yields a sample with randomized combinations of transformations. : （1-1） in, This represents an arrangement of the Q transformations; each transformation It is a ternary random parameter set, consisting of These three random variables are used to describe it; The parameter is Transformation function; The speech samples, after random combination transformation, are converted into log-Mel spectra using short-time Fourier transform, Mel filtering, and logarithmic operations. These spectra represent the time-frequency characteristics of the speech samples after random combination transformation. , represented as ,in, This refers to the random transformation module. This refers to the short-time Fourier transform, Mel filtering, and logarithmic operation.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the forged speech attribution method as described in any one of claims 1 to 8.

Images