An adversarial sample generation method and system for audio data

Abstract

The application discloses an audio data adversarial sample generation method and system, and is characterized in that the method comprises the following steps: step 1, defining an audio data adversarial sample generation parameter, and calling an adversarial sample generation algorithm based on the adversarial sample generation parameter; step 2, inputting the audio data and an initial adversarial disturbance into the adversarial sample generation algorithm to solve an adversarial sample; wherein the adversarial sample generation algorithm realizes iterative optimization of the adversarial disturbance based on a model classification loss, a multi-scale spectrum loss and a weighted sum of disturbance norms; and step 3, constructing an adversarial sample set according to the solved adversarial sample, and realizing interface display and man-machine interaction of the adversarial sample set by using an interactive interface. The application is effective and reliable, can obtain a local optimum at an arbitrary position, makes the adversarial sample primarily focus on a success rate in an attack process, and makes the adversarial sample comprehensive, accurate and effective.

Description

Technical Field

[0001] This invention relates to the field of audio data processing, and more specifically, to a method and system for generating adversarial examples from audio data. Background Technology

[0002] Currently, deep learning technology performs exceptionally well in audio classification tasks. However, adding adversarial perturbations to the original audio significantly reduces the prediction accuracy of deep learning audio classification models. Adversarial perturbations play a crucial role in further researching the robustness and security of audio classification models. Several methods for generating audio adversarial perturbations exist, but there is still room for improvement in terms of attack success rate, adversarial transferability, and attack efficiency.

[0003] In the Chinese patent "CN202010254686.2 - A General Perturbation Generation Method Based on Generative Adversarial Networks", the objective function is calculated only by constraining the perturbation norm term, without considering the model loss term and content loss term, resulting in low accuracy of the adversarial example discrimination network. Furthermore, the adversarial example generation method combines perturbations with arbitrary image datasets, making the generation method relatively simple.

[0004] In Chinese patents “CN202210485160.4 - A method and system for generating attack data for an intrusion detection system based on generative adversarial networks” and “CN202011364634.7 - A text generation method based on generative adversarial networks”, the generation methods based on generative adversarial networks are not general-purpose methods, but only applicable to intrusion detection systems and text generation systems. Similarly, the objective function calculation methods in these patents are overly simplistic, resulting in low accuracy in identifying adversarial network samples.

[0005] To address the aforementioned issues, a new method and system for generating adversarial examples from audio data is urgently needed. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a method and system for generating adversarial examples for audio data. The method calculates the loss function from multiple aspects, including model classification loss, time-frequency multi-dimensional loss, and perturbation loss. A greedy algorithm is used to gradually increase the weight of the model classification loss in the loss function, thereby obtaining the local optimal perturbation of the audio data.

[0007] The present invention adopts the following technical solution.

[0008] The first aspect of this invention relates to a method for generating adversarial examples from audio data. The method includes the following steps: Step 1, defining adversarial example generation parameters for the audio data, and calling an adversarial example generation algorithm based on the adversarial example generation parameters; Step 2, inputting the audio data and initial adversarial perturbation into the adversarial example generation algorithm to solve for adversarial examples; wherein, the adversarial example generation algorithm iteratively optimizes the adversarial perturbation based on a weighted sum of model classification loss, multi-scale spectral loss, and perturbation norm; Step 3, constructing an adversarial example set based on the solved adversarial examples, and using an interactive interface to display and interact with the adversarial example set.

[0009] Preferably, the adversarial sample generation parameters include at least: the name of the adversarial sample generation algorithm, the target attack category, the target attack method, the number of iterations, the batch sample size, the target label, and the normalization constraint size.

[0010] Preferably, the adversarial example generation algorithm is a deep learning algorithm based on iterative optimization of the loss function to generate adversarial perturbations; wherein the loss function is a weighted sum of the model classification loss, multi-scale spectral loss, and perturbation norm; and the weights in the weighted sum are iteratively increased based on a greedy algorithm to solve for all adversarial examples.

[0011] Preferably, the initial adversarial disturbance is a random noise signal with the same duration as the audio data.

[0012] Preferably, the method for solving adversarial examples further includes: Step 2.1, for each audio data in the audio training set input to the adversarial example generation algorithm, adding an initial adversarial perturbation to generate an initial adversarial example; Step 2.2, inputting the initial adversarial example into the target deep learning model to obtain the target output, and comparing the target output with the predefined target label in Step 1; Step 2.3, if the target output matches the predefined target label, retaining the initial adversarial example and adding it to the adversarial example set; Step 2.4, if the target output does not match the predefined target label, optimizing the initial adversarial perturbation into a new round of adversarial perturbation, and re-executing Steps 2.2 to 2.4 until an adversarial example that meets the requirements of the adversarial example set is generated for each audio data in the audio training set.

[0013] Preferably, optimizing the initial adversarial perturbation into a new round of adversarial perturbation further includes: calculating the loss function of the initial adversarial perturbation, and using the ADAM optimizer to calculate the gradient of the loss function, and updating the initial adversarial perturbation into a new round of adversarial perturbation according to the value of the gradient.

[0014] Preferably, the loss function is:

[0015] L = L dist +L spec +α·Lnet

[0016] In the formula, L dist Let L be the perturbation norm, and L be the perturbation norm. dist =||δ i ||2;

[0017] L spec For multi-scale spectral loss, and has

[0018] L net The model classification loss is L. net =CrossEntropyLoss(f(x′) i ), t);

[0019] α is the weight defined based on a greedy algorithm;

[0020] Where, δ i For the i-th audio data x i Corresponding counter-perturbations,

[0021] For the i-th audio data x i The time-domain sample obtained after time-frequency transformation, where j is the dimension number after time-frequency transformation.

[0022] The i-th counter-perturbation x′ i The time-domain sample obtained after time-frequency transformation

[0023] CrossEntropyLoss(f(x′ i ),t) is f(x′ i The cross-entropy loss f(x′) under t different target label categories i To counteract the disturbance x′ i The target output of the target deep learning model.

[0024] Preferably, the greedy algorithm increments the discrete values ​​of the weight α; when the initial value of α cannot satisfy a certain x′ i After the target output matches the predefined target label, the greedy algorithm assigns α to the next discrete value, until x′. i The target output must match the predefined target label.

[0025] Preferably, the adversarial sample generation parameters for each definition are recorded, as well as the adversarial sample set obtained based on each definition; the adversarial sample set corresponding to the current definition and the perturbation report generated based on the adversarial sample set are displayed in the form of a front-end page; the adversarial sample set corresponding to the historical definition and the perturbation report generated based on the adversarial sample set are stored, and the storage is queried.

[0026] A second aspect of this invention relates to an adversarial example generation system for audio data. The system includes a server and a client. The server and client are connected in communication, and the server includes a parameter customization module, an audio adversarial example set generation module, a dataset display module, and a log module. The parameter customization module is used to obtain adversarial example generation parameters for the audio data defined by the client and to call an adversarial example generation algorithm based on these parameters. The audio adversarial example set generation module is used to input the audio data and initial adversarial perturbations into the adversarial example generation algorithm to solve for adversarial examples. The dataset display module and the log module are used to construct an adversarial example set based on the solved adversarial examples and push the interface display and human-computer interaction results of the adversarial example set to the client.

[0027] The beneficial effects of this invention are that, compared with the prior art, the adversarial example generation method and system for audio data in this invention calculates the loss function from multiple aspects such as model classification loss, time-frequency multi-dimensional loss, and perturbation loss, and uses a greedy algorithm to gradually increase the weight of the model classification loss in the loss function, thereby obtaining the local optimal perturbation of the audio data. This invention is effective and reliable, and can obtain local optima at any position, making the adversarial examples primarily focused on the success rate during the attack process, thus making the adversarial examples comprehensive, accurate, and effective.

[0028] The beneficial effects of this application also include:

[0029] 1. We generate audio adversarial perturbations based on iterative optimization. An optimizer is used to optimize the loss function. Since the loss function measures the effectiveness of the adversarial perturbation, we believe that the optimal point of the loss function corresponds to the optimal solution of the adversarial perturbation. Furthermore, because the method does not necessarily rely on a specific attack algorithm, but rather expands the dataset by adding various natural noises, it improves the model's adaptability to more general situations.

[0030] 2. Based on multi-scale spectral loss, this invention designs a novel loss function that combines the characteristics of the model and the perturbation itself to comprehensively measure the attack effectiveness against the perturbation. The loss function consists of a perturbation norm loss term, a multi-scale spectral loss term, and a model classification loss term. By adjusting the weights of each loss term using a greedy algorithm, the method prioritizes increasing the attack success rate rather than reducing the noise level of the perturbation at the beginning of the iteration process.

[0031] 3. This invention constructs a novel method for generating adversarial perturbations and corresponding adversarial examples in audio. Based on a greedy iterative approach, this method calculates the loss function corresponding to the optimizer through optimization, obtaining the optimal solution for the adversarial perturbation corresponding to the optimal point of the loss function. Simultaneously, it maintains the similarity in content between the adversarial example and the original audio, and is not easily detected by the human ear.

[0032] This strengthens the ability to reasonably predict perturbation attacks and improves the ability of deep learning models to distinguish samples. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of an adversarial example generation method for audio data according to the present invention;

[0034] Figure 2 This is a flowchart illustrating the adversarial perturbation process in an adversarial sample generation method for audio data according to the present invention. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this invention are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments not described in this invention obtained by those skilled in the art based on the embodiments described in this invention without creative effort should fall within the protection scope of this invention.

[0036] The first aspect of this invention relates to a method for generating adversarial examples from audio data. The method includes the following steps: Step 1, defining adversarial example generation parameters for the audio data, and calling an adversarial example generation algorithm based on the adversarial example generation parameters; Step 2, inputting the audio data and initial adversarial perturbation into the adversarial example generation algorithm to solve for adversarial examples; wherein, the adversarial example generation algorithm iteratively optimizes the adversarial perturbation based on a weighted sum of model classification loss, multi-scale spectral loss, and perturbation norm; Step 3, constructing an adversarial example set based on the solved adversarial examples, and using an interactive interface to display and interact with the adversarial example set.

[0037] The adversarial perturbation generation method based on iterative optimization is a research approach for creating adversarial perturbations. This method uses an optimizer to optimize a loss function. Since the loss function measures the effectiveness of the adversarial perturbation, we believe that the optimal point of the loss function corresponds to the optimal solution of the adversarial perturbation. After the calculation is completed, the generated adversarial perturbation can be added to the original audio to obtain the corresponding adversarial sample.

[0038] Preferably, the adversarial sample generation parameters include at least: the name of the adversarial sample generation algorithm, the target attack category, the target attack method, the number of iterations, the batch sample size, the target label, and the normalization constraint size.

[0039] In this invention, it is necessary to predefine the adversarial example generation parameters manually. These generation parameters need to ensure that the invention can achieve iterative calculation and acquisition of adversarial examples in a specific manner.

[0040] Specifically, this invention may include a variety of different adversarial example generation algorithms. Typically, these algorithms are intelligent algorithms with multiple iterations, which use computer calculations to iteratively change the perturbation values, enabling the method to obtain the optimal perturbation values ​​and corresponding adversarial examples. Considering this, the design parameters of this invention include the names of the selectable algorithms.

[0041] Furthermore, adversarial attacks refer to the process of causing a classifier to misclassify a sample by adding a small perturbation to the input. Therefore, the target attacks in this invention can be categorized into untargeted attacks and targeted attacks based on the purpose of the attack. Specifically, an untargeted attack requires the attacker to cause the target model to misclassify the sample, without specifying which class it should be classified into. A targeted attack requires the attacker to specify a class, causing the target model not only to misclassify the sample but also to classify it into the specified class. In terms of difficulty, targeted attacks are more difficult to implement than untargeted attacks.

[0042] Furthermore, the attack methods can be categorized in various ways. For example, based on the attack environment, they can be divided into black-box attacks, white-box attacks, or gray-box attacks. Based on the intensity of the perturbation, they can be divided into infinity-norm attacks, norm 2 attacks, and norm 0 attacks. Based on the attack implementation, they can be categorized into gradient-based attacks, optimization-based attacks, decision-level-based attacks, or others.

[0043] Considering that although the present invention may include a variety of different algorithms, all algorithms have the characteristic of step-by-step iteration, the number of iterations can be uniformly designed as a parameter to end the iteration process of the algorithm.

[0044] In addition, batch size refers to the number of samples used in each deep learning computation, and target label refers to the label for each type after the deep learning classifies the input samples. Normalization constraint can be the coefficient used to limit the range of values ​​of the loss function when it needs to undergo a standardization or normalization process.

[0045] Through the above design, even if different algorithms are used, the parameters can still be set uniformly, thereby ensuring that the generator of interference samples can understand the meaning of the algorithm and make reasonable definitions of the algorithm parameters during human-computer interaction.

[0046] Furthermore, this invention can also define hyperparameters of other algorithms, such as the learning rate parameter and the post-processing function clip for perturbation, depending on the specific algorithm.

[0047] Preferably, the adversarial example generation algorithm is a deep learning algorithm based on iterative optimization of the loss function to generate adversarial perturbations; wherein the loss function is a weighted sum of the model classification loss, multi-scale spectral loss, and perturbation norm; and the weights in the weighted sum are iteratively increased based on a greedy algorithm to solve for all adversarial examples.

[0048] In this invention, the generation of adversarial examples also requires the use of the original deep learning algorithm to solve for the target output and estimate the loss function.

[0049] In order to reasonably define the perturbation, this invention comprehensively selects various loss terms obtained by different calculation methods to comprehensively consider the losses caused by deep learning algorithms from various perspectives. In addition, considering the iterative process of the algorithm, the method can also gradually change the weights of different aspects of the loss function, thereby gradually expanding the search range for local optima.

[0050] Preferably, the initial adversarial disturbance is a random noise signal with the same duration as the audio data.

[0051] In this invention, during the iterative optimization process, for each input original audio sample, a random noise of the same length is first initialized to obtain the final adversarial perturbation. Then, the noise is superimposed on the original audio to obtain the adversarial sample. The method iteratively optimizes the adversarial perturbation using a loss function. In this process, each epoch of optimization greedily adjusts the coefficients of the loss function based on the results of the previous epoch to accelerate the iteration speed. The iteration stops when the attack is successful, and the adversarial sample is saved.

[0052] In practice, because there are optimized perturbations in each iteration, the number of audio samples and adversarial perturbations gradually decreases with each iteration. Through multiple iterations, the greedy algorithm will eventually ensure that all perturbations meet the requirements of the perturbation sample set.

[0053] Preferably, the method for solving adversarial examples further includes: Step 2.1, for each audio data in the audio training set input to the adversarial example generation algorithm, adding an initial adversarial perturbation to generate an initial adversarial example; Step 2.2, inputting the initial adversarial example into the target deep learning model to obtain the target output, and comparing the target output with the predefined target label in Step 1; Step 2.3, if the target output matches the predefined target label, retaining the initial adversarial example and adding it to the adversarial example set; Step 2.4, if the target output does not match the predefined target label, optimizing the initial adversarial perturbation into a new round of adversarial perturbation, and re-executing Steps 2.2 to 2.4 until an adversarial example that meets the requirements of the adversarial example set is generated for each audio data in the audio training set.

[0054] The method can begin by setting the maximum number of iterations, learning rate, loss function, etc., for the algorithm. Simultaneously, iterate through the audio training set X. trian This is done to obtain all audio samples in each training set.

[0055] The method randomly generates one-dimensional audio samples x. i Counter-disturbance δ of the same length i This method can acquire adversarial samples for different needs under various attack scenarios. Based on actual requirements, the method can obtain the target attack type in step 1 and, from the algorithm's data sample library, extract one or more target labels that meet the requirements, according to the samples corresponding to non-real labels. In other words, by adding perturbation, the audio samples, after deep learning classification, are assigned to one or more of the aforementioned target labels. Furthermore, these are still non-real samples. In one embodiment of the invention, the target attack type is a no-target attack, therefore multiple labels can be selected.

[0056] Preferably, optimizing the initial adversarial perturbation into a new round of adversarial perturbation further includes: calculating the loss function of the initial adversarial perturbation, and using the ADAM optimizer to calculate the gradient of the loss function, and updating the initial adversarial perturbation into a new round of adversarial perturbation according to the value of the gradient.

[0057] For each audio x i First, calculate the adversarial sample x′ in this round. i =x i +δ i Then x′ i The input is fed into the target model f(), and the output is f(x′). i When f(x′) i When δi = t, skip this sample. The method then calculates the loss function, uses the ADAM optimizer to compute its gradient, and then updates δi.

[0058] Preferably, the loss function is:

[0059] L = L dist +L spec +α·L net

[0060] In the formula, L dist Let L be the perturbation norm, and L be the perturbation norm. dist =||δ i ||2;

[0061] L spec For multi-scale spectral loss, and has

[0062] L net The model classification loss is L. net =CrossEntropyLoss(f(x′) i ), t);

[0063] α is the weight defined based on a greedy algorithm;

[0064] Where, δ i For the i-th audio data x i Corresponding counter-perturbations,

[0065] For the i-th audio data x i The time-domain sample obtained after time-frequency transformation, where j is the dimension number after time-frequency transformation.

[0066] The i-th counter-perturbation x′ i The time-domain sample obtained after time-frequency transformation

[0067] CrossEntropyLoss(f(x′ i ),t) is f(x′ i The cross-entropy loss f(x′) under t different target label categories i To counteract the disturbance x′ i The target output of the target deep learning model.

[0068] Specifically, the target loss function is the main part of the iterative greedy calculation process, and the target loss function used in this invention consists of three parts.

[0069] First, attackers aim to minimize the impact of adversarial perturbations on the audio samples during their attacks. Therefore, this paper introduces a multi-scale spectral loss as an audio content loss term in the loss function. This loss consists of two parts, recording the loss of time-domain and frequency-domain information of the audio samples after sampling. In this way, the loss caused by the algorithm during the solution process can be analyzed at two different scales, in the time and frequency domains, and the possible perturbations can be reconstructed through this estimation.

[0070] Secondly, in the model classification loss, the attacker expects the classifier's classification result to output the specified label t even if the classification is incorrect. Therefore, the method of this invention introduces cross-entropy as a model loss term to evaluate the model classification result f(x′). i The difference between the target result t and the target result t.

[0071] Third, to reduce the likelihood of the adversarial perturbation being detected during an attack, a norm loss term is typically introduced to measure the magnitude of the current perturbation. This invention employs the L2 norm loss term, which is common in the field of adversarial attacks. Depending on the specific circumstances and the expected type of attack, this norm can also be changed to different norms in this invention.

[0072] Ultimately, the loss function used in the optimization training process of the iterative attack is a weighted sum of the above three factors.

[0073] It should be noted that the weight α is used to control the relative importance of other loss terms for classification accuracy. The α in the iteration will cause the starting point of the iteration process to focus more on improving the success rate of attacks rather than reducing the noise of adversarial perturbations.

[0074] The weight α in this invention allows the algorithm to initially prioritize classification errors caused by perturbations and the inherent properties of the audio signal. This approach ensures that local optima are closer to the perturbations created based on the characteristics of the input audio itself. Only when this method fails to find a solution will a greedy algorithm be used to increase the weight of the cross-entropy loss, seeking an attack angle from the limitations of deep learning algorithms.

[0075] Therefore, this invention achieves the optimal value of the target loss function through iterative optimization training, and finally obtains adversarial perturbations which are added to the original audio to generate audio adversarial perturbations.

[0076] Preferably, the greedy algorithm increments the discrete values ​​of the weight α; when the initial value of α cannot satisfy a certain x′ i After the target output matches the predefined target label, the greedy algorithm assigns α to the next discrete value, until x′. i The target output must match the predefined target label.

[0077] As mentioned above, each iteration of the greedy algorithm may generate multiple perturbations that meet the requirements. Therefore, after each iteration, these perturbations are not included in the next iteration. Perturbations that still do not meet the requirements are then used to modify the loss function and δ by changing their weights. i The optimization and iteration strategy is used to ultimately solve all solutions.

[0078] Preferably, the adversarial sample generation parameters for each definition are recorded, as well as the adversarial sample set obtained based on each definition; the adversarial sample set corresponding to the current definition and the perturbation report generated based on the adversarial sample set are displayed in the form of a front-end page; the adversarial sample set corresponding to the historical definition and the perturbation report generated based on the adversarial sample set are stored, and the storage is queried.

[0079] Employing common user interface (UI) methods, this approach can display perturbed sample sets in various ways through an interactive interface. It also supports human-computer interaction for filtering and selecting samples from the set. Furthermore, the method can automatically generate sample set-related reports as needed. These reports can include various conceivable sample characteristics, such as different statistical features of the sample set.

[0080] Each time a sample set is generated, the method records the interaction data and the results of the computer algorithm in generating the sample, thereby supporting functions such as historical backtracking queries.

[0081] A second aspect of this invention relates to an adversarial example generation system for audio data. The system includes a server and a client. The server and client are connected in communication, and the server includes a parameter customization module, an audio adversarial example set generation module, a dataset display module, and a log module. The parameter customization module is used to obtain adversarial example generation parameters for the audio data defined by the client and to call an adversarial example generation algorithm based on these parameters. The audio adversarial example set generation module is used to input the audio data and initial adversarial perturbations into the adversarial example generation algorithm to solve for adversarial examples. The dataset display module and the log module are used to construct an adversarial example set based on the solved adversarial examples and push the interface display and human-computer interaction results of the adversarial example set to the client.

[0082] It should be noted that this system adopts a local B / S architecture, displaying the user interface through a local browser front-end, while the internal logic of the system modules is implemented on a local server. The system requires the following conditions to run: a target model to be attacked and the original audio dataset are available for loss function optimization calculation. After calculation, adversarial perturbations and corresponding adversarial examples are created for the original audio dataset. The adversarial example audio is saved to a local directory and can be used for research on the robustness of deep learning models.

[0083] The front-end is implemented through a client-side interface. The local front-end page facilitates user interaction, allowing users to set key parameters for different system operating modes and connect to the local server. First, the user identifies the target model to attack on the front-end page, then customizes parameter settings and generates adversarial examples. By setting the generation parameters during training and inputting the path to the original audio dataset and the path to save the adversarial audio samples on the front-end page, the user triggers the server to perform audio adversarial perturbation and the creation and saving of the corresponding adversarial examples.

[0084] The server-side component is deployed on a local server. This local server is the core of the system's functional modules, responsible for implementing the specific module logic. The process of creating audio adversarial perturbations and adversarial examples is implemented through parameter settings on the front-end page and dataset uploads.

[0085] Specifically, users can customize the parameters in the adversarial example generation service on the front-end interface. The parameter customization module receives the data content defined in this way and uses it as parameters to call the pre-deployed algorithm.

[0086] The audio adversarial sample set generation module generates adversarial perturbations by calling the adversarial perturbation generation algorithm according to the parameters specified by the user, and adds them to the original audio dataset to obtain the audio adversarial sample set.

[0087] The dataset display module combines the results from the adversarial example set generation module and the report generation module, returning them to the front-end page of the prototype system, and supports customized display. The log module records information such as the model, dataset, and parameters for each experiment and supports querying.

[0088] It is understood that the server can be implemented through communication connections between one or more server devices. The device includes at least one processor, a bus system, and at least one communication interface. The processor consists of a central processing unit, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or other hardware. The memory consists of read-only memory (ROM), random access memory (RAM), etc. The memory can exist independently and be connected to the processor via a bus. The memory can also be integrated with the processor. The hard disk can be a mechanical hard disk (HDD) or a solid-state drive (SSD), etc. This embodiment of the invention does not limit this. The above embodiments are typically implemented using software and hardware. When implemented using software programs, it can be implemented in the form of a computer program product. This computer program product includes one or more computer instructions.

[0089] When computer program instructions are loaded and executed on a computer, the corresponding functions are implemented according to the process provided in the embodiments of this invention. The computer program instructions involved may be assembly instructions, machine instructions, or code written in a programming language, etc.

[0090] The beneficial effects of this invention are that, compared with the prior art, the adversarial example generation method and system for audio data in this invention calculates the loss function from multiple aspects such as model classification loss, time-frequency multi-dimensional loss, and perturbation loss, and uses a greedy algorithm to gradually increase the weight of the model classification loss in the loss function, thereby obtaining the local optimal perturbation of the audio data. This invention is effective and reliable, and can obtain local optima at any position, making the adversarial examples primarily focused on the success rate during the attack process, thus making the adversarial examples comprehensive, accurate, and effective.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for generating an adversarial sample of audio data, the method comprising: The method includes the following steps: Step 1: Define the adversarial sample generation parameters for the audio data, and call the adversarial sample generation algorithm based on the adversarial sample generation parameters; the adversarial sample generation parameters include at least: the name of the adversarial sample generation algorithm, the target attack category, the target attack method, the number of iterations, the batch sample size, the target label, and the normalization constraint size; Step 2: Input the audio data and initial adversarial perturbation into the adversarial example generation algorithm to solve for the adversarial examples; the adversarial example generation algorithm is a deep learning algorithm based on iterative optimization of the adversarial perturbation using a loss function; solving for the adversarial examples further includes: Step 2.1: For each audio data in the audio training set input to the adversarial example generation algorithm, add an initial adversarial perturbation to generate an initial adversarial example. Step 2.2: Input the initial adversarial sample into the target deep learning model to obtain the target output, and compare the target output with the target label predefined in Step 1; Step 2.3: If the target output matches a predefined target label, retain the initial adversarial sample and add it to the adversarial sample set; Step 2.4: If the target output does not match the predefined target label, optimize the initial adversarial perturbation into a new round of adversarial perturbation, and repeat steps 2.2 to 2.4 until adversarial samples that meet the requirements of the adversarial sample set are generated for each audio data in the audio training set. The adversarial example generation algorithm is based on the weighted sum of model classification loss, multi-scale spectral loss, and perturbation norm to achieve iterative optimization of adversarial perturbations. When no solution can be found in the iterative optimization, the weights in the weighted sum are iteratively increased based on a greedy algorithm to solve for all adversarial examples. Step 3: Construct an adversarial sample set based on the obtained adversarial samples, and use an interactive interface to display and interact with the adversarial sample set. The loss function is: In the formula, Let be the perturbation norm, and have ; For multi-scale spectral loss, and has ; The classification loss is used for the model, and there is ; The weights are defined based on a greedy algorithm; in, For the first audio data Corresponding counter-perturbations, For the first audio data The time-domain sample obtained after time-frequency transformation, where The dimension after time-frequency variation. No. One counter-disturbance The time-domain sample obtained after time-frequency transformation for exist Cross-entropy loss for different target label categories To counteract disturbances The target output of the target deep learning model; The greedy algorithm increments the weight. discrete values; when The initial value cannot make a certain After satisfying the requirement that its target output matches a predefined target label, the greedy algorithm will... Assign the value to the next discrete value, until the... The target output must match the predefined target label.

2. The method for generating adversarial examples for audio data according to claim 1, characterized in that: The initial adversarial disturbance is a random noise signal with the same duration as the audio data.

3. The method for generating adversarial examples for audio data according to claim 1, characterized in that: The optimization of the initial adversarial perturbation into a new round of adversarial perturbation also includes: Calculate the loss function of the initial adversarial perturbation, and use the ADAM optimizer to calculate the gradient of the loss function. Update the initial adversarial perturbation to a new round of adversarial perturbation based on the value of the gradient.

4. The method for generating adversarial examples for audio data according to claim 1, characterized in that: Record the adversarial sample generation parameters defined each time, and the adversarial sample set obtained based on each definition; The adversarial sample set corresponding to the current definition and the disturbance report generated based on the adversarial sample set are displayed in the form of a front-end page; The adversarial sample set corresponding to the historical definition and the perturbation report generated based on the adversarial sample set are stored, and the storage is queried.

5. An adversarial example generation system for audio data, executing the adversarial example generation method for audio data according to any one of claims 1 to 4, characterized in that: The system includes a server and a client; The server and client communicate with each other, and the server includes a parameter customization module, an audio adversarial example set generation module, a dataset display module, and a log module; wherein, The parameter customization module is used to obtain the adversarial sample generation parameters of the audio data defined by the client, and call the adversarial sample generation algorithm based on the adversarial sample generation parameters; The audio adversarial sample set generation module is used to input the audio data and the initial adversarial perturbation into the adversarial sample generation algorithm to solve for the adversarial samples; The dataset display module and log module are used to construct an adversarial sample set based on the adversarial samples obtained from the solution, and push the interface display and human-computer interaction results of the adversarial sample set to the client.

Images