Black box confrontation sample generation method for speech recognition system

Abstract

The invention discloses a black box confrontation sample generation method for a speech recognition system. The method comprises the following steps: inputting an original speech sample and a target text into an integrated attack output initial momentum and an initial confrontation sample; finite difference gradient estimation is carried out every fixed times, a current dominant frame in a voice confrontation sample is determined firstly, then the initial probability of selection of each sampling value is given, the final probability of selection of each sampling value is calculated according to the probability of selection of the dominant frame, and then finite difference gradient estimation is carried out; obtaining a gradient estimation value to update the probability that the sampling value is selected; for other iterations, selecting part of frames in the voice confrontation sample according to the probability that the dominant frame is selected, and adding a disturbance value to the selected frame to form a disturbed voice confrontation sample; the method has the advantages that on the premise that the voice content is not seriously damaged, it can be guaranteed that the voice is recognized as the pre-designed malicious content, and the mobility, the concealment, the query efficiency and the attack success rate of resisting the sample attack are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition systems, and in particular to a method for generating black-box adversarial samples for speech recognition systems. Background Art

[0002] A speech adversarial sample is a sample that has been intentionally slightly disturbed by the attacker. The main purpose of a speech adversarial sample is to make the target speech recognition system misidentify it, or even recognize it as the result specified by the attacker. In terms of auditory perception, a speech adversarial sample is similar to the sample before it was modified. In terms of function, a speech adversarial sample can make the target speech recognition system output an erroneous recognition result or output a malicious command to control some voice-controlled devices, such as some smart furniture controlled by voice.

[0003] The mainstream methods of speech adversarial sample attacks can be divided into two types according to whether the attacker can use the target speech recognition system: white-box attack and black-box attack. White-box attack refers to the assumption that the attacker can obtain the model information of the target speech recognition system (including network model and parameters, etc.). The white-box adversarial sample attack method mainly uses the logical output of the last layer of the network model of the target speech recognition system to design and optimize the objective function, and then uses the gradient back-propagation mechanism to continuously optimize the objective function, and finally obtains the adversarial sample that achieves the attacker's purpose. Since the white-box attack requires a complete understanding of all information about the network model of the target speech recognition system, and in real life, the attacker usually cannot obtain the internal information of the network model of the target speech recognition system, the white-box adversarial sample attack method cannot be applied to attacks in the actual physical world. Black-box attack refers to the assumption that the attacker cannot obtain the model information of the target speech recognition system (including network model and parameters, etc.), but can obtain the output of the target speech recognition system for the input speech. However, the current black-box adversarial sample attack method requires the output of the last layer of the network model of the target speech recognition system, rather than the text output. In addition, most of the current speech adversarial sample attack methods are only effective in specific speech recognition systems.

[0004] Black-box adversarial sample attack methods usually use gradient-free methods to generate speech adversarial samples. These gradient-free methods are mainly divided into two categories. The first category is intelligent optimization methods, such as genetic algorithms, particle swarm algorithms, etc.; the second category is gradient estimation methods based on zero-order optimization, such as finite differences, natural gradient descent algorithms, etc. Current research on black-box adversarial sample attacks often requires too much query consumption and too many adversarial perturbations. Fortunately, speech adversarial samples are transferable. The transferability of speech adversarial samples can be used to first use a transfer attack to generate a preliminary speech adversarial sample, and then use a query attack to optimize the speech adversarial sample. In this way, the attacker can greatly reduce query consumption and increase the success rate of the attack.

[0005] Research on adversarial sample attacks in speech recognition scenarios still faces many challenges and bottlenecks. (1) The problem of transferability of adversarial sample attacks. Existing methods often generate corresponding speech adversarial samples on specific speech recognition systems or datasets, which are difficult to transfer well to other network models or datasets. In addition, since speech recognition systems are more complex than classification models such as image / speech recognition, the transferability of speech adversarial samples generated in the field of speech recognition is relatively weak, and the success rate of attacks is difficult to guarantee. (2) The problem of the concealment of adversarial sample attacks. In the process of generating speech adversarial samples, a large number of iterative optimization processes are usually required to find the perturbations that can have the greatest impact on the original speech samples. However, it is necessary to keep the added perturbations as imperceptible as possible while maintaining the adversarial nature to improve the concealment of the attack. (3) The problem of query efficiency of black-box adversarial sample attacks. In black-box scenarios, attackers cannot directly access the parameters and gradient information of the network model of the target speech recognition system, so they need to rely more on the output of the network model to attack, resulting in an increase in the number of queries. Although a few studies on black-box query attacks have appeared, these methods still require a large number of queries. In actual scenarios, the network model of the target speech recognition system will limit the number of queries or the frequency of accessing the network model to prevent adversarial sample attacks. On the other hand, the complexity of the network model of the target speech recognition system itself is also increasing, which means that the accuracy of the estimated gradient will become lower, which also leads to higher query costs. (4) The success rate of adversarial sample attacks under complex target texts. At present, adversarial sample attacks have a high attack success rate under simple target texts, but they are still ineffective when facing complex target texts. Therefore, how to improve the attack success rate of complex target texts will also be a key research direction in the future. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a black-box adversarial sample generation method for a speech recognition system, which does not require the attacker to obtain the model information of the speech recognition system, combines migration attack and query attack, and uses a frame adaptation strategy and an improved gradient estimation method to generate speech adversarial samples. It can carry out adversarial sample attacks on the speech recognition system without seriously damaging the speech content, and can ensure that the correct content cannot be extracted from the speech, and even be identified as pre-designed malicious content, which greatly improves the migration, concealment, query efficiency and attack success rate of adversarial sample attacks.

[0007] The technical solution adopted by the present invention to solve the above technical problem is: a method for generating black box adversarial samples for a speech recognition system, characterized by comprising the following steps:

[0008] Step 1: Get a piece of audio as the original speech sample; then set the target text, which is the original speech sample that can be recognized by the speech recognition system after generating the adversarial sample through adversarial attack;

[0009] Step 2: Input the original speech sample and target text into the integrated attack, which outputs the initial momentum and the initial speech adversarial sample;

[0010] Step 3: Let k represent the number of iterations, and the initial value of k is 0;

[0011] Step 4: During the kth iteration, find Then randomly select a frame from all the dominant frames as the current dominant frame, and determine the current dominant frame in The corresponding continuous sampling values ​​in; where k = 0 is the initial speech adversarial sample output by the integrated attack, when k≠0 represents the speech adversarial sample obtained in the kth iteration;

[0012] Step 5: During the kth iteration, when k is divisible by inter and k = 0, based on the probability assigned to the current dominant frame, calculate The probability of each sampling value being selected in is then selected, and then some sampling values ​​are selected according to the probability; then a finite difference gradient estimation is performed on each selected sampling value to obtain a gradient estimation value of each selected sampling value, and then step 6 is executed; wherein inter is a preset positive integer;

[0013] In the kth iteration, when k is divisible by inter and When n=1,2,3…, based on the gradient estimates of all sample values ​​selected in the (n-1)th iteration, obtain The probability of each sampling value being selected in is calculated, and then some sampling values ​​are selected according to the probability; then, a finite difference gradient estimation is performed on each selected sampling value to obtain a gradient estimation value of each selected sampling value, and then step 6 is executed;

[0014] In the kth iteration, when k cannot be divided by inter, based on the probability of the current dominant frame being selected, Select some frames; then add a random Gaussian perturbation value to each sample value in each selected frame to obtain a frame with a perturbation value; then recompose all the frames with perturbation values ​​and all the unselected frames into a perturbed speech adversarial sample; then input the perturbed speech adversarial sample into the speech recognition system, the speech recognition system outputs a probability vector, and calculates the CTC loss function of the probability vector and the target text; finally, compare the CTC loss function calculated in this iteration with the CTC loss function calculated in step 7 of the previous iteration. If the former is less than the latter, it means that the added perturbation value is valid, and then execute step 6. If the former is greater than or equal to the latter, it means that the added perturbation value is invalid, and then execute step 6;

[0015] Step 6: During the kth iteration, calculate the momentum obtained at the k+1th iteration Then calculate the speech adversarial sample obtained by the k+1th iteration Where μ is the attenuation coefficient, k = 0 is the initial momentum of the integrated attack output, k≠0 represents the momentum obtained at the kth iteration, ‖‖ ∞ is the infinity norm symbol, when k is divisible by inter express The vector of gradient estimates of all sample values ​​selected in , when k cannot be divided by inter and the added perturbation value is valid express A vector of perturbations added to the sample values ​​in all frames selected in . When k is not divisible by inter and the perturbations added are invalid is 0, represents the initial speech adversarial sample of the integrated attack output, th represents the perturbation constraint threshold, th∈[0.01, 0.02], Represents the restriction vector The absolute value of each element in cannot exceed th, β is the control The size parameter, sign() is the sign function;

[0016] Step 7: During the kth iteration, The input is sent to the speech recognition system, which outputs the probability vector and the transcribed text. Then the CTC loss function of the probability vector and the target text is calculated, and the transcribed text is compared with the target text. If the two are the same, the output is And end, completing the black-box adversarial sample attack; otherwise, let k=k+1, and then return to step 4 to continue execution; where the "=" in k=k+1 is an assignment symbol.

[0017] The specific process of step 2 is:

[0018] Step 2.1: Let t represent the number of iterations, and the initial value of t is 0; let t max Indicates the maximum number of iterations; initializes the momentum, the initial value of the momentum M0 = 0;

[0019] Step 2.2: Calculate the momentum M obtained at the t+1th iteration t+1 , Where μ is the attenuation coefficient, M at t = 0 t M0, t≠0 t represents the momentum obtained at the tth iteration, ‖‖ ∞ is the symbol of the infinite norm, when t = 0, x t is the original speech sample x0, when t≠0, x t represents the speech adversarial sample obtained in the tth iteration, y tar represents the target text, It means to find J(x t ,y tar ) about x t The gradient of J(x t ,y tar ) represents x t Input into the speech recognition system, after the speech recognition system outputs the probability vector, the probability vector calculated is equal to y tar CTC loss function;

[0020] Step 2.3: Use the momentum M obtained at the t+1th iteration t+1 , calculate the speech adversarial sample x obtained at the t+1th iteration t+1 , x t+1 =x0+clip th (x t +β·sign(M t+1 )-x0), where x0 represents the original speech sample, th represents the perturbation constraint threshold, th∈[0.01, 0.02], clip th (x t +β·sign(M t+1 )-x0) represents the restriction vector x t +β·sign(M t+1 )-x0, the absolute value of each element cannot exceed th, β is the control sign (M t+1 ) is a parameter of size, sign() is a function to get the sign;

[0021] Step 2.4: Determine whether t+1 reaches t max If so, then M t+1As the initial momentum of the integrated attack output, x t+1 As the initial speech adversarial sample of the integrated attack output; otherwise, let t = t + 1, and then return to step 2.2 to continue execution; where the "=" in t = t + 1 is an assignment symbol.

[0022] In step 4, find The process of all dominant frames in is: Input into the speech recognition system, the speech recognition system will Divided into several frames, the speech recognition system outputs The corresponding transcripts are then compared using the longest common subsequence algorithm Corresponding transcribed text and target text, obtain the mismatching characters in the two texts, and then find the frames corresponding to these mismatching characters, and these found frames are called dominant frames.

[0023] In step 5, when k is divisible by inter and k=0, based on the probability of the current dominant frame being selected, calculate The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is:

[0024] Step 5.1a: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, and γ is the preset hyperparameter;

[0025] Step 5.2a: All sample values ​​in are assigned an initial probability of 0.1; then calculate The probability of each sample value being selected is The probability of any sample value being selected is the product of the initial probability of the sample value being selected and the probability of the frame where the sample value is located being selected;

[0026] Step 5.3a: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

[0027] In step 5, when k is divisible by inter and When n=1,2,3…, based on the gradient estimates of all sample values ​​selected in the (n-1)th iteration, obtain The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is:

[0028] Step 5.1b: From the gradient estimates of all the sample values ​​selected in step 5 during the (n-1)inter iteration, select the largest 20% of the gradient estimates, normalize these gradient estimates to the interval (0, 1] to obtain normalized gradient values, and record the indexes of the sample values ​​corresponding to these normalized gradient values;

[0029] Step 5.2b: Get The probability of each sample value being selected in If the index of the sample value is the same as one of the recorded indices, the probability of the sample value being selected is equal to the sum of the initial probability of the sample value being selected multiplied by 0.9 and the corresponding gradient normalized value in step 5.1b multiplied by 0.1; otherwise, the probability of the sample value being selected is equal to the initial probability of the sample value being selected; where, The initial probability of each sample value being selected is 0.1;

[0030] Step 5.3b: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

[0031] In step 5, when k cannot be divided by inter, based on the probability of the current dominant frame being selected, The specific process of selecting some frames is as follows:

[0032] Step 5.1c: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, and γ is the preset hyperparameter;

[0033] Step 5.2c: Take a random number in the interval (0, 1) and traverse For each frame in , compare the probability of the currently traversed frame being selected with the size of the random number. If the probability of the currently traversed frame being selected is greater than the random number, the currently traversed frame is selected; After all frames in are traversed, all selected frames are obtained.

[0034] In the step 5, when k cannot be divided by inter, an initial disturbance vector is randomly generated for each selected frame, the length of the initial disturbance vector is consistent with the length of the corresponding frame, the initial disturbance vector satisfies the Gaussian distribution, and the value range of each element is [-1, 1]; then each element in the initial disturbance vector is multiplied by 0.0001 as the final Gaussian disturbance value, and the final Gaussian disturbance value is added to the corresponding sampling value.

[0035] Compared with the prior art, the advantages of the present invention are:

[0036] 1) The method of the present invention combines migration attack and query attack to realize black-box adversarial sample attack on speech recognition system, and can carry out adversarial sample attack on different speech recognition systems. The migration is relatively strong, so that the success rate of the attack is guaranteed.

[0037] 2) In the black box attack scenario, previous methods need to add large disturbances to the entire voice file. The method of the present invention uses the characteristics of the voice sequence itself to locate the important content part of the voice during the attack, and uses the frame adaptation strategy to specifically add random Gaussian disturbances to some frames of the voice adversarial sample, which can reduce the addition of unnecessary disturbances. The added random Gaussian disturbance is highly imperceptible, which improves the concealment of the voice adversarial sample attack.

[0038] 3) The previous gradient estimation method directly uses the method in the image field without considering the characteristics of the speech itself, and therefore requires a large number of queries. The method of the present invention improves the gradient estimation method and can improve the accuracy of gradient estimation and query efficiency.

[0039] 4) Under the condition that the attacker cannot obtain the model information of the speech recognition system, the method of the present invention combines the migration attack and the query attack, uses the frame adaptation strategy, and improves the gradient estimation method to realize a novel adversarial attack framework, which can successfully generate speech adversarial samples of the speech recognition system and greatly improve the concealment, query efficiency and attack success rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a general implementation block diagram of the method of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below with reference to the accompanying drawings.

[0042] The present invention proposes a black box adversarial sample generation method for a speech recognition system, and its overall implementation block diagram is as follows: Figure 1 As shown, it includes the following steps:

[0043] Step 1: Get a piece of audio as the original speech sample; then set the target text, which is the original speech sample that can be recognized by the speech recognition system after generating the adversarial sample through adversarial attack; here, in actual scenarios, a piece of speech spoken by a person is usually obtained as the original speech sample.

[0044] Step 2: See Figure 1 In part (a), the original speech sample and the target text are input into the integrated attack, and the integrated attack outputs the initial momentum and the initial speech adversarial sample; here, the integrated attack is a type of transfer attack.

[0045] In this embodiment, the specific process of step 2 is:

[0046] Step 2.1: Let t represent the number of iterations, and the initial value of t is 0; let t max Indicates the maximum number of iterations. In this embodiment, t max =5000; initialize momentum, the initial value of momentum M0=0.

[0047] Step 2.2: Calculate the momentum M obtained at the t+1th iteration t+1 , Wherein, μ is the attenuation coefficient. In this embodiment, μ=0.9. When t=0, M t M0, t≠0 t represents the momentum obtained at the tth iteration, ‖‖ ∞ is the symbol of the infinite norm, when t = 0, x t is the original speech sample x0, when t≠0, x t represents the speech adversarial sample obtained in the tth iteration, y tar represents the target text, It means to find J(x t ,y tar ) about x t The gradient of J(x t ,y tar ) represents x t Input into the speech recognition system, after the speech recognition system outputs the probability vector, the probability vector calculated is equal to y tar The CTC loss function.

[0048] Step 2.3: Use the momentum M obtained at the t+1th iteration t+1 , calculate the speech adversarial sample x obtained at the t+1th iteration t+1 , x t+1=x0+clip th (x t +β·sign(M t+1 )-x0), where x0 represents the original speech sample, th represents the disturbance constraint threshold, th∈[0.01, 0.02], and the specific value of th can be determined according to the original speech sample. Usually, the original speech sample is sampled at a sampling rate of 16000 Hz, and the absolute value of each sampled value is calculated, and then the average of all absolute values ​​is calculated. If the average value is large, th can take a larger value in the interval [0.01, 0.02]. Otherwise, a smaller value is taken. th (x t +β·sign(M t+1 )-x0) represents the restriction vector x t +β·sign(M t+1 )-x0, the absolute value of each element cannot exceed th, β is the control sign (M t+1 ) is a parameter of size, in this embodiment, β=0.0001, and sign() is a sign function.

[0049] Step 2.4: Determine whether t+1 reaches t max If so, then M t+1 As the initial momentum of the integrated attack output, x t+1 As the initial speech adversarial sample of the integrated attack output; otherwise, let t = t + 1, and then return to step 2.2 to continue execution; where the "=" in t = t + 1 is an assignment symbol.

[0050] Step 3: Let k represent the number of iterations, and the initial value of k is 0.

[0051] Step 4: During the kth iteration, see Figure 1 In part (b), find Then randomly select a frame from all the dominant frames as the current dominant frame, and determine the current dominant frame in The corresponding continuous sampling values ​​in; where k = 0 is the initial speech adversarial sample output by the integrated attack, when k≠0 Represents the speech adversarial sample obtained in the kth iteration.

[0052] In this embodiment, in step 4, find The process of all dominant frames in is: Input into the speech recognition system, the speech recognition system will Divided into several frames, the speech recognition system outputs The corresponding transcripts are then compared using the longest common subsequence algorithm Corresponding transcribed text and target text, obtain the mismatching characters in the two texts, and then find the frames corresponding to these mismatching characters, and these found frames are called dominant frames.

[0053] Step 5: During the kth iteration, when k is divisible by inter and k = 0, refer to Figure 1 In part (e), based on the probability assigned to the current dominant frame, we calculate The probability of each sample value being selected in is then used to select some sample values ​​according to the probability; see Figure 1 (c-1) part in the equation, and then perform finite difference gradient estimation on each selected sampling value to obtain the gradient estimation value of each selected sampling value, and then execute step 6; wherein inter is a preset positive integer, such as taking inter=50.

[0054] In the kth iteration, when k is divisible by inter and When n=1,2,3…, based on the gradient estimates of all sample values ​​selected in the (n-1)th iteration, obtain The probability of each sample value being selected in is then used to select some sample values ​​according to the probability; see Figure 1 Then, perform finite difference gradient estimation on each selected sampling value to obtain the gradient estimation value of each selected sampling value, and then execute step 6.

[0055] In the kth iteration, when k is not divisible by inter, see Figure 1 In the (c-2) part, based on the probability of the current dominant frame being selected, Some frames are selected; then a random Gaussian perturbation value is added to each sampling value in each selected frame to obtain a frame with a perturbation value; then all frames with perturbation values ​​added and all unselected frames are reassembled into a perturbed speech adversarial sample; then the perturbed speech adversarial sample is input into the speech recognition system, the speech recognition system outputs a probability vector, and the CTC loss function of the probability vector and the target text is calculated; finally, the CTC loss function calculated in this iteration is compared with the CTC loss function calculated in step 7 of the previous iteration. If the former is less than the latter, it means that the added perturbation value is valid, and then step 6 is executed. If the former is greater than or equal to the latter, it means that the added perturbation value is invalid, and then step 6 is executed.

[0056] In this embodiment, in step 5, when k is divisible by inter and k=0, based on the probability of the current dominant frame being selected, The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is:

[0057] Step 5.1a: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, γ is the preset hyperparameter, and in this embodiment, γ=0.1, where the probability of the frames before and after the current dominant frame being selected gradually decays from small to large according to the interval with the current dominant frame, and the probability of the two frames before and after the current dominant frame and with the same interval with the current dominant frame being selected is assigned the same value.

[0058] Step 5.2a: All sample values ​​in are assigned an initial probability of 0.1; then calculate The probability of each sample value being selected is The probability of any sample value being selected is the product of the initial probability of the sample value being selected and the probability of the frame where the sample value is located being selected.

[0059] Step 5.3a: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

[0060] In this embodiment, in step 5, when k is divisible by inter and When n=1,2,3…, based on the gradient estimates of all sample values ​​selected in the (n-1)th iteration, obtain The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is:

[0061] Step 5.1b: From the gradient estimates of all the sample values ​​selected in step 5 during the (n-1)inter iteration, select the largest 20% of the gradient estimates, normalize these gradient estimates to the interval (0, 1] to obtain normalized gradient values, and record the indexes of the sample values ​​corresponding to these normalized gradient values.

[0062] Step 5.2b: Get The probability of each sample value being selected in If the index of the sample value is the same as one of the recorded indices, the probability of the sample value being selected is equal to the sum of the initial probability of the sample value being selected multiplied by 0.9 and the corresponding gradient normalized value in step 5.1b multiplied by 0.1; otherwise, the probability of the sample value being selected is equal to the initial probability of the sample value being selected; where, Each sample value in is selected with an initial probability of 0.1.

[0063] Step 5.3b: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

[0064] In this embodiment, in step 5, when k cannot be divided by inter, based on the probability of the current dominant frame being selected, The specific process of selecting some frames is as follows:

[0065] Step 5.1c: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, γ is the preset hyperparameter, and in this embodiment, γ=0.1, where the probability of the frames before and after the current dominant frame being selected gradually decays from small to large according to the interval with the current dominant frame, and the probability of the two frames before and after the current dominant frame and with the same interval with the current dominant frame being selected is assigned the same value.

[0066] Step 5.2c: Take a random number in the interval (0, 1) and traverse For each frame in , compare the probability of the currently traversed frame being selected with the size of the random number. If the probability of the currently traversed frame being selected is greater than the random number, the currently traversed frame is selected; After all frames in are traversed, all selected frames are obtained.

[0067] In this embodiment, in step 5, when k cannot be divided by inter, an initial disturbance vector is randomly generated for each selected frame, the length of the initial disturbance vector is consistent with the length of the corresponding frame, the initial disturbance vector satisfies the Gaussian distribution, and the value range of each element is [-1, 1]; then each element in the initial disturbance vector is multiplied by 0.0001 as the final Gaussian disturbance value, and the final Gaussian disturbance value is added to the corresponding sampling value.

[0068] Step 6: During the kth iteration, see Figure 1 Part (d) of the above example calculates the momentum obtained at the k+1th iteration. Then calculate the speech adversarial sample obtained by the k+1th iteration Wherein, μ is the attenuation coefficient. In this embodiment, μ=0.9, k=0 is the initial momentum of the integrated attack output, k≠0 represents the momentum obtained at the kth iteration, ‖‖ ∞ is the infinity norm symbol, when k is divisible by inter express The vector of gradient estimates of all sample values ​​selected in , when k cannot be divided by inter and the added perturbation value is valid express A vector of perturbations added to the sample values ​​in all frames selected in . When k is not divisible by inter and the perturbations added are invalid is 0, represents the initial speech adversarial sample of the integrated attack output, th represents the perturbation constraint threshold, th∈[0.01, 0.02], Represents the restriction vector The absolute value of each element in cannot exceed th, β is the control The parameter of size, in this embodiment, is set to β=0.0001, and sign() is a sign function.

[0069] Step 7: During the kth iteration, The input is sent to the speech recognition system, which outputs the probability vector and the transcribed text. Then the CTC loss function of the probability vector and the target text is calculated, and the transcribed text is compared with the target text. If the two are the same, the output is And end, completing the black-box adversarial sample attack; otherwise, let k=k+1, and then return to step 4 to continue execution; where the "=" in k=k+1 is an assignment symbol.

[0070] In order to further illustrate the feasibility and effectiveness of the method of the present invention, a simulation experiment was carried out on the method of the present invention.

[0071] The simulation experiment was carried out on a computer with two NVIDIA TM The experiment was carried out on a Windows machine with a GeForce RTX 3080ti (12GB) GPU, CUDA version 11.6, driver version 546.65, and all codes were written in Python 3.8.0 and PyTorch 1.12.0.

[0072] The simulation experiment selected the LibriSpeech dataset, which is a large corpus containing about 1,000 hours of English speech from the audiobooks of the LibriVox project. These English speech have been segmented and correctly aligned. 100 English speech were randomly selected from the LibriSpeech dataset, and these 100 English speech were intercepted to obtain 100 speech segments of the same duration, such as two seconds, as the original speech samples.

[0073] In the simulation experiments, DeepSpeech is used as the speech recognition system. DeepSpeech has been widely used as a target model in previous studies.

[0074] In the simulation experiment, four evaluation indicators were selected, namely, attack success rate (SR), number of queries (NoQ), Pearson coefficient (PCC), and signal-to-noise ratio (SNR). SR represents the ratio of the number of speech adversarial samples transcribed into target texts on the target model, NoQ represents the number of times the target model is queried to generate speech adversarial samples, PCC is used to measure the similarity between the original speech sample and the target speech waveform, and SNR is used to measure the noise level added to the generated speech adversarial samples.

[0075] In the simulation experiment, the comparison method uses the existing Monte Carlo Symbol Attack (MGSA) method, which has been documented in "Query-Efficient Adversarial Attack With Low Perturbation Against End-to-End Speech Recognition Systems".

[0076] Table 1 shows the SR average, NoQ average, PCC average, and SNR average comparison data of 500 speech adversarial samples generated by the existing MGSA method and the method of the present invention using 5 target texts to attack 100 original speech samples. Among them, the 5 complex target texts are "show me my last messages", "call the police for help quickly", "remove all photos in my phone", "clear all SMS from my phone", and "what is my schedule for tomorrow".

[0077] Table 1 Comparison data of SR average, NoQ average, PCC average, and SNR average of 500 speech adversarial samples generated by the existing MGSA method and the method of the present invention using 5 target texts to attack 100 original speech samples

[0078]

[0079] It can be seen from the data listed in Table 1 that when faced with complex target texts, the existing MGSA method has deficiencies in various evaluation indicators, while the method of the present invention shows significant advantages in various evaluation indicators.

[0080] The existing MGSA method can achieve targeted attacks to a certain extent, but it still requires more than 100,000 queries, and the attack success rate is mostly less than 80%. In contrast, the attack success rate of the method of the present invention is mostly more than 95%, the PCC can reach 0.99, and the number of queries required is mostly less than 50,000, which is significantly lower than the number of queries required by the existing MGSA method. This is enough to show that the method of the present invention is feasible and effective.

Claims

1. A black-box adversarial sample generation method for speech recognition systems, Features The following steps are involved: Step 1: Get a piece of audio as the original speech sample; then set the target text, which is the original speech sample that can be recognized by the speech recognition system after generating the adversarial sample through adversarial attack; Step 2: Input the original speech sample and target text into the integrated attack, which outputs the initial momentum and the initial speech adversarial sample; Step 3: Let k represent the number of iterations, and the initial value of k is 0; Step 4: During the kth iteration, find All dominant frames in ; Then randomly select a frame from all the dominant frames as the current dominant frame, and determine the current dominant frame in The corresponding continuous sampling values ​​in; where k = 0 is the initial speech adversarial sample output by the integrated attack, when k≠0 represents the speech adversarial sample obtained in the kth iteration; Step 5: During the kth iteration, when k is divisible by inter and k = 0, based on the probability assigned to the current dominant frame, calculate The probability of each sampling value being selected in is then selected, and then some sampling values ​​are selected according to the probability; then a finite difference gradient estimation is performed on each selected sampling value to obtain a gradient estimation value of each selected sampling value, and then step 6 is executed; wherein inter is a preset positive integer; In the kth iteration, when k is divisible by inter and When , based on the gradient estimates of all sample values ​​selected during the (n-1)th iteration, obtain The probability of each sampling value being selected in is calculated, and then some sampling values ​​are selected according to the probability; then, a finite difference gradient estimation is performed on each selected sampling value to obtain a gradient estimation value of each selected sampling value, and then step 6 is executed; In the kth iteration, when k cannot be divided by inter, based on the probability of the current dominant frame being selected, Select some frames; then add a random Gaussian perturbation value to each sample value in each selected frame to obtain a frame with a perturbation value; then recompose all the frames with perturbation values ​​and all the unselected frames into a perturbed speech adversarial sample; then input the perturbed speech adversarial sample into the speech recognition system, the speech recognition system outputs a probability vector, and calculates the CTC loss function of the probability vector and the target text; finally, compare the CTC loss function calculated in this iteration with the CTC loss function calculated in step 7 of the previous iteration. If the former is less than the latter, it means that the added perturbation value is valid, and then execute step 6. If the former is greater than or equal to the latter, it means that the added perturbation value is invalid, and then execute step 6; Step 6: During the kth iteration, calculate the momentum obtained at the k+1th iteration Then calculate the speech adversarial sample obtained by the k+1th iteration Where μ is the attenuation coefficient, k = 0 is the initial momentum of the integrated attack output, k≠0 represents the momentum obtained at the kth iteration, ‖ ‖ ∞ is the infinity norm symbol, when k is divisible by inter express The vector of gradient estimates of all sample values ​​selected in , when k cannot be divided by inter and the added perturbation value is valid express A vector of perturbations added to the sample values ​​in all frames selected in . When k is not divisible by inter and the perturbations added are invalid is 0, represents the initial speech adversarial sample of the integrated attack output, th represents the perturbation constraint threshold, th∈[0.01, 0.02], Represents the restriction vector The absolute value of each element in cannot exceed th, β is the control The size parameter, sign() is the sign function; Step 7: During the kth iteration, The input is sent to the speech recognition system, which outputs the probability vector and the transcribed text. Then the CTC loss function of the probability vector and the target text is calculated, and the transcribed text is compared with the target text. If the two are the same, the output is And end, completing the black-box adversarial sample attack; otherwise, set k=k+1, and then return to step 4 to continue execution; where "=" in k=k+1 is an assignment symbol.

2. According to claim 1, a method for generating black-box adversarial samples for a speech recognition system, Features The specific process of step 2 is as follows: Step 2.1: Let t represent the number of iterations, and the initial value of t is 0; let t max Indicates the maximum number of iterations; initializes the momentum, the initial value of the momentum M 0 =0; Step 2.2: Calculate the momentum M obtained at the t+1th iteration t+1 , Where μ is the attenuation coefficient, M at t = 0 t M 0 , t≠0 when M t represents the momentum obtained at the tth iteration, ‖‖ ∞ is the symbol of the infinite norm, when t = 0, x t is the original speech sample x 0 , when t≠0, x t represents the speech adversarial sample obtained in the tth iteration, y tar represents the target text, It means to find J(x t ,y tar ) about x t The gradient of J(x t ,y tar ) represents x t Input into the speech recognition system, after the speech recognition system outputs the probability vector, the probability vector calculated is equal to y tar CTC loss function; Step 2.3: Use the momentum M obtained at the t+1th iteration t+1 , calculate the speech adversarial sample c obtained in the t+1th iteration t+1 , c t+1 =x 0 +clip th (c t +β·sign(M t+1 )-x 0 ), where x 0 represents the original speech sample, th represents the perturbation constraint threshold, th∈[0.01, 0.02], clip th (x t +β·sign(M t+1 )-x 0 ) represents the restriction vector x t +β·sign(M t+1 )-x 0 The absolute value of each element in cannot exceed th, β is the control sign (M t+1 ) is a parameter of size, sign() is a function to get the sign; Step 2.4: Determine whether t+1 reaches t max If so, then M t+1 As the initial momentum of the integrated attack output, x t+1 As the initial speech adversarial sample of the integrated attack output; otherwise, let t = t + 1, and then return to step 2.2 to continue execution; where "=" in t = t + 1 is an assignment symbol.

3. According to claim 1, a method for generating black-box adversarial samples for a speech recognition system, Features In step 4, find The process of all dominant frames in is: Input into the speech recognition system, the speech recognition system will Divided into several frames, the speech recognition system outputs The corresponding transcripts are then compared using the longest common subsequence algorithm Corresponding transcribed text and target text, obtain the mismatching characters in the two texts, and then find the frames corresponding to these mismatching characters, and these found frames are called dominant frames.

4. According to claim 1, a method for generating black-box adversarial samples for a speech recognition system, Features In step 5, when k is divisible by inter and k=0, based on the probability of the current dominant frame being selected, calculate The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is: Step 5.1a: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, and γ is the preset hyperparameter; Step 5.2a: All sample values ​​in are assigned an initial probability of 0.1; then calculate The probability of each sample value being selected is The probability of any sample value being selected is the product of the initial probability of the sample value being selected and the probability of the frame where the sample value is located being selected; Step 5.3a: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

5. According to claim 1, a method for generating black-box adversarial samples for a speech recognition system, Features In step 5, when k is divisible by inter and When , based on the gradient estimates of all sample values ​​selected during the (n-1)th iteration, obtain The probability of each sample value being selected in , and then the specific process of selecting some sample values ​​according to the probability is: Step 5.1b: From the gradient estimates of all the sample values ​​selected in step 5 during the (n-1)inter iteration, select the largest 20% of the gradient estimates, normalize these gradient estimates to the interval (0, 1] to obtain normalized gradient values, and record the indexes of the sample values ​​corresponding to these normalized gradient values; Step 5.2b: Get The probability of each sample value being selected in If the index of the sample value is the same as one of the recorded indices, the probability of the sample value being selected is equal to the sum of the initial probability of the sample value being selected multiplied by 0.9 and the corresponding gradient normalized value in step 5.1b multiplied by 0.1; otherwise, the probability of the sample value being selected is equal to the initial probability of the sample value being selected; where, The initial probability of each sample value being selected is 0.1; Step 5.3b: Take a random number in the interval (0, 1) and traverse For each sample value in , compare the probability of the currently traversed sample value being selected with the size of the random number. If the probability of the currently traversed sample value being selected is greater than the random number, the currently traversed sample value is selected; After all the sampling values ​​in are traversed, all the selected sampling values ​​are obtained.

6. A method for generating black-box adversarial samples for a speech recognition system according to claim 1, Features In step 5, when k cannot be divided by inter, based on the probability of the current dominant frame being selected, The specific process of selecting some frames is as follows: Step 5.1c: Assign the probability of the current dominant frame being selected to 0.9; set the index number of the current dominant frame to j, then use the formula get The probability of each of the remaining frames being selected is, where P i express The probability of the frame with index number i being selected, abs() is the absolute value function, and γ is the preset hyperparameter; Step 5.2c: Take a random number in the interval (0, 1) and traverse For each frame in , compare the probability of the currently traversed frame being selected with the size of the random number. If the probability of the currently traversed frame being selected is greater than the random number, the currently traversed frame is selected; After all frames in are traversed, all selected frames are obtained.

7. A method for generating black-box adversarial samples for a speech recognition system according to claim 6, Features In the step 5, when k cannot be divided by inter, an initial disturbance vector is randomly generated for each selected frame, the length of the initial disturbance vector is consistent with the length of the corresponding frame, the initial disturbance vector satisfies the Gaussian distribution, and the value range of each element is [-1, 1]; then each element in the initial disturbance vector is multiplied by 0.0001 as the final Gaussian disturbance value, and the final Gaussian disturbance value is added to the corresponding sampling value.

Images