An adversarial noise generation method, device, equipment and medium for voiceprint recognition

Abstract

This disclosure provides a method, apparatus, device, and medium for generating adversarial perturbations for voiceprint recognition. The adversarial perturbation generation method includes: acquiring a training voiceprint sample set; initializing an adversarial sample generation network; determining a target recognition object for each original voiceprint sample in the training voiceprint sample set; inputting the original voiceprint samples into the adversarial sample generation network to obtain adversarial voiceprint samples; inputting the adversarial voiceprint samples into a voiceprint recognition network to obtain a first recognition result vector; determining a sample loss function for the adversarial sample generation network based on the original voiceprint samples, the adversarial voiceprint samples, a first probability, and a second probability; training the adversarial sample generation network based on the sample loss function to generate adversarial perturbations. Embodiments of this disclosure can improve the efficiency of generating adversarial perturbations and also enhance the dominance and stealth of the adversarial perturbations. Embodiments of this disclosure can be applied to artificial intelligence, communication security, etc.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to methods, apparatuses, devices and media for generating adversarial disturbances for voiceprint recognition. Background Technology

[0002] Voiceprint recognition technology is increasingly being applied to more and more scenarios. As the accuracy of voiceprint recognition systems improves, adversarial example generation techniques are also developing. Adversarial examples are samples obtained by adding an adversarial perturbation to the input sample, used to attack the voiceprint recognition system and cause it to obtain incorrect recognition results. Using adversarial examples to attack voiceprint recognition systems can improve their anti-interference capabilities. To achieve better attack results, the requirements for adversarial perturbation generation techniques are also becoming more sophisticated.

[0003] In existing technologies, there are two main types of adversarial perturbation generation techniques. One type generates different adversarial perturbations for different input samples. This approach struggles to specify the target object of the adversarial sample (the adversarial sample is incorrectly identified), lacks practicality, and requires regenerating the adversarial perturbation for each input sample, resulting in significant time and computational resource consumption. The other approach generates universal adversarial perturbations that are effective for different input samples. This method primarily involves training a large-scale generative adversarial network (GAN) for adversarial discrimination and then using the GAN to generate perturbations. However, due to the large network size, training is challenging and time-consuming. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and medium for generating adversarial disturbances for voiceprint recognition, which can improve the efficiency of generating adversarial disturbances and enhance the concealment and dominance of adversarial disturbances.

[0005] According to one aspect of this disclosure, a method for countering disturbance generation is provided, comprising:

[0006] Obtain a training voiceprint sample set, which contains multiple original voiceprint samples;

[0007] Initialize the adversarial example generation network, which is a single-layer neural network, wherein the number of neurons is equal to the preset number of frames of the adversarial perturbation, and each neuron corresponds to one frame of the adversarial perturbation;

[0008] For each of the original voiceprint samples in the training voiceprint sample set, a target recognition object is determined, wherein the target recognition object is different from the real recognition object of the original voiceprint sample;

[0009] The original voiceprint sample is input into the adversarial sample generation network to obtain voiceprint adversarial samples;

[0010] The adversarial voiceprint sample is input into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector includes a first probability that the adversarial voiceprint sample is recognized as the target recognition object and a second probability that it is recognized as an object other than the target recognition object.

[0011] Based on the original voiceprint sample, the adversarial voiceprint sample, the first probability, and the second probability, the sample loss function of the adversarial sample generation network is determined, and the adversarial sample generation network is trained based on the sample loss function to generate the adversarial perturbation.

[0012] According to one aspect of this disclosure, an apparatus for combating disturbance generation is provided, comprising:

[0013] The first acquisition unit is used to acquire a training voiceprint sample set, which contains multiple original voiceprint samples.

[0014] An initialization unit is used to initialize an adversarial sample generation network, which is a single-layer neural network, wherein the number of neurons is equal to the preset number of frames of the adversarial perturbation, and each neuron corresponds to one frame of the adversarial perturbation;

[0015] The first determining unit is used to determine a target recognition object for each of the original voiceprint samples in the training voiceprint sample set, wherein the target recognition object is different from the real recognition object of the original voiceprint sample.

[0016] The first input unit is used to input the original voiceprint sample into the adversarial sample generation network to obtain voiceprint adversarial samples.

[0017] The second input unit is used to input the adversarial voiceprint sample into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector includes a first probability that the adversarial voiceprint sample is recognized as the target recognition object and a second probability that it is recognized as an object other than the target recognition object.

[0018] The training unit is used to determine the sample loss function of the adversarial sample generation network based on the original voiceprint sample, the adversarial sample, the first probability, and the second probability, and to train the adversarial sample generation network based on the sample loss function to generate the adversarial perturbation.

[0019] Optionally, the first acquisition unit is specifically used for:

[0020] The training voiceprint sample set is randomly sampled from the overall voiceprint data set;

[0021] The anti-disturbance generation device further includes:

[0022] The second acquisition unit is used to randomly acquire a test voiceprint sample set from the total voiceprint sample set, wherein the test voiceprint sample set contains multiple test voiceprint samples.

[0023] The third input unit is used to input the test voiceprint sample into the adversarial sample generation network to obtain the test adversarial sample.

[0024] The fourth input unit is used to input the test adversarial sample into the voiceprint recognition network to obtain a second recognition result vector, input the test voiceprint sample into the voiceprint recognition network to obtain a third recognition result vector, and input the adversarial perturbation into the voiceprint recognition network to obtain a fourth recognition result vector.

[0025] The second determining unit is used to determine a first correlation based on the second identification result vector and the third identification result vector;

[0026] The third determining unit is used to determine the second correlation based on the second identification result vector and the fourth identification result vector;

[0027] The fourth determining unit is used to compare the first correlation with the second correlation to determine the degree of influence of the anti-disturbance on the test voiceprint sample.

[0028] Optionally, the training unit is specifically used for:

[0029] Based on the first probability and the second probability, the attack success rate of the adversarial sample being identified as the target object is determined;

[0030] Based on the original voiceprint sample and the voiceprint adversarial sample, the perturbation concealment value of the voiceprint adversarial sample is determined;

[0031] The sample loss function is determined based on the attack success rate and the perturbation concealment value.

[0032] Optionally, the training unit is specifically used for:

[0033] The attack success rate is obtained by subtracting the maximum value of the first probability from the second probability.

[0034] Obtain a first threshold and adjust the attack success rate according to the first threshold. If the attack success rate is greater than or equal to the first threshold, use the first threshold as the attack success rate.

[0035] Optionally, the training unit is specifically used for:

[0036] The l2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample is calculated as the perturbation concealment value.

[0037] Optionally, the training unit is specifically used for:

[0038] Obtain the concealment coefficient;

[0039] The sample loss function is obtained by subtracting the product of the concealment coefficient and the perturbation concealment value from the attack success rate.

[0040] Optionally, the preset number of frames for the adversarial perturbation is 3200 frames; the number of neurons in the adversarial sample generation network is 3200.

[0041] Optionally, the first input unit is specifically used for:

[0042] If the number of frames of the original voiceprint sample is greater than the number of neurons in the adversarial sample generation network, the neurons in the adversarial sample generation network are copied and added to the adversarial sample generation network;

[0043] If the number of frames of the original voiceprint sample is less than the number of neurons in the adversarial sample generation network, the extra neurons in the adversarial sample generation network are deleted to ensure that each frame of the original voiceprint sample corresponds to one neuron in the adversarial sample generation network.

[0044] If the number of frames of the original voiceprint sample is equal to the number of neurons in the adversarial sample generation network, the original voiceprint sample is input into the adversarial sample generation network.

[0045] According to one aspect of this disclosure, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the anti-disturbance generation method as described above.

[0046] According to one aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program that, when executed by a processor, implements the anti-disturbance generation method as described above.

[0047] According to one aspect of this disclosure, a computer program product is provided, the computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the anti-disturbance generation method as described above.

[0048] In this embodiment, adversarial examples are generated using an adversarial example generation network (APN). Unlike existing large-scale generative adversarial networks (GANs), the APN in this embodiment is a single-layer neural network with the number of neurons equal to the preset number of adversarial perturbation frames. Each neuron corresponds to one frame of the adversarial perturbation, and the neuron's parameters directly correspond to the adversarial perturbation. In other words, the APN itself is an adversarial perturbation. When the original voiceprint sample is input into the APN, one frame of the original voiceprint sample enters a neuron, and a frame of the corresponding adversarial perturbation is superimposed on it. Therefore, the voiceprint adversarial examples output by the APN are the result of superimposing one frame of adversarial perturbation on each frame of the original voiceprint sample. Compared to large-scale GANs, this embodiment uses a single-layer neural network to directly represent the adversarial perturbation, reducing training difficulty and improving training efficiency. This embodiment also specifies the incorrect object that the adversarial example should be identified as by using a target recognition object. After inputting the voiceprint adversarial example into the voiceprint recognition network, a first probability of the voiceprint adversarial example being identified as the target recognition object and a second probability of being identified as other objects are obtained. Based on the original voiceprint samples, adversarial voiceprint samples, and the first and second probabilities, a sample loss function is determined. The original and adversarial voiceprint samples are used to evaluate the stealth of the adversarial perturbation; higher stealth perturbations are less likely to be detected by the voiceprint recognition network, resulting in a more effective perturbation. The first and second probabilities are used to evaluate the success rate of recognizing the adversarial voiceprint samples as the target object; a higher success rate indicates greater dominance of the adversarial perturbation, which can guide the recognition direction of the voiceprint recognition network. Therefore, training the adversarial sample generation network based on the sample loss function can improve the stealth and dominance of the adversarial perturbation.

[0049] Other features and advantages of this disclosure will be set forth in the following description and will be apparent in part from the description or may be learned by practicing the disclosure. The objectives and other advantages of this disclosure may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0050] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0051] Figure 1 This is a system architecture diagram of an adversarial perturbation generation method for voiceprint recognition provided in an embodiment of this disclosure;

[0052] Figure 2 This is a flowchart of an embodiment of the adversarial perturbation generation method for voiceprint recognition in this disclosure;

[0053] Figures 3A-3B This is a schematic diagram of the structure of an adversarial sample generation network;

[0054] Figure 4 This is a schematic diagram of how the original voiceprint sample is processed by an adversarial sample generation network to generate adversarial voiceprint samples.

[0055] Figure 5 yes Figure 2 A detailed flowchart of step 240;

[0056] Figure 6 It is a flowchart of copying and cutting neurons in the adversarial sample generation network based on the number of frames of the original voiceprint sample;

[0057] Figure 7 This is a schematic diagram illustrating the cutting of neurons in an adversarial sample generation network;

[0058] Figures 8A-8C This is a schematic diagram illustrating the specific process of copying and cutting neurons in an adversarial sample generation network.

[0059] Figure 9 yes Figure 2 A detailed flowchart of step 260;

[0060] Figure 10 yes Figure 9 A detailed flowchart of step 910;

[0061] Figure 11 yes Figure 9 A detailed flowchart of step 930;

[0062] Figure 12 Is Figure 2 After step 260, add a specific flowchart of the process for evaluating the dominance of adversarial perturbations on the overall set of voiceprint samples;

[0063] Figure 13 This is a schematic diagram illustrating the process of evaluating the dominance of adversarial perturbations on the overall set of voiceprint samples;

[0064] Figure 14 This is an exemplary implementation flowchart of the present disclosure;

[0065] Figure 15 This is a structural block diagram of the anti-disturbance generation device according to an embodiment of the present disclosure;

[0066] Figure 16 According to embodiments of this disclosure Figure 2 The terminal structure diagram of the anti-disturbance generation method is shown;

[0067] Figure 17 According to embodiments of this disclosure Figure 2The server architecture diagram shown is for the method of generating adversarial disturbances. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0069] Before providing a further detailed description of the embodiments of this disclosure, the terms and concepts used in these embodiments are explained, and they are subject to the following interpretations:

[0070] Neural networks: A neural network is a machine learning technique that simulates the neural network of the human brain in an attempt to achieve artificial intelligence-like capabilities. A neural network contains an input layer, an output layer, and hidden layers. The input layer receives data, the output layer outputs results, and the hidden layers perform computations. The neurons in a neural network are connected by connections between different layers. Each connection corresponds to a weight. The training algorithm for a neural network aims to optimize the values ​​of these weights to achieve the best predictive performance for the entire network.

[0071] Voiceprint recognition: Voiceprint recognition uses algorithms and neural network models to enable machines to identify the voices of different people from speech signals and then pass them on to other systems for use.

[0072] Adversarial examples: Adding perturbations that are imperceptible to the human eye to the original samples (such perturbations do not affect human recognition, but can easily fool the model), causing the machine to make incorrect judgments.

[0073] Adversarial perturbation: Perturbations that are imperceptible to the human eye and are superimposed on the original sample to generate adversarial examples.

[0074] L2 norm: refers to the square root of the sum of the squares of the elements of a vector.

[0075] Perceptual evaluation of speech quality (PESQ): The PESQ algorithm objectively evaluates speech quality based on the difference between the source speech signal and the distorted signal. Its evaluation results reflect the subjective perception of speech quality by the human auditory system, making it more aligned with practical application needs. PESQ objectively evaluates speech quality from dimensions such as the energy of the source speech signal, the energy of the distorted signal, time-domain distortion, frequency-domain distortion, and additive noise.

[0076] Pearson correlation coefficient (PCC): Used to measure the linear correlation between two variables X and Y, with a range between -1 and 1.

[0077] Figure 1 This is a system architecture diagram of the anti-disturbance generation method applied according to embodiments of the present disclosure. It includes: server 110, gateway 120, Internet 130, and terminal 140.

[0078] Server 110 refers to a computer system designed to resist disturbances. Compared to terminal 140, server 110 has higher requirements in terms of stability, security, and performance. Server 110 can be a single high-performance computer in a network platform, or a combination of multiple high-performance computers (such as virtual machines). Server 110 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0079] Gateway 120, also known as an internetwork connector or protocol converter, is a computer system or device that enables network interconnection at the transport layer and acts as a translator. It bridges the gap between two systems using different communication protocols, data formats, languages, or even completely different architectures. Gateway 120 also provides filtering and security functions. Messages sent from terminal 140 to server 110 are forwarded to the corresponding server 110 via gateway 120. Messages sent from server 110 to terminal 140 are also forwarded to the corresponding terminal 140 via gateway 120.

[0080] Terminal 140 is a device used to view original voiceprint samples, initiate training for adversarial sample generation networks, and observe the effects of adversarial sample attacks. It can take various forms, including desktop computers, laptops, PDAs (personal digital assistants), mobile phones, in-vehicle terminals, home theater terminals, and dedicated terminals. Furthermore, it can be a single device or a collection of multiple devices. For example, multiple devices can be connected via a local area network, sharing a single display device to work collaboratively, forming a single terminal 140. Terminal 140 can also communicate with the Internet 130 via wired or wireless means to exchange data.

[0081] According to one embodiment of this disclosure, an adversarial perturbation generation method for voiceprint recognition is provided.

[0082] Adversarial perturbations are noise added to voiceprint samples, generating adversarial examples used to mislead the voiceprint recognition model's results. For example, if the sound in a voiceprint sample belongs to object A, after adding an adversarial perturbation, the voiceprint recognition model might identify the sound as object B. The purpose of using adversarial perturbations is to improve the robustness of the voiceprint recognition model, ensuring accuracy even when faced with auditory interference. Therefore, generating a subtle adversarial perturbation with a high success rate is crucial for improving the robustness of the voiceprint recognition model.

[0083] like Figure 2 As shown, according to one embodiment of this disclosure, the adversarial perturbation generation method for voiceprint recognition includes:

[0084] Step 210: Obtain the training voiceprint sample set, which contains multiple original voiceprint samples.

[0085] Step 220: Initialize the adversarial example generation network, which is a single-layer neural network;

[0086] Step 230: For each original voiceprint sample in the training voiceprint sample set, determine the target recognition object;

[0087] Step 240: Input the original voiceprint samples into the adversarial sample generation network to obtain voiceprint adversarial samples;

[0088] Step 250: Input the adversarial voiceprint sample into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector contains a first probability that the adversarial voiceprint sample is recognized as the target recognition object, and a second probability that it is recognized as an object other than the target recognition object.

[0089] Step 260: Based on the original voiceprint samples, adversarial voiceprint samples, the first probability and the second probability, determine the sample loss function of the adversarial sample generation network, train the adversarial sample generation network based on the sample loss function, and generate adversarial perturbations.

[0090] In step 210, the training voiceprint sample set is a set of samples used to train the adversarial example generation network, which contains multiple original voiceprint samples. Since the adversarial perturbation in this embodiment aims to make the voiceprint recognition network identify the original voiceprint sample as another object as much as possible, and uses the difference between the real object and the target object to determine the success of the attack, the voiceprint features in the original voiceprint samples are not important for determining the success rate of the adversarial perturbation attack. Furthermore, to ensure that the adversarial perturbation generated in this embodiment can be applied to various voiceprint samples and exhibit concealment on various voiceprint samples, the original voiceprint samples in the training voiceprint sample set can contain arbitrary voiceprint features, without requiring sample and feature preprocessing for training the adversarial example generation network.

[0091] The training voiceprint sample set can be obtained by either accessing the training voiceprint sample set stored locally on server 110 or by accessing the open-source training voiceprint sample set stored in the cloud.

[0092] Since the adversarial example generation network in this embodiment is a single-layer neural network with a relatively simple structure, it is not necessary to use a large training voiceprint sample set. In one embodiment, obtaining the training voiceprint sample set includes: randomly sampling the training voiceprint sample set from the total voiceprint data set.

[0093] The voiceprint dataset contains a large number of voiceprint samples. A training voiceprint sample set is randomly sampled from this dataset. The number of original voiceprint samples in the training voiceprint sample set is far less than the number of voiceprint samples in the dataset. Adversarial perturbations trained on the training voiceprint sample set can achieve a high attack success rate and stealth when transferred to the voiceprint data in the dataset.

[0094] The advantages of randomly sampling the training voiceprint sample set from the total voiceprint data set are: the training dataset is smaller, which improves the training speed and reduces the computational cost during the training process.

[0095] In step 220, the adversarial example generation network is initialized. The adversarial example generation network is a single-layer neural network, such as... Figure 3A As shown, the adversarial example generation network consists of a single hidden layer. The input sample passes through this single hidden layer to directly obtain the output sample. When initializing the adversarial example generation network, an initial value needs to be assigned to the parameters of each neuron. A Gaussian distribution can be used to initialize the neuron parameters.

[0096] The number of neurons in a single hidden layer is equal to the preset number of frames for adversarial perturbation. Each neuron corresponds to one frame of adversarial perturbation, and the parameters of the neurons directly represent the adversarial perturbation. For example... Figure 3B As shown, each neuron in the hidden layer corresponds one-to-one with each frame of adversarial perturbation; therefore, each neuron represents one frame of adversarial perturbation. When the original speaker samples are input into the adversarial sample generation network, each frame input to a neuron results in the superposition of one frame of adversarial perturbation. Therefore, training the adversarial sample generation network is essentially the process of optimizing the adversarial perturbations.

[0097] The preset frame rate for counter-disturbance is determined by balancing the success rate and stealth of the disturbance. Generally, a higher preset frame rate results in a higher attack success rate, but poorer stealth. Therefore, a preset frame rate needs to be set to balance attack success rate and stealth to achieve the best counter-disturbance effect.

[0098] In one embodiment, the preset frame count for adversarial perturbation is 3200 frames, meaning the adversarial sample generation network has 3200 neurons. Comparative experiments show that a single-layer adversarial sample generation network with 3200 neurons can balance the attack success rate and perturbation stealth of adversarial perturbations. Increasing the number of neurons increases the attack success rate to some extent, but significantly reduces the perturbation stealth. Therefore, this embodiment can improve the attack success rate of adversarial perturbations while maintaining perturbation stealth. The network size of the 3200-neuron adversarial sample generation network is much smaller than that of existing adversarial generative networks, which can reduce overfitting of the generated adversarial perturbations and improve training efficiency.

[0099] In step 230, for each original voiceprint sample in the training voiceprint sample set, a target recognition object is determined. The target recognition object is an object different from the true recognition object of the original voiceprint sample. The target recognition object is the object that the original voiceprint sample is expected to be recognized as after adding adversarial perturbations; the true recognition object is the object to which the original voiceprint sample actually belongs. For example, for an original voiceprint sample, the true recognition object is object A, and the target recognition object is object B. That is, the adversarial perturbation superimposed on the original voiceprint sample needs to be optimized in the direction of making the original voiceprint sample recognize object B. Determining the target recognition object for the original voiceprint sample can be expressed as Formula 1:

[0100] (Formula 1).

[0101] In Formula 1, This represents the original voiceprint sample; Indicates an attack disturbance; =x', where x' represents a voiceprint adversarial sample; This represents a voiceprint recognition model; This represents the first recognition result vector obtained by inputting adversarial voiceprint samples into the voiceprint recognition model; This represents the target object to be identified. Formula 1 defines the identification result obtained from the first identification result vector based on the voiceprint recognition model as the target object to be identified. If you want the target object to be a specified object, then you will directly use... Set it to this object; if the target identification object does not specifically refer to any object, then... Set to any object other than the actual identified object.

[0102] The purpose of setting a target object for recognition is to enhance the dominance of adversarial perturbations. This allows the generated adversarial perturbations to interfere with the voiceprint recognition network's recognition results in the direction of the target object, thus strengthening the dominance of the adversarial perturbations. In many practical applications, it is necessary to identify input voiceprint samples as target objects. Therefore, improving the dominance of adversarial perturbations during optimization can further enhance their practicality in real-world applications.

[0103] In step 240, the original voiceprint samples are input into the adversarial sample generation network to obtain voiceprint adversarial samples. Voiceprint adversarial samples are original voiceprint samples superimposed with adversarial perturbations. Each frame of the original voiceprint sample is input into the adversarial sample generation network, and a frame of adversarial perturbation is superimposed to obtain one frame of the voiceprint adversarial sample. For example... Figure 4 As shown, frame A in the original voiceprint sample is input into neuron a in the adversarial sample generation network to obtain frame A' in the voiceprint adversarial sample. This process is repeated to obtain the original voiceprint sample superimposed with adversarial perturbation as the voiceprint adversarial sample.

[0104] If the number of frames in the original speaker samples is different from the number of frames in the adversarial perturbation, then it is impossible to input each frame of the original speaker samples into a neuron of the adversarial sample generation network. Therefore, in one embodiment, as... Figure 5 As shown, the original voiceprint samples are input into the adversarial sample generation network, including:

[0105] Step 510: If the number of frames of the original voiceprint sample is greater than the number of neurons in the adversarial sample generation network, duplicate the neurons of the adversarial sample generation network and add them to the adversarial sample generation network.

[0106] Step 520: If the number of frames of the original voiceprint sample is less than the number of neurons in the adversarial sample generation network, delete the extra neurons in the adversarial sample generation network to ensure that each frame of the original voiceprint sample corresponds to one neuron in the adversarial sample generation network.

[0107] Step 530: If the number of frames of the original voiceprint sample is equal to the number of neurons in the adversarial sample generation network, input the original voiceprint sample into the adversarial sample generation network.

[0108] The purpose of this embodiment is to adjust the width of the adversarial sample generation network to be equal to the number of frames of the original voiceprint sample, so as to ensure that each frame of the original voiceprint sample can be superimposed with an adversarial perturbation.

[0109] The specific process of this embodiment is as follows: Figure 6As shown, the original voiceprint sample is first obtained, and the number of frames of the original voiceprint sample is compared with the number of neurons in the adversarial sample generation network. If the number of frames of the original voiceprint sample is equal to the number of neurons in the adversarial sample generation network, the original voiceprint sample is directly input into the adversarial sample generation network.

[0110] If the number of frames in the original speakerprint sample is less than the number of neurons in the adversarial example generation network, the extra neurons in the adversarial example generation network are deleted. For example... Figure 7 As shown, neurons e, f, and g in the adversarial sample generation network are additional neurons compared to the original voiceprint sample. Therefore, neurons e, f, and g in the adversarial sample generation network are deleted, and the original voiceprint sample is input into the adversarial sample generation network after deleting the redundant neurons.

[0111] If the number of frames in the original speakerprint sample is greater than the number of neurons in the adversarial sample generation network, then the neurons in the adversarial sample generation network are duplicated and added to the adversarial sample generation network. For example... Figure 8A As shown, compared to the neurons in the adversarial sample generation network, the original speaker samples have 3 more frames: E, F, and G. The neurons in the adversarial sample generation network are copied and added to the network, as shown below. Figure 8B As shown, after adding the duplicated neurons to the adversarial sample generation network, the input neurons are neuron a for frame E, neuron b for frame F, and neuron c for frame G.

[0112] After adding the duplicated neurons to the adversarial example generation network (APGN), the number of frames in the original speaker samples is compared again with the number of neurons in the APGN. If the number of frames in the original speaker samples is still greater than the number of neurons in the APGN, then the neurons are duplicated again and added to the APGN. If the number of frames in the original speaker samples is now equal to the number of neurons, then the original speaker samples are input into the APGN. If the number of frames in the original speaker samples is now less than the number of neurons, then the extra neurons in the APGN are deleted. Figure 8C As shown, after adding the duplicated neurons to the adversarial example generation network, there are redundant neurons. Therefore, the redundant neurons are deleted.

[0113] The advantage of this embodiment is that the number of neurons in the adversarial sample generation network can be adjusted according to the number of frames of the original voiceprint sample, so that adversarial perturbations can be superimposed on each frame of the original voiceprint sample, thereby improving the flexibility of generating adversarial perturbations.

[0114] In step 250, the adversarial voiceprint sample is input into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector can be a logits vector or a normalized logits vector. The first recognition result vector contains a first probability that the adversarial voiceprint sample is identified as the target object, and a second probability that it is identified as an object other than the target object. For example, the first recognition result vector is {object A: 1.85, object B: 2, object C: 0.5}, where object B is the target object with a first probability of 2; objects A and C are other objects, and the second probability that the adversarial voiceprint sample is identified as object A is 1.85, and the second probability that it is identified as object C is 0.5.

[0115] The purpose of obtaining the probability of being identified as different objects is to effectively assess the impact of adversarial perturbations on voiceprint recognition. If the difference between the first probability and the highest probability in the second probability is small, it indicates that the adversarial perturbation has a small impact on voiceprint recognition, and the attack success rate is low. Therefore, the attack success rate can be improved by widening the probability gap between the first and second probabilities.

[0116] In step 260, the sample loss function of the adversarial sample generation network is determined based on the original voiceprint sample, the adversarial voiceprint sample, the first probability, and the second probability. In order to optimize the adversarial perturbation in a more dominant and less detectable direction, the loss function considers both the perturbation success rate and the perturbation concealment.

[0117] In one embodiment, such as Figure 9 As shown, based on the original voiceprint samples, adversarial voiceprint samples, the first probability, and the second probability, the sample loss function of the adversarial sample generation network is determined, including:

[0118] Step 910: Based on the first probability and the second probability, determine the attack success rate of the adversarial voiceprint sample being identified as the target object;

[0119] Step 920: Based on the original voiceprint sample and the voiceprint adversarial sample, determine the perturbation concealment value of the voiceprint adversarial sample;

[0120] Step 930: Determine the sample loss function based on the attack success rate and the perturbation concealment value.

[0121] In this embodiment, the success rate is determined by using a first probability that the adversarial voiceprint sample is identified as the target object and a second probability that it is identified as an object other than the target object. If the difference between the first probability and the second probability is large, it indicates that the adversarial interference is relatively successful, and the voiceprint recognition system is being misled into identifying a completely wrong direction.

[0122] In one embodiment, such as Figure 10As shown, based on the first probability and the second probability, the attack success rate of voiceprint adversarial examples being identified as target objects is determined, including:

[0123] Step 1010: Subtract the maximum value between the first probability and the second probability to obtain the attack success rate;

[0124] Step 1020: Obtain the first threshold and adjust the attack success rate according to the first threshold. If the attack success rate is greater than or equal to the first threshold, use the first threshold as the attack success rate.

[0125] In this embodiment, the process of determining the attack success rate is as shown in Formula 2:

[0126] (Formula 2).

[0127] In Formula 2, Represents the original voiceprint sample Superimposed with anti-disturbance It was then identified as the target object. The success rate; This represents the first probability that the adversarial voiceprint sample is identified as the target object. This represents the second probability that the adversarial voiceprint sample is identified as an object other than the target object. Since there may be multiple other objects that the adversarial voiceprint sample is identified as, this is because... This refers to the maximum value among the second probabilities of being identified as another object. For example, the second probabilities might include: {Object A: 0.95, Object B: 0.24, Object C: 0.1}, then the maximum value among the second probabilities is 0.95. By subtracting the maximum value among the first and second probabilities, we obtain the attack success rate of the adversarial sample identifying the target object. For example, if the first probability is 1.5, based on the second probability in the example above, the attack success rate is 1.5 - 0.95 = 0.55.

[0128] Formula 2 The first threshold is used to prevent the adversarial sample generation network from overfitting to the original voiceprint sample. If the difference between the maximum value of the first probability and the maximum value of the second probability is large, directly using the difference as the attack success rate may lead to the optimized adversarial perturbation overfitting to the original voiceprint sample, while the interference effect on other original voiceprint samples is poor. Therefore, when the difference between the maximum value of the first probability and the maximum value of the second probability is greater than or equal to the first threshold, the first threshold is directly used as the attack success rate. For example, if the attack success rate is 1.5 when the first probability is subtracted from the maximum value of the second probability, and the first threshold is 1, since the attack success rate is greater than the first threshold, the first threshold is used as the attack success rate.

[0129] The advantage of this embodiment is that it reduces the overfitting of the adversarial sample generation network to the original voiceprint samples and improves the transferability of adversarial perturbations to other original voiceprint samples.

[0130] By using the original voiceprint sample and the adversarial voiceprint sample, the perturbation concealment value of the adversarial voiceprint sample is determined. If the difference between the original voiceprint sample and the adversarial voiceprint sample is small, it means that the adversarial perturbation superimposed on the original voiceprint sample is not easily detected and has good concealment.

[0131] In one embodiment, determining the perturbation concealment value of the adversarial voiceprint sample based on the original voiceprint sample and the adversarial voiceprint sample includes: calculating the L2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample as the perturbation concealment value. Since the L2 norm represents the distance between two vectors, calculating the L2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample can represent the magnitude of the adversarial perturbation superimposed on the original voiceprint sample.

[0132] To determine the impact of the L2 norm on perturbation concealment, this embodiment employs multiple methods to evaluate perturbation concealment.

[0133] In one embodiment, the perceptual evaluation of speech quality (PESQ) can be used to assess the stealth of adversarial perturbations. The PESQ algorithm objectively evaluates adversarial samples based on the difference between the original speaker samples and the adversarial samples. Its value ranges from -0.5 to 4.5; a higher value indicates better stealth of the adversarial perturbation. Experiments have shown that reducing the L2 norm can effectively improve the PESQ value.

[0134] In another embodiment, the stealth of the adversarial perturbation is evaluated by calculating the signal-to-noise ratio between the original voiceprint sample and the adversarial perturbation. The specific calculation formula is shown in Formula 3:

[0135] (Formula 3).

[0136] In Formula 3, This represents the L2 norm of the original voiceprint sample. The L2 norm of the adversarial perturbation is represented by Equation 3. It can be seen from Equation 3 that the smaller the L2 norm of the adversarial perturbation, the greater the signal-to-noise ratio between the original voiceprint sample and the adversarial perturbation, which means the higher the concealment of the adversarial perturbation.

[0137] Calculating the L2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample is equivalent to calculating the L2 norm of the adversarial perturbation. The smaller the L2 norm of the adversarial perturbation, the greater its concealment. For example, if the original voiceprint sample is [0,1,1.3,1.5,0.9,1.7], and the adversarial voiceprint sample is [1.1,1.4,2.1,0.4,1.3], the adversarial perturbation superimposed on the original voiceprint sample is [1.1,0.1,0.6,-0.5,-0.4], and the perturbation concealment value is [1.1,0.1,0.6,-0.5,-0.4]. =1.41.

[0138] The conclusion drawn from the evaluation of the correlation between the L2 norm and the concealment of disturbances is that the L2 norm can adequately represent the concealment against disturbances. Therefore, the advantage of using the L2 norm against disturbances as the disturbance concealment value is that a disturbance concealment value that fully reflects the concealment can be obtained using a relatively simple calculation method, which improves the computational efficiency and the accuracy of obtaining the disturbance concealment value.

[0139] The sample loss function is determined by both the attack success rate and the perturbation concealment value. In one embodiment, such as... Figure 11 As shown, based on the attack success rate and the perturbation concealment value, the sample loss function is determined, including:

[0140] Step 1110: Obtain the concealment coefficient;

[0141] Step 1120: Subtract the product of the concealment coefficient and the perturbation concealment value from the attack success rate to obtain the sample loss function.

[0142] The calculation process in this embodiment is shown in Formula 4:

[0143] (Formula 4).

[0144] In Formula 4, Represents the sample loss function; Represents the original voiceprint sample Superimposed with anti-disturbance It was then identified as the target object. The success rate of the attack; Indicates based on original voiceprint samples Adversarial samples with voiceprints ) Determined counter-disturbance The disturbance concealment value; This refers to the concealment coefficient, a hyperparameter used to adjust the attack success rate and the concealment value of the perturbation. Since attack success rate and concealment are difficult to achieve simultaneously in adversarial perturbations, generally, higher concealment leads to a higher attack success rate. However, a highly effective adversarial perturbation needs to be difficult for the voiceprint recognition network to detect while maintaining a high attack success rate. Therefore, a hyperparameter is required. This aims to balance the requirements of attack success rate and stealth in real-world application scenarios as much as possible. If stealth is a higher requirement in real-world application scenarios, then... Increase the value, and vice versa. The value of is reduced to focus the sample loss function on the attack success rate.

[0145] Therefore, the advantage of calculating the sample loss function based on the concealment coefficient is that the attack success rate and the impact of perturbation concealment value on the training of the adversarial example generation network can be adjusted by adjusting the concealment coefficient, thereby improving the flexibility and practicality of training the adversarial example generation network.

[0146] The advantage of the embodiment in steps 910-930 is that the sample loss function comprehensively considers the importance of assessing the attack success rate and the concealment of the perturbation, so that the optimized perturbation can have a high attack success rate and a high concealment.

[0147] After obtaining the sample loss function, an adversarial example generation network is trained based on it. Since each neuron in the adversarial example generation network corresponds to a frame of adversarial perturbation, independent adaptive learning rates are set for the parameters of different neurons. The training process specifically includes: calculating the current gradient with respect to the sample loss function; calculating the first moment of the current gradient (the average of past gradients and the current gradient); calculating the second moment of the current gradient (the average of the squares of past gradients and the squares of the current gradient); updating the learning rate of the parameters based on the first and second moments, and thus updating the parameters. Based on the above training process, by comprehensively considering the first and second moments of the current gradient, the updated learning rate can be calculated before updating the parameters, avoiding the gradient getting stuck in local minima.

[0148] After updating the parameters, they can be standardized to keep them within a specified range, thus maintaining their stability and facilitating model convergence.

[0149] After training, the neuron parameters of a single-layer adversarial example generation network constitute an adversarial perturbation that is both dominant and covert.

[0150] In one of the foregoing embodiments, the training voiceprint sample set is randomly sampled from the overall voiceprint data set. Since the adversarial perturbations in this embodiment are dominant, they can influence the recognition results of the voiceprint recognition network. To ensure that the adversarial perturbations optimized using a small-scale dataset maintain strong dominance when transferred to the complete voiceprint dataset, the perturbation effect needs to be evaluated after generating the perturbations. Therefore, in one embodiment, as... Figure 12 As shown, after training the adversarial example generation network based on the sample loss function and generating adversarial perturbations, the adversarial perturbation generation method also includes:

[0151] Step 1210: Randomly obtain a test voiceprint sample set from the total voiceprint sample set;

[0152] Step 1220: Input the test voiceprint sample into the adversarial sample generation network to obtain the test adversarial sample;

[0153] Step 1230: Input the test adversarial sample into the voiceprint recognition network to obtain the second recognition result vector; input the test voiceprint sample into the voiceprint recognition network to obtain the third recognition result vector; input the adversarial perturbation into the voiceprint recognition network to obtain the fourth recognition result vector.

[0154] Step 1240: Determine the first correlation based on the second and third identification result vectors;

[0155] Step 1250: Determine the second correlation based on the second identification result vector and the fourth identification result vector;

[0156] Step 1260: Compare the first correlation with the second correlation to determine the degree of influence of the adversarial disturbance on the test voiceprint sample.

[0157] In this embodiment, the test voiceprint sample set contains multiple test voiceprint samples. The test voiceprint samples are randomly obtained from the total voiceprint sample set, and the test results can represent the test effect of the entire voiceprint sample set.

[0158] like Figure 13 As shown, test voiceprint samples are input into an adversarial example generation network to obtain test adversarial samples. These test adversarial samples are generated by superimposing adversarial perturbations onto the test voiceprint samples; simultaneously, the adversarial perturbations are extracted from the adversarial example generation network. The test adversarial samples are input into a voiceprint recognition network to obtain a second recognition result vector; the test voiceprint samples are input into the voiceprint recognition network to obtain a third recognition result vector; and the adversarial perturbations are input into the voiceprint recognition network to obtain a fourth recognition result vector. The second, third, and fourth recognition result vectors all contain a first probability that the voiceprint recognition network identifies the target object and a second probability that it identifies it as another object.

[0159] Based on the second and third identification result vectors, a first correlation is determined. The first correlation indicates the similarity between the second and third identification result vectors. Based on the second and third identification result vectors, a second correlation is determined. The second correlation indicates the similarity between the second and third identification result vectors. The first and second correlations can be calculated using the Pearson correlation coefficient (PCC) algorithm.

[0160] After obtaining the first and second correlations, the degree of influence of adversarial perturbation on the test voiceprint samples is determined by comparing the first and second correlations. This assesses the dominance of adversarial perturbation on the test voiceprint samples and determines whether further optimization of the adversarial perturbation is needed based on the degree of influence. If the first correlation is greater than the second correlation, it means that the third and fourth recognition result vectors are closer to the second recognition result vector. This indicates that the adversarial perturbation superimposed on the test voiceprint samples has a relatively small perturbation effect on the voiceprint recognition network and cannot accurately identify the target. Conversely, if the first correlation is less than the second correlation, it means that the fourth recognition result vector is closer to the second recognition result vector. The voiceprint recognition network receives strong interference from the adversarial perturbation, making the recognition result of the test adversarial samples closer to the recognition result of the adversarial perturbation, i.e., tending towards the target object. This indicates that the adversarial perturbation application has a strong dominance on the test voiceprint samples.

[0161] Therefore, through the above process, the dominance of the adversarial perturbation after being transferred to the voiceprint sample set can be evaluated. If the evaluation results show that the dominance of the adversarial perturbation after being transferred to the voiceprint sample set is poor, then steps 210-260 need to be repeated to further optimize the adversarial perturbation. Conversely, if the dominance is strong, then training is stopped and an adversarial perturbation applicable to the voiceprint sample set is generated.

[0162] This embodiment evaluates the dominance of adversarial perturbations by transferring the generated perturbations to a set of voiceprint samples. If the dominance is poor, further optimization of the perturbations is implemented. Therefore, the advantage of this embodiment is that it further verifies the dominance of the perturbations, ensuring that adversarial perturbations generated using small-scale samples can also have good perturbation effects on large-scale samples.

[0163] In summary, the specific process of the adversarial perturbation generation method for voiceprint recognition in this disclosure embodiment can be exemplarily summarized as follows: Figure 14As shown, the process includes: randomly sampling a training voiceprint sample set from the overall voiceprint data set; initializing the adversarial example generation network; inputting the original voiceprint samples into the adversarial example generation network, and copying and cutting neurons in the adversarial example generation network according to the frame number of the original voiceprint samples to ensure that each frame corresponds to one neuron; inputting voiceprint adversarial samples into the adversarial example generation network, and generating a first recognition result vector based on the voiceprint adversarial samples; determining the current gradient based on the first recognition result vector; updating the parameters of the adversarial example generation network according to the current gradient and optimizing the adversarial perturbation; standardizing the adversarial perturbation; optimizing the adversarial perturbation for one original voiceprint sample, and then optimizing the adversarial perturbation for the next original voiceprint sample, until an adversarial perturbation with high dominance and concealment for the entire training voiceprint sample set is generated.

[0164] The apparatus and device according to embodiments of this disclosure will now be described.

[0165] It is understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated in this embodiment, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps.

[0166] It should be noted that in various specific embodiments of this application, when processing is required based on data related to the characteristics of the target virtual machine, such as target virtual machine attribute information or attribute information sets, the permission or consent of the target virtual machine will be obtained first. Furthermore, the collection, use, and processing of this data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when embodiments of this application need to obtain target virtual machine attribute information, separate permission or consent from the target virtual machine will be obtained through pop-up windows or redirection to a confirmation page. Only after obtaining the separate permission or consent of the target virtual machine will the necessary target virtual machine-related data for the normal operation of the embodiments of this application be obtained.

[0167] Figure 15 This is a structural diagram of an anti-disturbance generation device 1500 provided in an embodiment of this disclosure. The anti-disturbance generation device 1500 includes:

[0168] The first acquisition unit 1510 is used to acquire a training voiceprint sample set, which contains multiple original voiceprint samples.

[0169] Initialization unit 1520 is used to initialize the adversarial example generation network. The adversarial example generation network is a single-layer neural network, in which the number of neurons is equal to the preset number of adversarial perturbation frames, and each neuron corresponds to one frame of adversarial perturbation.

[0170] The first determining unit 1530 is used to determine the target recognition object for each original voiceprint sample in the training voiceprint sample set. The target recognition object is different from the real recognition object of the original voiceprint sample.

[0171] The first input unit 1540 is used to input the original voiceprint sample into the adversarial sample generation network to obtain voiceprint adversarial samples.

[0172] The second input unit 1550 is used to input the adversarial voiceprint sample into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector includes a first probability that the adversarial voiceprint sample is recognized as the target recognition object and a second probability that it is recognized as an object other than the target recognition object.

[0173] Training unit 1560 is used to determine the sample loss function of the adversarial sample generation network based on the original voiceprint samples, adversarial voiceprint samples, the first probability, and the second probability, and to train the adversarial sample generation network based on the sample loss function to generate adversarial perturbations.

[0174] Optionally, the first acquisition unit 1510 is specifically used for:

[0175] A training voiceprint sample set is randomly sampled from the overall voiceprint data set;

[0176] The anti-disturbance generation device also includes:

[0177] The second acquisition unit (not shown) is used to randomly acquire a test voiceprint sample set from the total voiceprint sample set, which contains multiple test voiceprint samples.

[0178] The third input unit (not shown) is used to input the test voiceprint sample into the adversarial sample generation network to obtain the test adversarial sample.

[0179] The fourth input unit (not shown) is used to input the test adversarial sample into the voiceprint recognition network to obtain the second recognition result vector, input the test voiceprint sample into the voiceprint recognition network to obtain the third recognition result vector, and input the adversarial perturbation into the voiceprint recognition network to obtain the fourth recognition result vector.

[0180] The second determining unit (not shown) is used to determine the first correlation based on the second identification result vector and the third identification result vector;

[0181] The third determining unit (not shown) is used to determine the second correlation based on the second identification result vector and the fourth identification result vector;

[0182] The fourth determining unit (not shown) is used to compare the first correlation with the second correlation to determine the degree of influence of the anti-disturbance on the test voiceprint sample.

[0183] Optionally, training unit 1560 is specifically used for:

[0184] Based on the first probability and the second probability, the attack success rate of voiceprint adversarial sample being identified as the target object is determined;

[0185] Based on the original voiceprint samples and adversarial voiceprint samples, the perturbation concealment value of the adversarial voiceprint samples is determined.

[0186] The sample loss function is determined based on the attack success rate and the perturbation concealment value.

[0187] Optionally, training unit 1560 is specifically used for:

[0188] Subtract the maximum value between the first probability and the second probability to obtain the attack success rate;

[0189] Obtain a first threshold and adjust the attack success rate based on the first threshold. If the attack success rate is greater than or equal to the first threshold, use the first threshold as the attack success rate.

[0190] Optionally, training unit 1560 is specifically used for:

[0191] The l2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample is calculated as the perturbation concealment value.

[0192] Optionally, training unit 1560 is specifically used for:

[0193] Obtain the concealment coefficient;

[0194] The sample loss function is obtained by subtracting the product of the concealment coefficient and the perturbation concealment value from the attack success rate.

[0195] Optionally, the preset number of frames for adversarial perturbation is 3200 frames; the number of neurons in the adversarial sample generation network is 3200.

[0196] Optionally, the first input unit 1540 is specifically used for:

[0197] If the number of frames of the original voiceprint sample is greater than the number of neurons in the adversarial sample generation network, duplicate the neurons of the adversarial sample generation network and add them to the adversarial sample generation network.

[0198] If the number of frames in the original voiceprint sample is less than the number of neurons in the adversarial sample generation network, the extra neurons in the adversarial sample generation network are deleted to ensure that each frame of the original voiceprint sample corresponds to one neuron in the adversarial sample generation network.

[0199] If the number of frames of the original voiceprint sample is equal to the number of neurons in the adversarial sample generation network, then the original voiceprint sample is input into the adversarial sample generation network.

[0200] Reference Figure 16 , Figure 16 To implement the structural block diagram of a portion of the terminal 140 in this embodiment, the terminal 140 includes: a radio frequency (RF) circuit 1610, a memory 1615, an input unit 1630, a display unit 1640, a sensor 1650, an audio circuit 1660, a wireless fidelity (WiFi) module 1670, a processor 1680, and a power supply 1690, among other components. Those skilled in the art will understand that... Figure 16 The terminal structure shown does not constitute a limitation on mobile phones or computers and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0201] The RF circuit 1610 can be used to receive and transmit signals during information transmission or calls. In particular, it receives downlink information from the base station and processes it with the processor 1680; in addition, it transmits uplink data to the base station.

[0202] The memory 1615 can be used to store software programs and modules. The processor 1680 executes various terminal functions and data processing by running the software programs and modules stored in the memory 1615.

[0203] The input unit 1630 can be used to receive input numeric or character information, and to generate key signal inputs related to the terminal's settings and function control. Specifically, the input unit 1630 may include a touch panel 1631 and other input devices 1632.

[0204] The display unit 1640 can be used to display input or provided information, as well as various menus of the terminal. The display unit 1640 may include a display panel 1641.

[0205] Audio circuitry 1660, speaker 1661, and microphone 1662 provide an audio interface.

[0206] In this embodiment, the processor 1680 included in the terminal can execute the anti-disturbance generation method of the previous embodiment.

[0207] The terminal 140 in this disclosure includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. This invention can be applied to various scenarios, including but not limited to artificial intelligence and communication security.

[0208] Figure 17 This is a partial structural block diagram of a server 110 implementing an embodiment of the present disclosure. The server 110 can vary significantly due to different configurations or performance characteristics, and may include one or more central processing units (CPUs) 1722 (e.g., one or more processors) and memory 1732, and one or more storage media 1730 (e.g., one or more mass storage devices) for storing application programs 1742 or data 1744. The memory 1732 and storage media 1730 may be temporary or persistent storage. The program stored in the storage media 1730 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server 110. Furthermore, the CPU 1722 may be configured to communicate with the storage media 1730 and execute the series of instruction operations in the storage media 1730 on the server 110.

[0209] Server 110 may also include one or more power supplies 1726, one or more wired or wireless network interfaces 1750, one or more input / output interfaces 1758, and / or one or more operating systems 1741, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0210] The central processing unit 1722 in server 110 can be used to execute the anti-disturbance generation method of the present disclosure embodiments.

[0211] This disclosure also provides a computer-readable storage medium for storing program code for executing the anti-disturbance generation methods of the foregoing embodiments.

[0212] This disclosure also provides a computer program product comprising a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the aforementioned method for generating anti-disturbance data.

[0213] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0214] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0215] It should be understood that in the description of the embodiments disclosed herein, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0216] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0217] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0218] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0219] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server 110, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0220] It should also be understood that the various implementation methods provided in this disclosure can be combined arbitrarily to achieve different technical effects.

[0221] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A method for generating adversarial perturbations for voiceprint recognition, characterized in that, include: Obtain a training voiceprint sample set, which contains multiple original voiceprint samples; Initialize the adversarial example generation network, which is a single-layer neural network, wherein the number of neurons is equal to the preset number of frames of the adversarial perturbation, and each neuron corresponds to one frame of the adversarial perturbation; For each of the original voiceprint samples in the training voiceprint sample set, a target recognition object is determined, wherein the target recognition object is different from the real recognition object of the original voiceprint sample; The original voiceprint sample is input into the adversarial sample generation network to obtain voiceprint adversarial samples; The adversarial voiceprint sample is input into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector includes a first probability that the adversarial voiceprint sample is recognized as the target recognition object and a second probability that it is recognized as an object other than the target recognition object. Based on the first probability and the second probability, the attack success rate of the adversarial sample being identified as the target object is determined; Based on the original voiceprint sample and the voiceprint adversarial sample, the perturbation concealment value of the voiceprint adversarial sample is determined; Based on the attack success rate and the perturbation concealment value, the sample loss function of the adversarial sample generation network is determined, and the adversarial sample generation network is trained based on the sample loss function to generate the adversarial perturbation.

2. The method for generating anti-disturbance according to claim 1, characterized in that, The acquisition of the training voiceprint sample set includes: The training voiceprint sample set is randomly sampled from the overall voiceprint data set; After training the adversarial example generation network based on the sample loss function and generating the adversarial perturbation, the adversarial perturbation generation method further includes: A test voiceprint sample set is randomly obtained from the total voiceprint sample set, and the test voiceprint sample set contains multiple test voiceprint samples. The test voiceprint sample is input into the adversarial sample generation network to obtain the test adversarial sample; The test adversarial sample is input into the voiceprint recognition network to obtain a second recognition result vector. The test voiceprint sample is input into the voiceprint recognition network to obtain a third recognition result vector. The adversarial perturbation is input into the voiceprint recognition network to obtain a fourth recognition result vector. Based on the second identification result vector and the third identification result vector, a first correlation is determined; Based on the second identification result vector and the fourth identification result vector, a second correlation is determined; By comparing the first correlation with the second correlation, the degree of influence of the adversarial perturbation on the test voiceprint sample is determined.

3. The method for generating anti-disturbance according to claim 1, characterized in that, The step of determining the attack success rate of the adversarial sample being identified as the target object based on the first probability and the second probability includes: The attack success rate is obtained by subtracting the maximum value of the first probability from the second probability. Obtain a first threshold and adjust the attack success rate according to the first threshold. If the attack success rate is greater than or equal to the first threshold, use the first threshold as the attack success rate.

4. The method for generating anti-disturbance according to claim 1, characterized in that, Based on the original voiceprint sample and the adversarial voiceprint sample, the perturbation concealment value of the adversarial voiceprint sample is determined, including: The l2 norm of the difference between the original voiceprint sample and the adversarial voiceprint sample is calculated as the perturbation concealment value.

5. The method for generating anti-disturbance according to claim 1, characterized in that, Based on the attack success rate and the perturbation concealment value, the sample loss function of the adversarial sample generation network is determined, including: Obtain the concealment coefficient; The sample loss function is obtained by subtracting the product of the concealment coefficient and the perturbation concealment value from the attack success rate.

6. The method for generating anti-disturbance according to claim 1, characterized in that, The preset number of frames for the adversarial perturbation is 3200 frames; the number of neurons in the adversarial sample generation network is 3200.

7. The method for generating anti-disturbance according to claim 1, characterized in that, Inputting the original voiceprint sample into the adversarial sample generation network includes: If the number of frames of the original voiceprint sample is greater than the number of neurons in the adversarial sample generation network, the neurons in the adversarial sample generation network are copied and added to the adversarial sample generation network; If the number of frames of the original voiceprint sample is less than the number of neurons in the adversarial sample generation network, the extra neurons in the adversarial sample generation network are deleted to ensure that each frame of the original voiceprint sample corresponds to one neuron in the adversarial sample generation network. If the number of frames of the original voiceprint sample is equal to the number of neurons in the adversarial sample generation network, the original voiceprint sample is input into the adversarial sample generation network.

8. An adversarial disturbance generation device for voiceprint recognition, characterized in that, include: The first acquisition unit is used to acquire a training voiceprint sample set, which contains multiple original voiceprint samples. An initialization unit is used to initialize an adversarial sample generation network, which is a single-layer neural network, wherein the number of neurons is equal to the preset number of frames of the adversarial perturbation, and each neuron corresponds to one frame of the adversarial perturbation; The first determining unit is used to determine a target recognition object for each of the original voiceprint samples in the training voiceprint sample set, wherein the target recognition object is different from the real recognition object of the original voiceprint sample. The first input unit is used to input the original voiceprint sample into the adversarial sample generation network to obtain voiceprint adversarial samples. The second input unit is used to input the adversarial voiceprint sample into the voiceprint recognition network to obtain a first recognition result vector. The first recognition result vector includes a first probability that the adversarial voiceprint sample is recognized as the target recognition object and a second probability that it is recognized as an object other than the target recognition object. The training unit is configured to determine the attack success rate of the adversarial voiceprint sample being identified as the target object based on the first probability and the second probability; determine the perturbation concealment value of the adversarial voiceprint sample based on the original voiceprint sample and the adversarial voiceprint sample; determine the sample loss function of the adversarial sample generation network based on the attack success rate and the perturbation concealment value; train the adversarial sample generation network based on the sample loss function; and generate the adversarial perturbation.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the anti-disturbance generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the anti-disturbance generation method according to any one of claims 1 to 7.

11. A computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the anti-disturbance generation method according to any one of claims 1 to 7.

Images