A method for constructing adversarial samples for voiceprint recognition defense modules

Abstract

The present invention discloses a method for constructing adversarial samples for a voiceprint recognition defense module, including obtaining an initial adversarial perturbation, original audio, and an acoustic masking sound set; selecting an acoustic masking sound from the acoustic masking sound set that can minimize auditory distortion; and combining the obtained initial adversarial perturbation with the original audio to calculate an adversarial sample. The present invention innovatively proposes a method for constructing adversarial samples for a voiceprint recognition system defense module, utilizing a carefully crafted acoustic masking sound to minimize human auditory distortion, and superimposing it on the initialized adversarial perturbation for perturbation masking. Simultaneously, multiple representative defense modules are considered and integrated learning methods are used to fuse them with adaptive weights of a learnable system. Ultimately, an adversarial sample is generated that satisfies attack effectiveness, human imperceptibility, and adaptability.

Description

Technical Field

[0001] The present invention relates to the fields of voiceprint recognition and adversarial samples, and in particular to a method for constructing adversarial samples for a voiceprint recognition defense module. Background Art

[0002] In recent years, with the rapid development of deep learning technology, voiceprint recognition has been widely used in automatic speech recognition systems, such as Apple's Siri, Google's OK Google, WeChat's voiceprint lock, and HSBC's Voice ID, making it a popular biometric technology. Research indicates that the global voice biometrics market exceeded $1.1 billion in 2020 and is expected to reach $3.9 billion by 2026, demonstrating the broad development prospects of voice technology. However, while people enjoy the convenience brought by voiceprint recognition services, the vulnerability of deep learning to adversarial attacks poses a serious security threat. Recent research has demonstrated that voiceprint recognition systems based on deep neural networks can be easily deceived by superimposing subtle perturbations on innocent speech. This phenomenon has attracted widespread public attention and considerable research interest. To combat such attacks, recent research has proposed deploying defense modules in front of voiceprint recognition systems to eliminate destructive adversarial perturbations. However, previous work has examined whether adversarial perturbations can maintain both attack effectiveness and human imperceptibility in the presence of defense modules. The present invention discloses a method for constructing adversarial samples for voiceprint recognition defense modules. The method uses carefully crafted acoustic masking sounds to achieve imperceptibility and combines it with integrated learning technology to enable adversarial disturbances to effectively attack multiple different defense modules, ultimately achieving targeted attacks to deceive the voiceprint recognition system.

[0003] Adversarial example attacks target the invocation of neural network models. Attackers add malicious perturbations (also known as adversarial perturbations) to normal speech, which are imperceptible to humans. This causes the neural network model to make incorrect predictions about the perturbed samples (i.e., adversarial examples). Existing adversarial example attacks in the voiceprint field assume that the voiceprint recognition model does not implement any defense measures. This makes their adversarial examples easily destroyed by defense modules, rendering the attack ineffective, and their imperceptibility is poor. Summary of the Invention

[0004] The present invention proposes a method for constructing adversarial samples for voiceprint recognition defense modules, with the goals of attack effectiveness, adaptability, and human imperceptibility. By adopting an ensemble learning method for a sufficient number of voiceprint recognition defense modules and introducing a carefully crafted acoustic masking sound, it is possible to generate voice adversarial samples that can successfully bypass multiple voiceprint recognition system defense modules and are imperceptible to the human ear. Furthermore, attacks can be carried out on different voiceprint recognition models.

[0005] The present invention is achieved through the following technical solutions:

[0006] The present invention discloses a method for constructing an adversarial sample for a voiceprint recognition defense module, comprising the following steps:

[0007] Obtaining an initial adversarial perturbation, original audio, and acoustic masker set;

[0008] Select an acoustic masking sound from the acoustic masking sound set that can minimize the auditory distortion, and combine it with the obtained initial adversarial perturbation and the original audio to calculate the adversarial sample;

[0009] Set multiple alternative defense module types and hyperparameters to obtain multiple voiceprint recognition systems with different alternative defense modules;

[0010] The respective decision scores are further calculated through various voiceprint recognition systems with different alternative defense modules and the obtained adversarial samples;

[0011] The obtained decision scores are combined with learnable adaptive weights through ensemble learning methods to obtain the final loss function;

[0012] The final speech adversarial sample is generated by repeating the above optimization process a certain number of times through the projected gradient descent algorithm and the obtained loss function.

[0013] As a further improvement, the present invention selects an acoustic masking sound from the acoustic masking sound set that can minimize the auditory distortion, and combines it with the obtained initial adversarial perturbation and the original audio to calculate the adversarial sample, specifically:

[0014] (1) By observing the time-frequency spectra of various masking sounds in advance and calculating their PESQ to characterize their masking capabilities, an evaluation index is proposed. Where F(j) is the jth frequency bin corresponding to the selected masker, K is the total number of frequency bins, and M(i,j) is the corresponding filtered time-frequency spectrum of the masker.

[0015] (2) The value range of M-Sup is 0-1. The larger the value, the stronger the masking ability. Therefore, by calculating and comparing the M-Sup index of different masking sounds, the acoustic masking sound with the strongest masking ability is selected.

[0016] (3) Given SNR r1, calculate the scaling factor And according to the adversarial disturbance amplitude A δ and the amplitude A of the selected acoustic masker M Calculate the scaled adversarial perturbation

[0017] (4) Obtain the scaled acoustic masking sound according to the scaling factor Given the original audio x and SNR r2, and following the same process as above to get the adversarial sample Among them A r (i, j) is a function used to achieve the masking effect, which inserts audio j into audio i under a given SNR r to obtain the masked audio i = A r (i,j).

[0018] As a further improvement, the present invention provides multiple voiceprint recognition systems with different alternative defense modules by setting multiple types of alternative defense modules and hyperparameters, specifically:

[0019] (1) Based on the simple characteristics of no training and plug-and-play, the defense method based on signal processing is mainly considered as the defense module of the voiceprint recognition system. The defense methods are divided into three categories: directly destroying disturbances with very small amplitudes, filtering out adversarial disturbances in high and low frequency bands, and filtering out adversarial disturbances below the hearing threshold curve;

[0020] (2) By observing the relationship between the hyperparameter values ​​of each defense module and its defense strength in advance, and by observing its impact on the performance of the voiceprint recognition model itself, the critical point setting of the hyperparameters of each defense module is found, and finally a variety of alternative defense modules (F0, β0),…, (F n-1 ,β n-1 ) voiceprint recognition model Among them F i and β i is the type and corresponding hyperparameters of the i-th defense module.

[0021] As a further improvement, among the three types of defense methods described in the present invention, the first type mainly considers requantization and Mel filtering, whose principles are based on the use of lossy filters to directly destroy tiny adversarial disturbances; the second type mainly considers resampling and bandpass filtering, whose principles are based on the Nyquist-Shannon sampling theorem, which directly filters out or destroys adversarial disturbances in high and low frequency bands; the third type mainly considers psychoacoustic filtering, whose principle is based on the human ear hearing threshold curve of psychoacoustics, and realizes the defense function by directly filtering out adversarial disturbances below the threshold; a total of the five most representative methods are considered as alternative defense modules: requantization, Mel filtering, resampling, bandpass filtering, and psychoacoustic filtering.

[0022] As a further improvement, the present invention combines the obtained decision scores with the learnable adaptive weights through an ensemble learning method to obtain the final loss function, specifically:

[0023]

[0024]

[0025] Among them, x is the input speech sample, y t is the target label, A M is the masking function used to constrain the adversarial perturbation δ and the acoustic masker M, For the alternative defense module F i And the hyperparameter β i The corresponding loss function of the voiceprint recognition system is set, w i It is the learnable weight factor for integrating various voiceprint recognition systems.

[0026] As a further improvement, the present invention uses the projected gradient descent algorithm and the obtained loss function to iterate a certain number of times to repeat the above optimization process to generate the final speech adversarial sample, specifically:

[0027] 1) After masking the initial adversarial perturbation with the selected acoustic masking sound, it is combined with the original audio to obtain the adversarial sample Then input multiple F i In the voiceprint recognition system, the corresponding target score is obtained And the confidence κ is introduced to ensure the score is enough to exceed the preset threshold θ i Thus, it is successfully identified as the target user y t , so the following loss is calculated for each voiceprint recognition system:

[0028]

[0029] 2) Use ensemble learning methods for each loss to obtain a synthetic loss function where w i To integrate the learnable weight factors of various voiceprint recognition systems;

[0030] 3) Use the projected gradient descent method to update the adversarial perturbation δ and generate new adversarial samples

[0031]

[0032]

[0033] Until the early stopping condition is met, the final adversarial sample is generated.

[0034] The beneficial effects of the present invention are as follows:

[0035] The present invention proposes a method for constructing adversarial samples for the voiceprint recognition defense module. Existing adversarial sample generation techniques for voiceprint recognition systems do not consider whether the voiceprint recognition system has any defense modules or measures, and are easily destroyed by the defense modules, rendering the attack ineffective, and their imperceptibility is poor. The present invention innovatively proposes a method for constructing adversarial samples for the defense module of the voiceprint recognition system, using carefully crafted acoustic masking sounds to minimize the distortion of human hearing, and superimposing them on the initialized adversarial perturbations for perturbation masking. At the same time, multiple representative defense modules are considered, and an ensemble learning method is used to fuse them with the adaptive weights of the learnable system. Ultimately, adversarial samples that meet the requirements of attack effectiveness, human imperceptibility, and adaptability are generated.

[0036] In the evaluation of attack effectiveness, the average ASR of the present invention for the five defense modules on the three voiceprint models of d-vector, x-vector and ecapa-tdnn respectively reached 91.7%, 97.1% and 99.1%, which are 48.7%, 29.8% and 63.4% higher than the traditional PGD method, which fully demonstrates the effectiveness of the present invention on different systems.

[0037] In the imperceptibility evaluation, the adversarial samples generated by the present invention achieved SNRs of 32.04dB, 35.55dB, and 36.53dB on three voiceprint models, respectively, a 30% average improvement over PGD. Furthermore, the PESQs reached 4.06, 4.14, and 4.32, respectively, a 73% improvement over PGD. This fully demonstrates the strong imperceptibility of the adversarial samples generated by the present invention.

[0038] In the adaptive evaluation, the present invention tested attacking five defense modules on three voiceprint recognition models. After each hyperparameter reached a critical point, the present invention's ASR achieved 98.7%, 95.5%, 90.6%, 93.7%, and 94% accuracy on the five defense modules, respectively. This fully demonstrates the present invention's adaptability to different defense modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a framework flow chart of the present invention;

[0040] Figure 2 This is a schematic diagram of the integrated defense module;

[0041] Figure 3 Figure 3 is an attack performance diagram of the present invention under different defense modules and their different hyperparameters, including (a) attack performance against the requantization method, (b) attack performance against the resampling method, (c) attack performance against the bandwidth filtering method, (d) attack performance against the psychoacoustic filtering method, and (e) attack performance against the Mel filtering method. DETAILED DESCRIPTION

[0042] To make the purpose, technical solutions, and advantages of the examples of the present invention more clear, the technical solutions of the examples of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the examples of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0043] The present invention discloses a method for constructing adversarial samples for a voiceprint recognition defense module. Figure 1 The framework flow chart of the present invention is shown, which includes four parts, namely acoustic masking sound selection, defense module setting, defense module integration and adversarial perturbation optimization. First, a carefully crafted acoustic masking sound is selected to minimize the distortion of human hearing perception, and it is superimposed on the initialized adversarial perturbation for perturbation masking. The adversarial perturbation and the original audio are then combined to generate adversarial samples, which will be input into multiple voiceprint recognition systems with different defense modules. The types and hyperparameters of these defense modules are carefully considered and designed by the present invention. After these voiceprint recognition systems output their respective decision scores, the system will further integrate the loss functions corresponding to each voiceprint recognition system and merge them with the adaptive weights of the learnable system through an ensemble learning method. Finally, the projected gradient descent method is used to optimize the adversarial perturbation, and the final speech adversarial sample is generated through a certain number of iterative optimizations.

[0044] The present invention discloses a method for constructing adversarial samples for a voiceprint recognition defense module, wherein the method is specifically carried out in the following steps:

[0045] Step 1: Provide an original audio and sample it to obtain an initial adversarial perturbation that satisfies the normal distribution;

[0046] Step 2: Select an acoustic masking sound that minimizes the distortion of the auditory perception and combine it with the obtained initial adversarial perturbation and the original audio to calculate the adversarial sample;

[0047] Step 3: Obtain multiple voiceprint recognition systems with different alternative defense modules by setting multiple alternative defense module types and hyperparameters;

[0048] Step 4: Further calculate the decision scores of various voiceprint recognition systems with different alternative defense modules and the obtained adversarial samples;

[0049] Step 5: The obtained decision scores are combined with the learnable adaptive weights through an ensemble learning method to obtain the final loss function.

[0050] Step 6: Use the projected gradient descent algorithm and the obtained loss function to iterate a certain number of times to repeat the above optimization process to generate the final speech adversarial sample.

[0051] The following is a detailed description of how to select the acoustic masking sound with the lowest distortion and how to superimpose it on the anti-disturbance process. Different acoustic masking sounds have different masking properties. By observing the masking effects of different masking sounds on white noise in advance, it is concluded that the more the energy distribution of the masking sound is concentrated in the high frequency band and the more continuous it is, the stronger its acoustic masking ability is. Then, according to the evaluation index proposed by the present invention, Calculate the score of each masker, where F(j) is the jth frequency bin corresponding to the selected masker, K is the total number of frequency bins, and M(i,j) is the corresponding filtered time-frequency spectrum of the masker. The value range of M-Sup is 0-1, and the larger the value, the stronger the masking ability. By calculating and comparing the M-Sup indicators of different maskers, the acoustic masker with the strongest masking ability is selected. Then, given the SNR r1, calculate the scaling factor And according to the adversarial disturbance amplitude A δ and the amplitude A of the selected acoustic masker M Calculate the scaled adversarial perturbation Get the scaled acoustic masking sound according to the scaling factor Given SNR r2, and following the same process as above, we can get the adversarial sample. Among them A r (i, j) is a function used to achieve the masking effect, which inserts audio j into audio i under a given SNR r to obtain the masked audio

[0052] The following details the process of selecting and configuring various defense modules and their hyperparameters. Signal processing-based defense methods are considered as defense modules for voiceprint recognition systems, due to their training-free and plug-and-play characteristics. Based on their underlying filtering defense principles, they can be divided into three categories: directly destroying perturbations with very small amplitudes, filtering out adversarial perturbations in high and low frequency bands, and filtering out adversarial perturbations below the threshold of hearing. For the first category, requantization and Mel filtering are considered. Their principle is to use lossy filters to directly destroy small adversarial perturbations. For the second category, resampling and bandpass filtering are considered. Their principle is based on the Nyquist-Shannon sampling theorem and frequency band extraction, directly filtering out or destroying adversarial perturbations in high and low frequency bands. For the third category, psychoacoustic filtering is considered. Its principle is based on the human hearing threshold curve and directly filters out adversarial perturbations below the threshold of hearing. Therefore, five representative methods are considered as alternative defense modules: requantization, Mel filtering, resampling, bandpass filtering, and psychoacoustic filtering. By pre-observing the relationship between the hyperparameter values ​​of each defense module and its defense strength, and by observing its impact on the performance of the voiceprint recognition model itself, we find the critical point setting of the hyperparameters of each defense module, which can maximize the defense strength while ensuring the normal working performance of voiceprint recognition.

[0053] The following is a detailed description of the anti-perturbation optimization process and the adversarial sample generation process. For various possible defense modules and their hyperparameter settings (F0, β0),…, (F n-1 ,β n-1 ), aims to construct an optimized adversarial perturbation δ to generate adversarial samples As a result, each alternative voiceprint recognition system mistakenly identifies it as the target user y t First, randomly sample from the normal distribution to initialize the adversarial perturbation In each update iteration, the selected acoustic masking sound is applied to the adversarial perturbation according to the above process and combined with the original audio x to generate an adversarial sample The adversarial sample is then input into each voiceprint recognition system with an alternative defense module to obtain the corresponding target user score. Based on this result, the following losses are calculated:

[0054]

[0055] Among them, θ i is the preset threshold of each alternative voiceprint recognition system, and κ is the confidence introduced to ensure the score is enough to exceed the threshold θ i Thus, it is successfully identified as the target user y t Based on this objective function, the gradient projection descent method is used to continuously update the adversarial perturbation and adversarial examples Until the early stopping condition is met.

[0056] The following is a detailed description of the integrated defense module. The present invention also discloses an integrated defense module technology, which enhances the adaptability of the adversarial disturbance to meet the needs of attacking voiceprint systems with different defense modules. Specifically, the technology integrates the outputs of multiple voiceprint systems with different defense modules in the disturbance optimization process. When a voice sample x is input into the optimization process, the corresponding adversarial perturbation is generated based on δ; then, the generated adversarial sample is input to a voiceprint model system with n defense modules selected from a predefined set of defense modules, rather than just one model, to obtain various outputs. After that, the n outputs are aggregated into a whole by weighted summation. Taking into account the small differences such as optimization direction conflicts among different defense modules in the optimization latent space, the present invention introduces a learnable weight factor for real-time adjustment of each system, that is, by iterating the weight coefficient w i Dynamically adjust each system The composition ratio of Figure 2 is an integration diagram; therefore, after using ensemble learning to integrate voiceprint systems with different defense modules, the optimization problem to be solved is converted into the following form:

[0057]

[0058]

[0059] By integrating learning and learnable weight factors, adversarial perturbation expands its target from voiceprint models targeting a single defense module to voiceprint models with their own defense modules, thereby achieving adaptability to the defense modules and further improving the effectiveness of the attack.

[0060] To verify the technical effects of the present invention, adversarial perturbations were generated by minimizing the objective function on a server (Intel Xeon Silver 4210R, 256GB RAM, NVIDIA RTX A6000) to construct adversarial samples. By default, 5 seconds of audio were randomly selected from the ESC-50 dataset as the acoustic masking sound. For the five defense modules, the corresponding hyperparameters were set according to the above critical points. In addition, SNR(M,δ)=18, SNR(x,M)=15, and confidence level κ=0.1 were set. The adversarial perturbations were optimized using the gradient projection descent method based on the cosine annealing decay strategy to generate adversarial samples. In addition, three training datasets (taken from LibriSpeech, Voxceleb1, and Voxceleb2) were used to train three mainstream voiceprint model architectures (d-vector, x-vector, and ecapa-tdnn), with their respective recognition performance of 4.68%, 3.81%, and 0.57%.

[0061] During the attack test, 60,000 voice adversarial samples were generated for 10 target users using the voices of 20 attackers. In addition, an attack method based on the gradient projection descent optimization method was implemented as a baseline, with perturbation limits of 0.003, 0.003, and 0.002 on the three voiceprint models. Therefore, a total of 396,000 attack tests were launched on the three voiceprint systems with / without the five defense modules. The following indicators are used to evaluate the performance of the present invention: 1) Attack Success Rate (ASR): ASR = n s / n t × 100%, where n t is the total number of test samples, n s is the number of successful attacks. 2) Signal-to-Noise Ratio (SNR): SNR = 10log 10 (P x / P δ ), where P x and P δ The SNR represents the signal power of the original speech and the adversarial perturbation, respectively. A higher SNR indicates less distortion and less perceptible perturbation. 3) Perceptual evaluation of speech quality (PESQ): This is an objective speech quality measurement score that describes the auditory quality perceived by the human ear. The score ranges from -0.5 to 4.5, with higher values ​​indicating better speech quality.

[0062] Evaluation of attack effectiveness and imperceptibility. As shown in Table 1 below, the present invention is compared with the most typical baseline attack method based on the projected gradient descent method in the field. It can be seen that the average ASR of the present invention for the five defense modules on the three voiceprint models of d-vector, x-vector and ecapa-tdnn respectively reached 91.7%, 97.1% and 99.1%, which are 48.7%, 29.8% and 63.4% higher than the baseline method respectively. In addition, the SNR of the adversarial samples generated by the present invention on the three voiceprint models reached 32.04dB, 35.55dB and 36.53dB, respectively, which is an average improvement of 30% compared to PGD. And the PESQ reached 4.06, 4.14 and 4.32, respectively, which is an improvement of 73% over PGD. This fully proves that the adversarial samples generated by the present invention have extremely strong attack effectiveness and imperceptibility.

[0063] Table 1 Comparison of the effectiveness and imperceptibility of the proposed method and the baseline attack

[0064]

[0065]

[0066] Adaptive evaluation. Figure 3 Figure 2 shows the ASR of the present invention and the baseline under different hyperparameters for different defense modules. It can be seen that for the three defense modules (requantization, resampling, and bandwidth filtering), the ASR increases monotonically with increasing hyperparameters, reaching 93%, before reaching stability beyond a critical point. For the psychoacoustic filtering method, the ASR decreases monotonically with increasing hyperparameters, initially maintaining an ASR of 98%, then rapidly declining after exceeding a critical point. For the Mel filter method, the ASR initially increases monotonically with increasing hyperparameters to 94%, then remains stable until a critical point, after which it begins to decline. For these five defense modules, within the hyperparameter range where the voiceprint recognition system works effectively (i.e., EER below 10%), the average ASR of the present invention is 15.4%, 48.2%, 55.4%, 36.0%, and 62.9% higher than the baseline, respectively. These results fully demonstrate the present invention's strong anti-defense capabilities and its ability to adaptively attack multiple defense modules with different hyperparameters.

[0067] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A method for constructing adversarial samples for a voiceprint recognition defense module, characterized in that: The steps include: Obtaining an initial adversarial perturbation, original audio, and acoustic masker set; Select an acoustic masking sound from the acoustic masking sound set that can minimize the auditory distortion, and combine it with the obtained initial adversarial perturbation and the original audio to calculate the adversarial sample; Set multiple alternative defense module types and hyperparameters to obtain multiple voiceprint recognition systems with different alternative defense modules; The respective decision scores are further calculated through various voiceprint recognition systems with different alternative defense modules and the obtained adversarial samples; The obtained decision scores are combined with learnable adaptive weights through ensemble learning methods to obtain the final loss function; The final speech adversarial sample is generated by repeating the above optimization process a certain number of times through the projected gradient descent algorithm and the obtained loss function.

2. The method for constructing adversarial samples for the voiceprint recognition defense module according to claim 1, characterized in that: The method of selecting an acoustic masking sound from the acoustic masking sound set that can minimize the hearing distortion and combining it with the obtained initial adversarial perturbation and the original audio to calculate the adversarial sample is as follows: 1) By observing the time-frequency spectra of various masking sounds in advance and calculating their PESQ to characterize their masking capabilities, an evaluation index is proposed. Where F(j) is the jth frequency bin corresponding to the selected masker, K is the total number of frequency bins, and M(i,j) is the corresponding filtered time-frequency spectrum of the masker. 2) The value range of M-Sup is 0-1. The larger the value, the stronger the masking ability. Therefore, by calculating and comparing the M-Sup indicators of different masking sounds, the acoustic masking sound with the strongest masking ability is selected. 3) Given SNR r1, calculate the scaling factor And according to the adversarial disturbance amplitude A δ and the amplitude A of the selected acoustic masker M Compute the scaled adversarial perturbation 4) Obtain the scaled acoustic masking sound according to the scaling factor Given the original audio x and SNR r2, and following the same process as above to get the adversarial sample Among them A r (i, j) is a function used to achieve the masking effect, which inserts audio j into audio i under a given SNR r to obtain the masked audio 3. The method for constructing adversarial samples for the voiceprint recognition defense module according to claim 1, characterized in that: The aforementioned setting of multiple alternative defense module types and hyperparameters to obtain multiple voiceprint recognition systems with different alternative defense modules is specifically as follows: 1) Based on the simplicity of no training and plug-and-play, we mainly consider signal processing-based defense methods as the defense module of the voiceprint recognition system. These defense methods are divided into three categories: directly destroying disturbances with very small amplitudes, filtering out adversarial disturbances in high and low frequency bands, and filtering out adversarial disturbances below the hearing threshold curve; 2) By observing the relationship between the hyperparameter values ​​of each defense module and its defense strength in advance, and by observing its impact on the performance of the voiceprint recognition model itself, the critical point setting of the hyperparameters of each defense module is found, and finally a variety of alternative defense modules (F0, β0),…, (F n-1 ,β n-1 ) voiceprint recognition model Among them F i and β i is the type and corresponding hyperparameters of the i-th defense module.

4. The method for constructing adversarial samples for the voiceprint recognition defense module according to claim 3, characterized in that: Among the three types of defense methods described, the first type mainly considers requantization and Mel filtering, whose principle is based on the use of lossy filters to directly destroy tiny adversarial disturbances; the second type mainly considers resampling and bandpass filtering, whose principle is based on the Nyquist-Shannon sampling theorem, directly filtering out or destroying adversarial disturbances in high and low frequency bands; the third type mainly considers psychoacoustic filtering, whose principle is based on the human ear hearing threshold curve in psychoacoustics, and achieves defense function by directly filtering out adversarial disturbances below the threshold; a total of the five most representative methods are considered as alternative defense modules: requantization, Mel filtering, resampling, bandpass filtering, and psychoacoustic filtering.

5. The method for constructing an adversarial sample for a voiceprint recognition defense module according to claim 1, 2, 3, or 4, characterized in that: The decision scores obtained are integrated with the learnable adaptive weights through the ensemble learning method to obtain the final loss function, which is specifically: Among them, x is the input speech sample, y t is the target label, A M is the masking function used to constrain the adversarial perturbation δ and the acoustic masker M, For the alternative defense module F i And the hyperparameter β i The corresponding loss function of the voiceprint recognition system is set, w i It is the learnable weight factor for integrating various voiceprint recognition systems.

6. The method for constructing adversarial samples for the voiceprint recognition defense module according to claim 5, characterized in that: The projected gradient descent algorithm and the resulting loss function are iterated a certain number of times to iterate the above optimization process to generate the final speech adversarial sample, specifically: 1) After masking the initial adversarial perturbation with the selected acoustic masking sound, it is combined with the original audio to obtain the adversarial sample Then input multiple F i In the voiceprint recognition system, the corresponding target score is obtained And the confidence κ is introduced to ensure the score is enough to exceed the preset threshold θ i Thus, it is successfully identified as the target user y t , so the following loss is calculated for each voiceprint recognition system: 2) Use ensemble learning methods for each loss to obtain a synthetic loss function where w i To integrate the learnable weight factors of various voiceprint recognition systems; 3) Use the projected gradient descent method to update the adversarial perturbation δ and generate new adversarial samples Until the early stopping condition is met, the final adversarial sample is generated.

Images