Equipment robustness evaluation method, device and system for voice replay attack detection

Abstract

The invention relates to an equipment robustness evaluation method, device and system for voice replay attack detection. The method comprises the following steps: acquiring a real voice data set; constructing a playback voice data set of different equipment combinations based on the real voice data set; the device combination comprises an attacker playing device, an attacker recording device and a system recording device; based on the to-be-evaluated detection model, respectively carrying out authenticity detection on the real voice data set and the playback voice data set to obtain a first prediction score of the real voice data set and a second prediction score of the playback voice data set; and calculating an evaluation index of the to-be-evaluated detection model under each equipment combination according to the first prediction score and the second prediction score. Through the method and the device, a real attack scene is simulated to construct the playback voice data set, the evaluation indexes of the to-be-evaluated detection model under different equipment combinations are calculated, and the performance of the to-be-evaluated detection model on voice playback attacks of different equipment combinations is evaluated, so that the robustness of different equipment is evaluated.

Description

Technical Field

[0001] The present application relates to the technical field of voice replay detection, and in particular to a method, device and system for evaluating the robustness of equipment for voice replay attack detection. Background Art

[0002] In recent years, with the rapid development of the Internet of Things and artificial intelligence technologies, voice interaction technologies such as speech synthesis, speech recognition, and voiceprint recognition have developed rapidly. As a contactless, user-friendly new biometric recognition method, voiceprint recognition has been widely deployed in smart homes, mobile finance, education and medical care, public security and other fields. For example, smart devices equipped with intelligent voice assistants can determine the user's voiceprint through the user's voice, thereby achieving contactless unlocking and identity authentication.

[0003] A replay attack is a form of attack in which the attacker records the target user's authentication voice and replays it with a loudspeaker to simulate the target user's speaking scene, thereby bypassing the voiceprint recognition system's identity verification. The current replay attack detection system can use a phased detection scheme to determine whether the voice comes from a real person speaking or a loudspeaker replay. However, the audio equipment used to record and play user voices in real scenarios is complex and diverse, resulting in different degrees of gain and distortion in the replayed voice, which may interfere with the detection results of voice replay attacks. Therefore, there is an urgent need for a method that can evaluate the robustness of voice replay attack detection on different devices. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device and system for evaluating the robustness of a voice replay attack detection device, which can evaluate the robustness of a voice replay attack detection device, in response to the above technical problems.

[0005] In a first aspect, a method for evaluating device robustness of voice replay attack detection is provided in this embodiment, including:

[0006] Get a real speech dataset;

[0007] Based on the real voice data set, construct a replayed voice data set of different device combinations; the device combination includes an attacker playback device, an attacker recording device, and a system recording device;

[0008] Based on the detection model to be evaluated, respectively, the real speech data set and the replayed speech data set are detected for authenticity, to obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set;

[0009] An evaluation index of the detection model to be evaluated under each of the device combinations is calculated according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

[0010] In some embodiments, constructing a playback voice dataset of different device combinations based on the real voice dataset includes:

[0011] Using a standard playback device to play the real speech in the real speech data set;

[0012] Using different attacker recording devices to respectively collect the real voices to generate attacker recorded voice data sets;

[0013] Using different attacker playback devices to respectively play the recorded voices in the attacker recorded voice data set;

[0014] The recorded voices are respectively collected using different recording devices of the system to generate the replayed voice data set.

[0015] In some of the embodiments, a standard playback device and different attacker recording devices are deployed in the voice collection space, and when playing the real voice, all the attacker recording devices are synchronously controlled to collect the voice;

[0016] And, different attacker playback devices and system recording devices are deployed in the voice collection space, and each time the playback voice is played, all the system recording devices are synchronously controlled to collect the voice.

[0017] In some embodiments, the performing authenticity detection on the real speech dataset and the replayed speech dataset based on the detection model to be evaluated, respectively, to obtain a first prediction score of the real speech dataset and a second prediction score of the replayed speech dataset, comprises:

[0018] Preprocessing and feature extraction are performed on the real speech and the replayed speech to be detected to obtain the real speech feature and the replayed speech feature;

[0019] The real speech feature and the replayed speech feature are input into the detection model to be evaluated for authenticity detection, and the first prediction score and the second prediction score are obtained respectively.

[0020] In some embodiments, calculating the evaluation index of the to-be-evaluated detection model under each device combination according to the first prediction score and the second prediction score includes:

[0021] Constructing a true label sequence according to the first prediction score and the second prediction score;

[0022] After classifying the first prediction score and the second prediction score as true or false based on a preset threshold, constructing a prediction label sequence;

[0023] Calculate the false acceptance rate and false rejection rate of all speech in each device combination according to the real label sequence and the predicted label sequence;

[0024] According to the false acceptance rate and the false rejection rate, an equal error rate is obtained as the evaluation index.

[0025] In some embodiments, after classifying the first prediction score and the second prediction score as true or false based on a preset threshold, constructing a prediction label sequence further includes:

[0026] After respectively setting different preset thresholds to classify the first prediction score and the second prediction score as true or false, a number of prediction label sequences are correspondingly constructed.

[0027] In a second aspect, in this embodiment, a device robustness assessment apparatus for voice replay attack detection is provided, including:

[0028] A data set acquisition module is used to acquire a real speech data set;

[0029] A data set construction module, used to construct a replayed voice data set of different device combinations based on the real voice data set; the device combination includes an attacker playback device, an attacker recording device and a system recording device;

[0030] A model prediction module, used to perform authenticity detection on the real speech data set and the replayed speech data set respectively based on the detection model to be evaluated, and obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set;

[0031] A robustness evaluation module is used to calculate the evaluation index of the detection model to be evaluated under each device combination according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

[0032] In a third aspect, in this embodiment, a device robustness assessment system for voice replay attack detection is provided, comprising: a standard playback device, an attacker playback device, an attacker recording device, a system recording device, and a device robustness assessment apparatus for voice replay attack detection as described in the second aspect;

[0033] The standard playback device is used to play the real speech in the real speech data set;

[0034] The attacker recording device is used to collect the real voice played by the standard playback device to generate an attacker recorded voice data set;

[0035] The attacker playback device is used to respectively play the recorded voices in the attacker recorded voice data set;

[0036] The system recording device is used to respectively collect the recorded voices played by the attacker's playback device to generate a playback voice data set.

[0037] In a fourth aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the device robustness evaluation method for voice replay attack detection described in the first aspect above is implemented.

[0038] In a fifth aspect, in this embodiment, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the device robustness assessment method for voice replay attack detection described in the first aspect above is implemented.

[0039] Compared with the related art, the device robustness evaluation method, apparatus and system for voice replay attack detection provided in this embodiment obtains a real voice data set; based on the real voice data set, constructs a replay voice data set of different device combinations; the device combination includes an attacker playback device, an attacker recording device and a system recording device; based on the detection model to be evaluated, respectively performs authenticity detection on the real voice data set and the replay voice data set to obtain a first prediction score of the real voice data set and a second prediction score of the replay voice data set; according to the first prediction score and the second prediction score, calculates the evaluation index of the detection model to be evaluated under each device combination, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index. Through this embodiment, a replay voice data set is constructed by simulating a real attack scenario, taking into account the factors of the attacker recording device, the playback device and the system recording device, and calculating the evaluation index of the detection model to be evaluated under different device combinations, so as to systematically evaluate the performance of the detection model to be evaluated against voice replay attacks of different device combinations, and realize the evaluation of the robustness of different devices.

[0040] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0042] Figure 1 is a hardware structure block diagram of a terminal of a method for evaluating device robustness of voice replay attack detection in an embodiment;

[0043] Figure 2 is a flow chart of a method for evaluating device robustness of voice replay attack detection in an embodiment;

[0044] Figure 3 is a schematic diagram of a process of constructing a playback speech data set in an embodiment;

[0045] Figure 4 is a schematic diagram of equipment deployment in a voice collection space in an embodiment;

[0046] Figure 5 is a flow chart of a method for evaluating device robustness of voice replay attack detection in another embodiment;

[0047] Figure 6 The present invention is a structural block diagram of a device robustness assessment apparatus for voice replay attack detection in an embodiment.

[0048] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, data set acquisition module; 20, data set construction module; 30, model prediction module; 40, robustness evaluation module. DETAILED DESCRIPTION

[0049] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0050] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0051] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 4 is a hardware structure block diagram of a terminal of the device robustness evaluation method for voice replay attack detection of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 102 and memory 104 for storing data, wherein processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0052] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the device robustness assessment method for voice replay attack detection in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0053] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0054] In recent years, with the rapid development of the Internet of Things and artificial intelligence technologies, voice interaction technologies such as speech synthesis, speech recognition, and voiceprint recognition have developed rapidly. As a contactless, user-friendly new biometric recognition method, voiceprint recognition has been widely deployed in smart homes, mobile finance, education and medical care, public security and other fields. For example, smart devices equipped with intelligent voice assistants can determine the user's voiceprint through the user's voice, thereby achieving contactless unlocking and identity authentication.

[0055] A replay attack is a form of attack in which the attacker records the target user's authentication voice and replays it with a loudspeaker to simulate the target user's speaking scene, thereby bypassing the voiceprint recognition system's identity authentication. Replay attacks have the characteristics of low implementation threshold and high attack damage, posing a great threat to the user's privacy and property security. Therefore, replay attack detection systems have emerged. The current replay attack detection system can adopt a staged detection scheme to determine whether the voice comes from the real speaker or the loudspeaker playback. However, the audio equipment used to record and play user voices in real scenarios is complex and diverse, resulting in different degrees of gain and distortion in the replayed voice, which may interfere with the detection results of voice replay attacks. Therefore, there is an urgent need for a method that can evaluate the robustness of voice replay attack detection on different devices.

[0056] In this embodiment, a method for evaluating device robustness of voice replay attack detection is provided. Figure 2 FIG. 4 is a flow chart of a method for evaluating device robustness of voice replay attack detection in this embodiment. Figure 2 As shown, the method comprises the following steps:

[0057] Step S201, obtaining a real speech data set.

[0058] Specifically, you can obtain real speech datasets by recording yourself or from high-quality open source speech libraries such as VCTK, LibriSpeech, LJSpeech, AISHELL-3, etc. The real speech samples in the real speech dataset come from different speakers and can have different accents, languages, etc.

[0059] Taking the high-fidelity open source speech corpus VCTK (English) as an example, the interference of environmental noise and room reverberation can be reduced. The VCTK corpus contains voice data from 110 speakers with different accents, and each person reads out about 400 sentences from newspapers and speeches. All voice data are recorded in a professional anechoic room in a high-quality format (sampling rate 96kHz, sampling bit 24 bits) using the same professional recording equipment (for example, omnidirectional microphone DPA 4035 and condenser microphone Sennheiser MKH 800). All recordings are converted to 16 bits, downsampled to 48kHz, and artificial endpoint processing is performed. In order to avoid the overall recording taking too long, 30 to 50 voice samples are randomly selected for each speaker in the embodiment by random sampling to obtain a real voice data set D1.

[0060] Step S202, based on the real voice data set, construct a playback voice data set of different device combinations; the device combination includes an attacker playback device, an attacker recording device and a system recording device.

[0061] Specifically, in real-world scenarios, the audio equipment used to record and play voices is complex and diverse. For example, different recording devices have completely different frequency response curves and noise processing capabilities when recording audio due to differences in microphone components and underlying signal processing software and hardware. Different voice playback devices have different degrees of gain and distortion effects in the played back voices due to differences in speaker component quality and power amplifiers. The differences between different devices determine the quality of the voice signal, which may interfere with the judgment of the detection system.

[0062] In order to better simulate the real attack scenario, based on the real voice dataset, a replayed voice dataset of different device combinations is constructed. Each device combination includes the attacker's playback device, the attacker's recording device, and the system recording device. However, the specific playback devices and recording devices in each device combination are different. For example, recording devices with different recording qualities can use smart phones, recorders, microphones, etc. For the attacker's playback device, speakers with different playback sound qualities can be used. Among them, the playback device and the recording device are deployed in the voice collection space, and the scene of real people speaking is restored by playing real voices through high-quality speakers, and the real voices are collected through different attacker recording devices to simulate the attacker's secret recording and theft of user voices. The attacker's recorded voice is played through speakers with different playback sound qualities, and recorded through system recording devices with different recording qualities, and finally a replayed voice dataset collected by the system is generated to simulate the replay attack carried out by the attacker on the voiceprint recognition system on different devices in the actual scenario.

[0063] Step S203: Based on the detection model to be evaluated, authenticity detection is performed on the real speech data set and the replayed speech data set respectively to obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set.

[0064] Specifically, a speech replay attack detection model to be evaluated is deployed, and the real speech and replayed speech in the data set are input into the detection model to be evaluated after signal preprocessing and feature extraction, so that the model input meets the input form required by the model, and the processed features are input into the detection model to be evaluated, and a first prediction score output by the model for the real speech data set and a second prediction score output by the model for the replayed speech data set are obtained. In this embodiment, the score output by the model is a floating point number representing confidence. The larger the value, the more likely it is a speech sample of a real speaker, and vice versa.

[0065] Step S204, calculating the evaluation index of the detection model to be evaluated under each device combination according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

[0066] Specifically, the replayed voice samples are grouped according to different device combinations (attacker playback device, attacker recording device, system recording device), and the true label sequence and predicted label sequence are constructed according to the first prediction score and the second prediction score in each group. Each bit in the label sequence represents the true label and predicted label of the corresponding sample (the true voice is 1, and the replayed voice is 0). The equal error rate (EER) is calculated based on the true label sequence and the predicted label sequence of the first prediction score and the second prediction score as an evaluation indicator to evaluate the device robustness of the detection model to be evaluated according to the evaluation indicator.

[0067] Through the above steps, we simulate real attack scenarios to build a cross-device replay voice dataset, taking into account the factors of the attacker's recording device, playback device and system recording device, and calculate the evaluation indicators of the detection model to be evaluated under different device combinations. This can systematically and uniformly evaluate the performance of the detection model to be evaluated on voice replay attacks with different device combinations, realize the evaluation of the robustness of different devices, and provide a basis for ensuring and improving the detection reliability of the system, which is conducive to exploring the deep correlation between replay attack detection and device factors in the future.

[0068] In some embodiments, constructing the playback voice datasets of different device combinations based on the real voice dataset in step S202 includes the following steps:

[0069] Use standard playback equipment to play the real voice in the real voice dataset; use different attacker recording devices to collect real voices and generate attacker recorded voice datasets; use different attacker playback devices to play the recorded voices in the attacker recorded voice dataset; use different system recording devices to collect recorded voices and generate replayed voice datasets.

[0070] Specifically, Figure 3 FIG. 1 is a schematic diagram of the process of constructing a playback speech data set in this embodiment. Figure 3 As shown, a standard playback device is used to play the real speech in the real speech data set D1, wherein the standard playback device can be a monitoring speaker with low distortion, wide and flat frequency response, and the ability to restore high, medium and low frequency sounds in a balanced manner. The speech data played by the device can approximately restore the scene of a real person speaking.

[0071] Different attacker recording devices are used to collect real voices and generate attacker recorded voice data set D2. The real voice is collected by different attacker recording devices to simulate the attacker's secret recording and stealing of the user's voice. Among them, different attacker recording devices can be divided into three grades: high, medium, and low according to their recording quality. High-fidelity condenser microphones, mobile phones, conference microphones, etc. can be selected respectively. For example, the sampling rate of voice recording can be 44.1kHz and the sampling bit number is 16 bits.

[0072] Use different attacker playback devices to play the recorded voices in the attacker recorded voice data set respectively; use different system recording devices to collect the recorded voices respectively, and generate the replayed voice data set D3 for subsequent input into the model to be evaluated for authenticity detection. Replay through speakers with different playback sound quality, and record through recording devices with different recording quality to simulate the replay attack carried out by attackers on voiceprint recognition systems on different devices in actual scenarios. Among them, the attacker playback devices can be divided into three grades: high, medium, and low, and high-fidelity speakers, portable Bluetooth speakers, mobile phones, etc. can be selected respectively. Different system recording devices can be divided into three grades: high, medium, and low according to their recording quality, and high-fidelity condenser microphones, mobile phones, conference microphones, etc. can be selected respectively. For example, the sampling rate of voice recording can be 44.1kHz, and the sampling bit number is 16 bits.

[0073] Deploy standard playback devices and different attacker recording devices in the voice collection space, and synchronously control all attacker recording devices to collect when playing real voice; and deploy different attacker playback devices and system recording devices in the voice collection space, and synchronously control all system recording devices to collect each time the voice is played back.

[0074] in, Figure 4 Schematic diagram of the equipment deployment of the voice collection space in this embodiment. Figure 4 As shown, the audio recording device and the audio playback device are placed face to face and close to each other (0.5 to 1 meter) on a table in a quiet recording space. When the playback device is a standard playback device, different attacker recording devices are deployed accordingly; when the playback device is an attacker playback device, different system recording devices are deployed accordingly.

[0075] If the playback device is a wired or Bluetooth speaker, it is connected through a mobile workstation (laptop); if the playback device is a mobile terminal (such as a mobile phone), it can be played directly using the built-in multimedia player of the mobile phone. If the recording device is an independent microphone device, it is connected through a mobile workstation; if the recording device is a mobile terminal (such as a mobile phone), it can be recorded directly using the built-in recorder of the mobile phone. During the simulation process, the real voice sample to be played is first uploaded to the laptop or mobile phone, and then the recording function of all recording devices is synchronously turned on, and the real voice is played. After the playback is completed, the recording is stopped and the recorded voice data is saved to the storage medium, and the playback device and recording device label information of all voice samples are recorded at the same time.

[0076] By simulating real attack scenarios in this embodiment to build a cross-device replay voice data set, the factors of the attacker's recording device, playback device and system recording device are taken into account, and the interference of other non-device factors is eliminated through synchronous and close-range replay collection. In addition, the attacker's secret recording of the user's voice is simulated by playing clean voice with high-quality standard playback devices, which greatly simplifies the actual restoration of the attacker's recording process. At the same time, the recording and playback based on high-fidelity monitoring speakers can restore the voice timbre of the real user when speaking to the maximum extent, saving the time cost and economic cost required for collecting real voice, and promoting the relevant research and system development of replay attack detection. At the same time, it also has good scalability. It only needs to increase the device type in the recording and playback process to expand the evaluation range of the device robustness, thereby improving the coverage and credibility of the robustness evaluation itself. Since the audio recording is carried out synchronously, the additional time cost required to increase the device type is controllable.

[0077] In some embodiments, the above step S203 performs authenticity detection on the real speech data set and the replayed speech data set based on the detection model to be evaluated, respectively, to obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set, including the following steps:

[0078] The real speech and the replayed speech to be detected are preprocessed and feature extracted to obtain real speech features and replayed speech features; the real speech features and the replayed speech features are input into the detection model to be evaluated for authenticity detection to obtain a first prediction score and a second prediction score respectively.

[0079] Specifically, taking the Gaussian mixture model and the RawNet2 model as the deployed detection model to be evaluated as an example, the Gaussian mixture model accepts constant-Q cepstral coefficient features (constant-Q cepstral coefficients, CQCC) as input, and the RawNet2 model accepts 16kHz waveform feature input. Therefore, when the authenticity of the voice sample is detected, it is necessary to first undergo signal preprocessing and feature extraction algorithms to make it conform to the input form required by the voice playback attack detection model to be evaluated. For the RawNet2 model, the original voice samples (real voice and playback voice) are downsampled to 16kHz. For the Gaussian mixture model, the downsampled voice data is further subjected to constant Q transformation and cepstral coefficient extraction, and finally a two-dimensional feature with a dimension of (L, C) is obtained, where L is the number of frames of the feature in the time domain (variable according to the actual length of the voice), and C is the number of cepstral coefficients, which is exemplary, 13 in the embodiment. The processed real speech features and replayed speech features are input into the detection model to be evaluated, and the first prediction score and the second prediction score of the model output are obtained respectively. The prediction score output by the model is a floating point number representing the confidence level. The larger the value, the more likely it is a speech sample of the real speaker, otherwise it is a replayed speech sample.

[0080] In this embodiment, the detection model to be evaluated is used to output a first prediction score for the real speech and a second prediction score for the replayed speech, so that the first prediction score and the second prediction score can be used to evaluate the robustness of the subsequent device.

[0081] In some embodiments, the step S204 calculates the evaluation index of the detection model to be evaluated under each device combination according to the first prediction score and the second prediction score, including the following steps:

[0082] A true label sequence is constructed according to the first prediction score and the second prediction score; after classifying the first prediction score and the second prediction score as true or false based on a preset threshold, a prediction label sequence is constructed; based on the true label sequence and the prediction label sequence, the false acceptance rate and false rejection rate of all speech under each device combination are calculated; based on the false acceptance rate and the false rejection rate, an equal error rate is obtained as an evaluation indicator.

[0083] Specifically, the replayed voice samples are grouped according to different device combinations (attacker playback device, attacker recording device, system recording device), and the error rate is calculated as the evaluation index based on the first prediction score and the second prediction score in each group. The first prediction score of the real voice and the second prediction score of the replayed voice are sorted (for example, sorted from small to large), and the score sequence X = {x i |i∈(0,N)} and the true label sequence Y={y i|i∈(0,N)}, where x i is the prediction score of the i-th sample, y i is the true label of the i-th sample (1 for real speech and 0 for playback speech), and N is the number of samples in the corresponding group.

[0084] According to the preset threshold θ, the first prediction score and the second prediction score are classified as true or false, and the prediction label sequence of all samples under this threshold is obtained. in is the predicted label of the ith sample, defined as:

[0085]

[0086] It means that the prediction labels of samples with prediction scores less than the preset threshold are set to 0, and the prediction labels of samples with prediction scores greater than or equal to the preset threshold are set to 1.

[0087] According to the predicted label sequence and the true label sequence, the false acceptance rate (False Accept Rate, FAR) and false rejection rate (False Reject Rate, FRR) of all samples are calculated. The calculation formulas of FAR and FRR are:

[0088]

[0089] Among them, TP is the real speech sample predicted to be true TN is a replay attack sample predicted to be a replay FP is a replay attack sample that is predicted to be true FN is the real speech sample predicted to be reproduced

[0090] The calculation formula of EER is:

[0091]

[0092] Furthermore, different preset thresholds may be set to classify the first prediction score and the second prediction score as true or false, and then a number of prediction label sequences may be constructed accordingly.

[0093] Among them, different preset thresholds θ∈[min(x i ),max(x i )], after classifying the first prediction score and the second prediction score as true or false, several prediction label sequences are correspondingly constructed. The above calculations are performed on the prediction label sequence and the true label sequence respectively, so as to obtain the FAR sequence and FRR sequence under different preset thresholds, and thus obtain several equal error rate sequences, and take the smallest EER value as the final evaluation index.

[0094] By calculating the evaluation index of the detection model to be evaluated under different device combinations in this embodiment, the performance of the detection model to be evaluated against voice replay attacks of different device combinations can be systematically and uniformly evaluated, thereby realizing the evaluation of the robustness of different devices.

[0095] The present embodiment is described and illustrated below through preferred embodiments.

[0096] Figure 5 FIG. 4 is a flow chart of a method for evaluating device robustness of voice replay attack detection in this embodiment. Figure 5 As shown, the method comprises the following steps:

[0097] Step S501, obtaining a real voice data set, using a standard playback device to play the real voice in the real voice data set, using different attacker recording devices to synchronously collect the real voice respectively, and generating an attacker recorded voice data set.

[0098] Step S502: different attacker playback devices are used to respectively play the recorded voices in the attacker recorded voice data set; different system recording devices are used to synchronously collect the recorded voices to generate a playback voice data set.

[0099] Step S503, preprocessing and feature extraction are performed on the real speech and replayed speech to be detected to obtain real speech features and replayed speech features; the real speech features and replayed speech features are input into the detection model to be evaluated for authenticity detection to obtain a first prediction score and a second prediction score, respectively.

[0100] Step S504, constructing a true label sequence according to the first prediction score and the second prediction score; after setting different preset thresholds to classify the first prediction score and the second prediction score as true or false, a number of corresponding prediction label sequences are constructed.

[0101] Step S505, calculate the false acceptance rate and false rejection rate of all speech under each device combination according to the real label sequence and the predicted label sequence; obtain the equal error rate as the evaluation index according to the false acceptance rate and the false rejection rate, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

[0102] By simulating real attack scenarios in this embodiment to construct a cross-device replayed voice data set, the factors of the attacker's recording device, playback device, and system recording device are taken into account, and the evaluation indicators of the detection model to be evaluated under different device combinations are calculated. It can systematically and uniformly standardize the performance of the detection model to be evaluated for voice replay attacks on different device combinations, realize the evaluation of the robustness of different devices, and provide a basis for ensuring and improving the detection reliability of the system. At the same time, it also has good scalability. It only needs to increase the type of equipment in the recording and playback process to expand the evaluation scope of the device robustness, thereby improving the coverage and credibility of the robustness evaluation itself. Since the audio recording is carried out synchronously, the additional time cost required to increase the device type is controllable.

[0103] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0104] In this embodiment, a device for evaluating the robustness of a voice replay attack detection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be described again. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0105] Figure 6 is a structural block diagram of the device robustness evaluation apparatus for voice replay attack detection of this embodiment, such as Figure 6 As shown, the device comprises:

[0106] A data set acquisition module 10 is used to acquire a real speech data set;

[0107] A data set construction module 20 is used to construct a replayed voice data set of different device combinations based on the real voice data set; the device combination includes an attacker playback device, an attacker recording device and a system recording device;

[0108] A model prediction module 30 is used to perform authenticity detection on the real speech data set and the replayed speech data set based on the detection model to be evaluated, and obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set;

[0109] The robustness evaluation module 40 is used to calculate the evaluation index of the detection model to be evaluated under each device combination according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

[0110] Through the device provided in this embodiment, it is possible to simulate real attack scenarios to construct a cross-device replayed voice data set, taking into account the factors of the attacker's recording device, playback device and system recording device, and calculate the evaluation indicators of the detection model to be evaluated under different device combinations. It is possible to systematically and uniformly evaluate the performance of the detection model to be evaluated for voice replay attacks on different device combinations, realize the evaluation of the robustness of different devices, and provide a basis for ensuring and improving the detection reliability of the system.

[0111] In some of the embodiments, the data set construction module 20 is further used to use a standard playback device to play the real voice in the real voice data set; use different attacker recording devices to respectively collect the real voice and generate an attacker recorded voice data set; use different attacker playback devices to respectively play the recorded voice in the attacker recorded voice data set; use different system recording devices to respectively collect the recorded voice and generate a playback voice data set.

[0112] In some of the embodiments, the data set construction module 20 is also used to deploy standard playback devices and different attacker recording devices in the voice collection space, and synchronously control all attacker recording devices to collect when playing real voice; and deploy different attacker playback devices and system recording devices in the voice collection space, and synchronously control all system recording devices to collect each time the voice is played back.

[0113] In some of the embodiments, the model prediction module 30 is also used to preprocess and extract features of the real speech and replayed speech to be detected, so as to obtain real speech features and replayed speech features; and input the real speech features and replayed speech features into the detection model to be evaluated for authenticity detection, so as to obtain a first prediction score and a second prediction score, respectively.

[0114] In some of the embodiments, the robustness evaluation module 40 is also used to construct a true label sequence based on the first prediction score and the second prediction score; after classifying the first prediction score and the second prediction score as true or false based on a preset threshold, construct a prediction label sequence; calculate the false acceptance rate and false rejection rate of all speech under each device combination based on the true label sequence and the predicted label sequence; and obtain an equal error rate as an evaluation indicator based on the false acceptance rate and the false rejection rate.

[0115] In some of the embodiments, the robustness evaluation module 40 is further used to respectively set different preset thresholds to classify the first prediction score and the second prediction score as true or false, and then construct a number of prediction label sequences accordingly.

[0116] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0117] In this embodiment, a device robustness assessment system for voice replay attack detection is also provided, including: a standard playback device, an attacker playback device, an attacker recording device, a system recording device, and a device robustness assessment device for voice replay attack detection as in the above embodiments.

[0118] A standard playback device, used to play the real speech in the real speech dataset;

[0119] The attacker's recording device is used to collect the real voice played by the standard playback device and generate the attacker's recorded voice dataset;

[0120] The attacker's playback device is used to play the recorded voices in the attacker's recorded voice dataset respectively;

[0121] The system recording device is used to collect the recorded voices played by the attacker's playback device and generate a replayed voice dataset.

[0122] Among them, Figure 4 As shown, the audio recording device and the audio playback device are placed face to face and close to each other (0.5 to 1 meter) on a table in a quiet recording space. When the playback device is a standard playback device, different attacker recording devices are deployed accordingly; when the playback device is an attacker playback device, different system recording devices are deployed accordingly.

[0123] If the playback device is a wired or Bluetooth speaker, it is connected through a mobile workstation (laptop); if the playback device is a mobile terminal (such as a mobile phone), it can be played directly using the built-in multimedia player of the mobile phone. If the recording device is an independent microphone device, it is connected through a mobile workstation; if the recording device is a mobile terminal (such as a mobile phone), it can be recorded directly using the built-in recorder of the mobile phone. During the simulation process, the real voice sample to be played is first uploaded to the laptop or mobile phone, and then the recording function of all recording devices is synchronously turned on, and the real voice is played. After the playback is completed, the recording is stopped and the recorded voice data is saved to the storage medium, and the playback device and recording device label information of all voice samples are recorded at the same time.

[0124] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0125] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0126] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0127] In addition, in combination with the device robustness assessment method for voice replay attack detection provided in the above embodiments, a storage medium can also be provided in this embodiment for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any device robustness assessment method for voice replay attack detection in the above embodiments is implemented.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0129] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0130] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0131] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0132] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A device robustness evaluation method for voice replay attack detection, characterized in that: include: Get a real speech dataset; Based on the real voice data set, construct a playback voice data set of different device combinations; The device combination includes an attacker playback device, an attacker recording device, and a system recording device; Based on the detection model to be evaluated, respectively, the real speech data set and the replayed speech data set are detected for authenticity, to obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set; An evaluation index of the detection model to be evaluated under each of the device combinations is calculated according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

2. The method according to claim 1, characterized in that The step of constructing a playback voice data set of different device combinations based on the real voice data set includes: Using a standard playback device to play the real speech in the real speech data set; Using different attacker recording devices to respectively collect the real voices to generate attacker recorded voice data sets; Using different attacker playback devices to respectively play the recorded voices in the attacker recorded voice data set; The recorded voices are respectively collected using different recording devices of the system to generate the replayed voice data set.

3. The method according to claim 2, characterized in that Deploy a standard playback device and different attacker recording devices in the voice collection space, and synchronously control all the attacker recording devices to collect the voice when playing the real voice; And, different attacker playback devices and system recording devices are deployed in the voice collection space, and each time the playback voice is played, all the system recording devices are synchronously controlled to collect the voice.

4. The method according to claim 1, characterized in that: The method of performing authenticity detection on the real speech data set and the replayed speech data set based on the detection model to be evaluated, respectively, to obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set, comprises: Preprocessing and feature extraction are performed on the real speech and the replayed speech to be detected to obtain the real speech feature and the replayed speech feature; The real speech feature and the replayed speech feature are input into the detection model to be evaluated for authenticity detection, and the first prediction score and the second prediction score are obtained respectively.

5. The method according to claim 1, characterized in that The calculating, according to the first prediction score and the second prediction score, the evaluation index of the to-be-evaluated detection model under each of the device combinations comprises: Constructing a true label sequence according to the first prediction score and the second prediction score; After classifying the first prediction score and the second prediction score as true or false based on a preset threshold, constructing a prediction label sequence; Calculate the false acceptance rate and false rejection rate of all speech in each device combination according to the real label sequence and the predicted label sequence; According to the false acceptance rate and the false rejection rate, an equal error rate is obtained as the evaluation index.

6. The method according to claim 5, characterized in that After classifying the first prediction score and the second prediction score as true or false based on a preset threshold, constructing a prediction label sequence, further comprising: After respectively setting different preset thresholds to classify the first prediction score and the second prediction score as true or false, a number of prediction label sequences are correspondingly constructed.

7. A device robustness assessment apparatus for voice replay attack detection, characterized in that: include: A data set acquisition module is used to acquire a real speech data set; A data set construction module, used to construct a playback voice data set of different device combinations based on the real voice data set; The device combination includes an attacker playback device, an attacker recording device, and a system recording device; A model prediction module, used to perform authenticity detection on the real speech data set and the replayed speech data set respectively based on the detection model to be evaluated, and obtain a first prediction score of the real speech data set and a second prediction score of the replayed speech data set; A robustness evaluation module is used to calculate the evaluation index of the detection model to be evaluated under each device combination according to the first prediction score and the second prediction score, so as to evaluate the device robustness of the detection model to be evaluated according to the evaluation index.

8. A device robustness evaluation system for voice replay attack detection, characterized in that: include: A standard playback device, an attacker playback device, an attacker recording device, a system recording device, and a device robustness assessment apparatus for voice playback attack detection as claimed in claim 7; The standard playback device is used to play the real speech in the real speech data set; The attacker recording device is used to collect the real voice played by the standard playback device to generate an attacker recorded voice data set; The attacker playback device is used to respectively play the recorded voices in the attacker recorded voice data set; The system recording device is used to collect the recorded voices played by the attacker's playback device to generate a playback voice data set.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the device robustness assessment method for voice replay attack detection according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for evaluating the robustness of a device for detecting a voice playback attack according to any one of claims 1 to 6 are implemented.

Images