Sample construction method and replay detection method for a speech replay detection model

Abstract

This specification discloses a sample construction method and a playback inspection method for a speech playback detection model. The scheme first constructs a simulated speech playback attack chain and configures various acoustic parameters within it. Then, according to the order in which the acoustic parameters are used in the speech playback attack chain, the acquired target real speech is cascaded to obtain simulated playback speech. Finally, based on the obtained simulated playback speech, training samples are constructed for training the speech playback detection model.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the field of artificial intelligence technology, and in particular to a sample construction method and a playback detection method for a speech playback detection model. Background Technology

[0002] With the widespread application of voice interaction technology in smart homes, mobile payments, and identity authentication, the security of voice biometric recognition has received increasing attention. Among these attacks, replay attacks are a common method. Attackers record users' voices using recording devices and then play the recording back to the target device or system (such as a smart speaker, smartphone, or smart glasses), thereby bypassing the identity verification process and posing a significant security risk to users' information and assets.

[0003] To defend against such attacks, existing technologies typically employ speech replay detection techniques. The basic principle is to extract specific features from the speech (such as spectral distortion, high-frequency attenuation, and background noise statistical characteristics) and use a classifier to determine whether the input speech is spoken by a real person in real time or is replayed from a recording device.

[0004] Existing speech playback detection models are mostly based on deep learning architectures, such as Convolutional Neural Networks (CNNs) or Long Short-Term Memory (LSTMs), which train the speech playback detection model's recognition ability through a large amount of labeled training data.

[0005] However, existing technologies have significant limitations in acquiring training data. The main reason is that current training data is often collected using fixed equipment in fixed recording and playback environments, making it difficult to cover the diverse and complex situations in the real world. Consequently, speech playback detection models trained on this data can only recognize recordings produced in the same environments as the training data. If they encounter recordings from scenarios not covered by the training data, recognition will fail, posing a risk to users' personal information and property. Summary of the Invention

[0006] In view of the above, one or more embodiments of this specification provide the following technical solutions: According to a first aspect of one or more embodiments of this specification, a sample construction method for a speech playback detection model is proposed, the method comprising: A simulated voice replay attack chain is constructed. The voice replay attack chain is used to simulate the replay attack process of recording real voice with a recording device and playing the recorded voice back to the target acquisition device by configuring various acoustic parameters. The acoustic parameters include: device acoustic parameters, environmental acoustic parameters, and acoustic algorithm processing parameters. The device acoustic parameters are used to characterize the acoustic characteristics of the device involved in the voice replay attack chain. The environmental acoustic parameters are used to characterize the acoustic characteristics of the space where the device is located. The acoustic algorithm processing parameters are used to represent the algorithm parameters used by the device to process the voice internally. Acquire the target's real speech and configure various acoustic parameters in the speech replay attack chain; The target real speech is cascaded according to the order in which the acoustic parameters are used in the speech replay attack chain to obtain simulated replay speech. Based on the simulated replayed speech, training samples are constructed, which are used to train the speech replay detection model to be trained.

[0007] According to a second aspect of one or more embodiments of this specification, a replay inspection method is proposed, applied to a target acquisition device, comprising: Voice is collected in the space where the target acquisition device is located to obtain the voice to be detected. The speech to be detected is input into a pre-trained speech replay detection model to detect whether the speech to be detected is a replay attack speech based on the speech replay detection model. The speech playback detection model is trained using training samples constructed using the sample construction method described above for the speech playback detection model.

[0008] According to a third aspect of one or more embodiments of this specification, a voice replay attack simulation system is proposed, the system comprising: a link construction module, a configuration module, a simulation module, and a data encapsulation module; The link construction module is used to construct a simulated voice replay attack link. The voice replay attack link is used to simulate the replay attack process of recording real voice with a recording device and playing the recorded voice back to the target acquisition device by configuring various acoustic parameters. The acoustic parameters include: device acoustic parameters, environmental acoustic parameters, and acoustic algorithm processing parameters. The device acoustic parameters are used to characterize the acoustic characteristics of the device involved in the voice replay attack link. The environmental acoustic parameters are used to characterize the acoustic characteristics of the space where the device is located. The acoustic algorithm processing parameters are used to represent the algorithm parameters used by the device to process voice internally. The configuration module is used to acquire the target's real speech and configure various acoustic parameters in the speech replay attack chain; The simulation module is used to perform concatenated processing on the target real speech according to the order in which the acoustic parameters are used in the speech replay attack chain, so as to obtain simulated replay speech. The data encapsulation module is used to encapsulate the simulated replayed speech into training samples, which are used to train the speech replay detection model to be trained.

[0009] According to a fourth aspect of one or more embodiments of this specification, an electronic device is provided for storing a memory containing processor-executable instructions; wherein the processor implements the steps of the method described above by executing the executable instructions.

[0010] According to a fifth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0011] According to a sixth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described above.

[0012] In the sample construction method for the speech playback detection model provided in this specification, a simulated speech playback attack chain is first constructed, and various acoustic parameters are configured in the speech playback attack chain. Then, the target real speech is cascaded according to the order in which the various acoustic parameters are used in the speech playback attack chain to obtain simulated playback speech. Then, training samples for training the speech playback detection model are constructed based on the obtained simulated playback speech.

[0013] As can be seen from the above methods, by employing the aforementioned techniques, the simulated speech replay attack chain supports flexible configuration and dynamic combination of various acoustic parameters, covering a wide range of combinations of recording devices, acquisition devices, acoustic environments, and pickup algorithms that may occur in reality. This generates a large number of diverse simulated replay speech samples. Using these samples to train a speech replay detection model allows the model to learn a broad range of acoustic variation features during the training phase. Consequently, in practical applications, this significantly improves the recognition accuracy of the speech replay detection model and its replay detection coverage in various environments. Attached Figure Description

[0014] Figure 1 This document provides a flowchart illustrating a sample construction method for a speech playback detection model. Figure 2 A schematic diagram illustrating the process of configuring the acoustic parameters provided in this manual; Figure 3 A schematic diagram of the multi-segment filter bank structure for simulating the cavity effect of the device provided in this specification; Figure 4 A schematic diagram illustrating the process of constructing training samples for the voice replay attack simulation system provided in this manual and transmitting them to the training cluster device to complete model training; Figure 5 This is a schematic diagram of the playback detection method provided in this manual; Figure 6 A schematic diagram of the module structure of the voice replay attack simulation system provided in this manual; Figure 7 This is a schematic structural diagram of a device provided in this specification. Detailed Implementation

[0015] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this manual are all information and data authorized by the user or fully authorized by all parties. The collection, use and processing of related data shall comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals shall be provided for users to choose to authorize or refuse.

[0016] Specifically, the mainstream training data currently comes from two main sources: one is publicly available standard datasets (such as ASVspoof), and the other is small-scale experimental data collected in a laboratory environment.

[0017] While public datasets are large in scale, the device models, recording environments, and voice playback scenarios they cover are relatively fixed, making it difficult to cover the ever-changing attack combinations in the real world.

[0018] For example, differences in microphone frequency response between different mobile phone brands, reverberation characteristics in different rooms, and nonlinear modifications to speech signals by various sound pickup algorithms (such as noise reduction and automatic gain control) can all lead to drastic changes in the acoustic characteristics of reproduced speech. Furthermore, because existing training data lacks systematic coverage of these acoustic variations, the trained detection models suffer from severely insufficient generalization ability when faced with unknown combinations of devices or complex acoustic environments.

[0019] Specifically, when attackers use recording devices not present in the training set or play back recordings in a room with special reverberation characteristics, speech playback detection models often fail to accurately identify the playback features caused by newly introduced acoustic distortions, resulting in false negatives.

[0020] In addition, the cost and time required to collect actual data are high, and it is difficult to exhaust all possible combinations of "recording-playback-environment", which makes data-driven model optimization a bottleneck.

[0021] Therefore, how to generate high-quality simulated playback speech covering diverse acoustic variations without large-scale on-site data collection, in order to train a speech playback detection model with strong generalization ability, is a technical problem that urgently needs to be solved.

[0022] To address this, this specification provides a sample construction method for a speech replay detection model. The core of this method lies in constructing a parameterized simulated speech replay attack chain, transforming uncontrollable acoustic variations in the physical world into quantifiable and configurable parameters, thereby automatically generating a massive amount of diverse simulated replay speech samples. Through the generated training samples, a speech replay detection model with strong generalization ability is trained. In practical applications, the trained speech replay detection model can accurately identify speech replay attacks, ensuring the security of users' personal information and property.

[0023] To better describe the sample construction method for the speech playback detection model provided in this manual, some concepts involved in this manual will be explained in advance: Wall sound absorption coefficient: This refers to the ratio of sound energy absorbed by the wall material to the total incident sound energy when a sound wave is incident on the surface of the wall material. The value of this coefficient is usually between 0 and 1. The larger the value, the stronger the material's ability to absorb sound and the weaker the reflection.

[0024] In this specification, the wall absorption coefficient is one of the fundamental parameters for constructing a room's acoustic model, directly affecting the room's reverberation characteristics. Different wall materials (such as concrete, gypsum board, sound-absorbing cotton, glass, etc.) have different absorption coefficients, and these coefficients typically vary with frequency. By configuring different wall absorption coefficients, the acoustic environments of various real-world spaces, from highly reflective bathrooms to highly sound-absorbing recording studios, can be simulated.

[0025] Reverberation time: usually denoted as or Reverberation density refers to the time required for the sound energy density in a room to decrease by 60 decibels (dB) after the sound source stops emitting sound. It is the most important statistical indicator for measuring the strength of reverberation in a room.

[0026] The length of reverberation time depends on the size and shape of the room, as well as the sound absorption characteristics of the wall materials.

[0027] In this specification, reverberation time is both a target constraint for generating the room impulse response (RIR) and an important dimension when sampling spatial parameters. For example, small rooms with a lot of sound-absorbing material have shorter reverberation times (e.g., 0.3 seconds), while large, open halls have longer reverberation times (e.g., more than 2.0 seconds).

[0028] By sampling different reverberation times, the simulation system can cover a variety of acoustic scenarios, from quiet offices to noisy shopping malls.

[0029] Room Impulse Response (RIR) is a core concept in acoustic simulation and audio processing, used to fully describe the physical characteristics of sound propagation within a specific space.

[0030] RIR (Resonant Infrared) records the energy distribution waveforms of all direct sound, early reflections, and late reverberation received by a receiver (such as a microphone) at different points in time after an ideal unit pulse signal is emitted from a sound source. In digital signal processing, convolving any speech signal with an RIR can accurately simulate the propagation effect of the signal within a room, including time delay, attenuation, spectral coloration, and spatial perception.

[0031] In this specification, RIR is the core data form characterizing environmental acoustic parameters. It can be generated by geometric acoustic algorithms or retrieved from a measured database.

[0032] Frequency response: refers to the gain or attenuation characteristics of a device (such as a microphone, speaker, headphones, etc.) for sound signals of different frequencies. Ideally, a device should maintain the same gain (i.e., a flat response) for all frequencies, but in practice, hardware is limited by physical structure and often has resonant peaks or attenuation troughs in certain frequency bands.

[0033] Frequency response is typically represented by an amplitude-frequency curve. In this specification, the acoustic parameters of the device are primarily described in terms of frequency response. For example, the frequency response of a recording device simulates the bandwidth limitations of a microphone (such as high-frequency cutoff) and the distortion characteristics of a speaker during playback (such as low-frequency boom).

[0034] By applying specific frequency response filters to the simulated voice replay attack chain, the acoustic "fingerprints" brought about by different brands and models of hardware can be realistically reproduced.

[0035] Sound pickup algorithm: refers to the digital signal processing (DSP) algorithm integrated inside the device, used to optimize the quality of the acquired or played voice.

[0036] Common speech pickup algorithms include Automatic Gain Control (AGC), noise suppression (including Active Noise Suppression (ANS) or Noise Suppression (NS), acoustic echo cancellation (AEC), and dynamic range compression. These algorithms are typically nonlinear and time-varying, and can introduce specific artifacts or distortions into the speech signal.

[0037] In this specification, acoustic algorithm processing parameters are used to simulate the behavior of these algorithms. By abstracting the sound pickup algorithm into a configurable signal processing module, it is possible to simulate the differentiated performance of different devices when processing speech, such as the speech interruptions or spectral smoothing caused by the strong noise reduction effects of some mobile phones.

[0038] Noise suppression level: This parameter controls the processing intensity of the noise suppression module in the sound pickup algorithm. It determines how aggressive the algorithm is in removing background noise.

[0039] A higher noise suppression level means that the algorithm will more aggressively estimate and subtract noise components, which can improve the signal-to-noise ratio, but may also lead to excessive smoothing, distortion, or even "musical noise" artifacts in the speech signal; while a lower level retains more original signal details, but may leave more background noise.

[0040] In this specification, by sampling the noise suppression level, various simulated playback speech samples ranging from "slight noise reduction" to "heavy noise reduction" can be generated, thereby training the speech playback detection model to recognize the changes in acoustic features caused by different noise reduction intensities.

[0041] Gain target value: This is a parameter that controls the desired output signal strength of the AGC algorithm. The role of AGC is to dynamically adjust the amplification factor of the input signal, ensuring that the amplitude of the output signal remains near the preset target level. This avoids clipping distortion due to excessive volume or a low signal-to-noise ratio due to insufficient volume. The gain target value is usually expressed in decibels (dB).

[0042] In this specification, different target gain values ​​result in simulated speech reproduction exhibiting varying overall loudness and dynamic range characteristics. By sampling different target gain values, the simulation environment can mimic volume variations experienced by a user speaking at different distances or with different device sensitivity settings, enhancing the robustness of the speech reproduction detection model to volume variations.

[0043] Echo cancellation depth: This parameter controls the intensity of echo signal suppression by the AEC algorithm. In hands-free calling or smart speaker scenarios, the sound emitted by the device may be picked up again by the microphone of the same device, creating an echo. The AEC algorithm aims to estimate and cancel this echo. The echo cancellation depth determines the algorithm's tolerance for echo remnants and its handling strategy against two-way talking (both parties speaking simultaneously). A deeper cancellation depth can remove echoes more thoroughly, but may accidentally delete near-end speech in two-way talking; a shallower depth retains more near-end speech, but may leave echo remnants.

[0044] In this specification, by sampling the echo cancellation depth, the echo processing effect of different devices in complex interactive scenarios can be simulated, so that the simulated replay of speech covers various echo residual modes and improves the speech replay detection model's ability to recognize echo artifacts.

[0045] Cascaded processing: This is a structure that connects multiple processing units (or stages) in series. Each unit completes a part of the work in sequence, and the output of the previous unit directly becomes the input of the next unit.

[0046] In this specification, by using the various acoustic parameters configured in the speech replay attack chain, and by cascading the acquired speech according to the order in which these acoustic parameters are used in the speech replay attack chain, it is possible to simulate the replay attack process of recording real speech with a recording device and playing the recorded speech back to the target acquisition device, thereby generating a massive amount of diverse simulated replay speech samples.

[0047] It should be noted that in the method provided in this specification, the speech generated in the simulation system may not be real speech; its specific form can be data transmitted in the simulation system in the form of a signal. In actual speech playback detection, the target acquisition device acquires speech that propagates as sound waves in the real environment. The speech to be detected, obtained by processing the acquired speech, also belongs to data transmitted in the form of a signal. For ease of description, in this specification, whether it propagates as sound waves in the real environment or is transmitted as a signal in a system or device, it is referred to as speech.

[0048] The technical solution to be protected in this specification will now be described in detail with reference to the accompanying drawings.

[0049] Figure 1 This is a flowchart illustrating a sample construction method for a speech playback detection model provided in this specification. Figure 1 As shown, the method includes the following steps: S100: Construct a simulated voice replay attack chain.

[0050] The sample construction method for the speech replay detection model provided in this specification can be implemented by various entities. For example, it can be a terminal device such as a desktop computer or laptop computer, a client installed on the terminal device capable of performing simulation tasks, a server or a cluster of multiple servers, or a speech replay attack simulation system composed of a terminal device and one or more servers. For ease of description, the following will only use the aforementioned speech replay attack simulation system as an example to explain in detail the sample construction method for the speech replay detection model provided in this specification.

[0051] In this manual, users can perform simulation tasks based on the voice replay attack simulation system they are using. By executing these simulation tasks, simulated replay voices covering various scenarios can be generated.

[0052] Among them, the voice replay attack simulation system can respond to the user's simulation operation and build a simulated voice replay attack link in a preset simulation environment.

[0053] A voice replay attack chain refers to the entire process of recording real speech using a recording device and then playing that recorded speech back to the target acquisition device. For example, during a user's voice payment using smart glasses, the smart glasses normally collect the user's voice and verify through a backend server whether the voice is from the user (i.e., the identity verification process). After confirming that the user has passed the identity verification process, semantic recognition is performed on the user's voice to complete the voice payment process.

[0054] However, during a voice replay attack, the attacker may have previously recorded the user's actual voice using a recording device. Therefore, after obtaining the user's smart glasses, the attacker can play the recorded voice within the space where the smart glasses are located, thus achieving the voice replay attack. In this process, the user's smart glasses become the target acquisition device in the voice replay attack chain.

[0055] The aforementioned voice replay attack chain actually involves acoustic variations at multiple stages. This specification constructs a parameterized voice replay attack chain and abstracts these variations into three types of acoustic parameters: device acoustic parameters, environmental acoustic parameters, and acoustic algorithm processing parameters.

[0056] Device acoustic parameters are used to characterize the acoustic features of devices involved in a voice playback attack chain. In real-world attack scenarios, recording devices (such as smartphones and voice recorders) and target acquisition devices (such as Bluetooth speakers, smartphones, and smart glasses) have different hardware characteristics, primarily manifested in differences in frequency response. Frequency response describes the device's gain or attenuation characteristics for different frequency audio signals.

[0057] For example, the microphone of a low-end mobile phone may experience significant attenuation in the high-frequency range, while a small Bluetooth speaker may have resonance peaks in the low-frequency range. In this specification, the acoustic parameters of the device may primarily include the frequency response of the recording device.

[0058] In one embodiment, the frequency response can be obtained by measuring the frequency response curve of a real device and stored as filter coefficients or a frequency response lookup table. In another embodiment, the frequency response can also be retrieved from a pre-built device acoustic library based on device identifiers (such as device model), which covers the frequency response characteristics of mainstream devices on the market.

[0059] Environmental acoustic parameters are used to characterize the acoustic features of the space where the devices involved in a speech replay attack chain are located. When sound propagates through space, it is reflected, scattered, and absorbed by objects such as walls and furniture, creating reverberation and echoes. This spatial effect significantly impacts the time-frequency structure of the speech signal.

[0060] In the simulated speech playback attack chain of this specification, environmental acoustic parameters may include the RIR of the space where the recording device is located, and the RIR of the space where the target acquisition device is located. The RIR is a complete fingerprint describing the acoustic characteristics of the room, and any sound emitted in the room can be regarded as the convolution result of the original speech and the room's RIR.

[0061] In one embodiment, the Resonance Indicator (RIR) can be calculated using a geometric acoustic model (such as the image source method) based on parameters such as room size, wall absorption coefficient, and reverberation time. In another embodiment, the RIR can also be retrieved from open-source databases or measured datasets that cover RIRs for various real-world room types.

[0062] Acoustic algorithm processing parameters are used to represent the algorithm parameters used internally by the devices involved in the speech replay attack chain to process speech. Modern recording equipment and target acquisition equipment typically incorporate multiple digital signal processing algorithms, such as AGC, ANS, and AEC. These algorithms perform non-linear processing on the speech signal, introducing specific distortions or artifacts. In this specification, acoustic algorithm processing parameters may include the algorithm parameters of the sound pickup algorithm used by the recording equipment, as well as the algorithm parameters of the sound pickup algorithm used by the target acquisition device.

[0063] It should be noted that the aforementioned algorithm parameters can actually be determined through the algorithm's control parameters, which can refer to parameters such as noise suppression level, target gain value, and echo cancellation depth. These control parameters can be converted into specific coefficients within the algorithm through mapping relationships, such as the filter order and over-attenuation factor.

[0064] By configuring the above three types of parameters, various acoustic variations that occur in the real world can be highly reproduced in the simulated speech replay attack chain, laying the foundation for generating high-quality simulated replay speech.

[0065] In this specification, conventional acoustic simulation software can be used to construct the simulated speech playback attack chain. It should be noted that the acoustic simulation software used must have a parameterized configuration interface. This interface allows external devices or programs to dynamically input various parameters (such as the aforementioned spatial parameters and control parameters), thereby configuring various acoustic parameters in the simulation environment to simulate the speech playback attack chain.

[0066] S102: Obtain the target's real speech and configure various acoustic parameters in the speech replay attack link.

[0067] S104: The target real speech is cascaded according to the order in which the acoustic parameters are used in the speech replay attack chain to obtain simulated replay speech.

[0068] After constructing the aforementioned speech replay attack chain, the speech replay attack simulation system can obtain the target's real speech. The target's real speech can be a clean human voice from a publicly available dataset, or high-quality real human speech collected in a laboratory environment. Because the target's real speech is a real human voice, it is characterized by not containing any acoustic distortions introduced by replay attacks.

[0069] Subsequently, the voice replay attack simulation system can configure various acoustic parameters within the simulated voice replay attack chain. The configuration process can be random or based on sampling according to a specific strategy.

[0070] In one embodiment, configuring the acoustic parameters may include: sampling the spatial parameters of the space where the recording device is located to determine the RIR of the recording environment; sampling the recording device identifier to query the frequency response of the recording device; and sampling the control parameters of the pickup algorithm used by the recording device to determine the algorithm parameters. Similarly, the environment, device, and algorithm parameters of the playback end are sampled and configured. The specific configuration process for each acoustic parameter will be described in subsequent sections.

[0071] Once configured, the voice replay attack simulation system sequentially processes the target real speech using the configured acoustic parameters according to the physical order of the voice replay attack chain. This process simulates the entire process of sound from its production, recording, storage, playback, to its re-acquisition.

[0072] Specifically, the processing is divided into two stages: recording and playback.

[0073] During the recording phase, the speech playback attack simulation system mimics the process of an attacker using a recording device to record the target's real speech. This phase includes: convolving the target's real speech with the room impulse response of the space where the recording device is located, and then processing the processed speech sequentially through the frequency response of the recording device and the algorithm parameters of the sound pickup algorithm used by the recording device to obtain the filtered speech of the recording device. The specific details can be found in the following formula:

[0074] in, This indicates the acquired target's actual speech. Representing a time variable, therefore, The specific data format is actually a discrete time series array, which consists of a series of amplitude sampling points collected at a fixed sampling rate. Each sampling point represents the instantaneous amplitude value of the speech at a given moment.

[0075] This indicates the room impulse response of the space where the recording equipment is located. Also representing a time variable, therefore, The data is also in the form of a discrete time series array, which records the distribution of reflected sound energy received by the recording device at different time points after the sound source emits a unit pulse.

[0076] Indicates the frequency response of the recording device. Representing a frequency variable, therefore, The data is typically represented as a complex array or a real gain curve in the frequency domain. It describes the amplitude attenuation or amplification factor of different frequency components (from low to high frequencies) by the recording equipment.

[0077] This refers to the sound pickup algorithm in the recording device. This indicates the filtered audio obtained from the recording device.

[0078] Therefore, as can be seen from the above formula, during the recording stage, the target real speech is actually first convolved using the room impulse response of the space where the recording equipment is located, to obtain the impulse convolution result of the recording equipment (i.e., ） This is to simulate the reflection and reverberation effects of sound in the room where the recording equipment is located.

[0079] Then, the frequency response of the recording device is used to convolve the pulse convolutional speech of the recording device to obtain the frequency response convolution result of the recording device (i.e., ）This is to simulate the attenuation or enhancement of different frequency components by the recording equipment. Finally, the built-in pickup algorithm of the recording equipment also needs to process the frequency response convolution result of the recording equipment to simulate the nonlinear signal processing (such as noise reduction and gain control) process inside the recording equipment, so as to obtain the filtered speech of the recording equipment.

[0080] During the playback phase, the speech replay attack simulation system simulates the process of an attacker using a recording device to play a recording onto a target acquisition device. This phase includes: performing convolution processing on the filtered speech using the room impulse response of the space where the target acquisition device is located to obtain the impulse convolution speech of the target acquisition device; and then, based on the impulse convolution speech of the target acquisition device, obtaining the simulated replay speech.

[0081] If the speech collected by the target acquisition device is needed as a training sample in the subsequent process, the pulse convolution speech of the target acquisition device can be directly used as the simulated playback speech. The simulated playback speech obtained in this case is speech that is not affected by the hardware of the target acquisition device and its internal sound pickup algorithm.

[0082] If the speech input from the target acquisition device to the speech playback detection model is needed in subsequent steps, after obtaining the pulse convolutional speech from the target acquisition device, it can be further processed by the frequency response of the target acquisition device and the algorithm parameters of the sound pickup algorithm used by the target acquisition device to obtain the simulated playback speech. The specific formula is as follows:

[0083] in, This refers to the audio played by the recording device (i.e., the filtered audio mentioned above). This represents the room impulse response of the space where the target acquisition device is located. This represents the frequency response of the target acquisition device. This refers to the sound pickup algorithm in the target acquisition device. This indicates the final simulated playback audio. For a detailed explanation of each parameter in the formula, please refer to the explanation of the parameters in the recording stage above; further details will not be provided here.

[0084] Therefore, as can be seen from the above formula, during the playback phase, the room impulse response of the space where the target acquisition device is located is first used to perform convolution processing on the speech played by the recording device, thus obtaining the impulse convolution result of the target acquisition device (i.e., ） This is to simulate the reflection and reverberation effects of sound in the room where the target acquisition device is located.

[0085] Then, the frequency response of the target acquisition device is used to perform convolution processing on the pulse convolution result of the target acquisition device to obtain the frequency response convolution result of the target acquisition device (i.e., ） This is to simulate the attenuation or enhancement of different frequency components by the target acquisition device.

[0086] Finally, the built-in sound pickup algorithm of the target acquisition device also needs to process the frequency response convolution result of the target acquisition device to simulate the nonlinear signal processing (such as noise reduction and gain control) process inside the target acquisition device, so as to obtain the final simulated playback speech.

[0087] It should be noted that the above... and This indicates the sound pickup algorithm used. However, in this manual, what needs to be configured are the control parameters in the algorithm, so as to determine the algorithm parameters in the algorithm through the control parameters. For the sake of simplicity, these algorithm parameters are not reflected in the above formula.

[0088] S106: Based on the simulated replayed speech, construct training samples, which are used to train the speech replay detection model to be trained.

[0089] After obtaining the simulated replayed audio, the audio replay attack simulation system uses it as a training sample and labels it as "replay". Simultaneously, the system can acquire several real audio samples (i.e., clean human voices that have not passed through the replay attack chain or real human voices directly captured on the target acquisition device), and also use these real audio samples as training samples, labeling them as "real human". Finally, the simulated replayed audio and real audio samples are combined to form the training sample set.

[0090] Since the above method can generate a large number of parameter combinations with different environmental, device and algorithm characteristics by traversing or randomly sampling the parameter space, thereby constructing simulated replay speech covering a variety of situations, this greatly enriches the richness of samples involving replay attacks in the training sample set.

[0091] Building upon this foundation, training samples can be input into the speech playback detection model to be trained in subsequent processes. This model can be a deep learning model based on a Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), or Transformer architecture. The input to the speech playback detection model can be a spectrogram or the original waveform of the speech, and the output is a binary classification result (real person or playback).

[0092] During training, the speech playback detection model performs speech playback detection on each training sample, obtaining a detection result. This result indicates whether the speech playback detection model determines the training sample to be genuine human speech or a replayed speech. Then, the deviation between the detection result and the corresponding label of the training sample is calculated, and the loss value is determined based on this deviation.

[0093] In this specification, the loss function can take various forms, such as cross-entropy loss. Regarding the loss value mentioned above, it is positively correlated with the deviation; that is, the greater the deviation, the greater the loss value.

[0094] Next, the model's weight parameters can be updated using the backpropagation algorithm based on the loss value to minimize the loss. After multiple rounds of iterative training, the speech playback detection model gradually learns the key features that distinguish real speech from played speech, such as the high-frequency attenuation unique to played speech, abnormal reverberation patterns, and artifacts handled by the algorithm.

[0095] The obtained speech replay detection model, trained using the above methods, possesses the ability to generalize to unknown device combinations and complex acoustic environments, and can effectively defend against various speech replay attacks.

[0096] It should be noted that in the above-described simulated voice playback attack chain, only the influence of the frequency response of the recording unit in the recording device on the voice was considered, and the influence of the frequency vector of the playback unit in the recording device on the voice was not considered. Therefore, to further ensure that the simulated voice playback attack chain process is closer to reality, in this specification, the frequency response of the recording device includes: the frequency response of the recording unit in the recording device, and the frequency response of the playback unit in the recording device. The recording unit can refer to a component in the recording device, such as a microphone, that can capture voice, while the playback unit can refer to a component in the recording device, such as a speaker, that can play voice.

[0097] Furthermore, the specific process of the recording stage can be referenced using the following formula:

[0098] In the above formula, That is, used to represent the frequency response of the recording unit in a recording device. Right now Used to represent the frequency response of the playback unit in a recording device.

[0099] Accordingly, in this specification, the frequency response of the target acquisition device mentioned above may refer to the frequency response of the recording unit in the target acquisition device.

[0100] Furthermore, in practical applications, the main function of the built-in sound pickup algorithm in both recording devices and target acquisition devices is noise reduction. Therefore, the above process actually ignores the noise impact on sound as it propagates in space (because most of the noise superimposed on the speech will be filtered out by the sound pickup algorithm in the device).

[0101] To obtain the simulated replay audio more accurately, the noise effect in the space where the device is located can also be considered in the simulated audio replay attack link.

[0102] Specifically, during the recording stage, the speech replay attack simulation system can simulate the ambient noise in the space where the recording device is located, and mix the target real speech propagating in the space with this ambient noise to obtain a first mixed speech. This ambient noise can take various forms, such as Gaussian white noise or recorded real ambient noise (e.g., street noise, office noise, etc.).

[0103] The first mixed speech can then be processed in the manner described above to obtain filtered speech.

[0104] The same applies to the playback stage described above. Specifically, the voice playback attack simulation system can simulate the ambient noise in the space where the target acquisition device is located, and mix the recorded voice (i.e., the filtered voice mentioned above) propagating in the space with this ambient noise to obtain a second mixed voice. Then, the obtained second mixed voice can be processed in the manner described above to obtain the simulated playback voice.

[0105] In step S102 mentioned above, it was discussed how to configure various acoustic parameters in the voice replay attack chain. To further illustrate how to achieve diversified parameter configuration, this specification combines... Figure 2 The specific implementation method of parameter configuration is described in detail.

[0106] Figure 2 This is a schematic diagram illustrating the process of configuring the acoustic parameters provided in this manual.

[0107] like Figure 2 As shown, the parameter configuration process is essentially the process of sampling from an abstract parameter space and instantiating it into a specific signal processing module.

[0108] For environmental acoustic parameters, the speech playback attack simulation system samples the spatial parameters of the space where the recording equipment is located to determine the room impulse response based on the sampled spatial parameters. These spatial parameters may include: room dimensions, wall absorption coefficients, reverberation time, etc.

[0109] In one embodiment, the voice playback attack simulation system can preset multiple room templates (such as small conference rooms, large auditoriums, and outdoor streets), each template corresponding to a set of spatial parameter ranges. During sampling, specific values ​​are randomly selected from the range, and the corresponding RIR matrix is ​​generated in real time through an acoustic simulation engine (such as an algorithm based on image sources).

[0110] In another embodiment, the voice playback attack simulation system maintains a large-scale RIR database, where each RIR file is labeled with its corresponding spatial parameter metadata. During sampling, a subset of RIRs that meet specific spatial parameter ranges is first filtered according to requirements, and then an RIR file is randomly selected as the current instance. Similarly, the spatial parameters of the space where the target acquisition device is located can be sampled in the same way to determine the room impulse response of the space where the playback device is located.

[0111] For the device acoustic parameters, the voice replay attack simulation system samples the recording device identifier and the target acquisition device identifier, and based on the sampled identifiers, queries the frequency response of the recording device and the target acquisition device used in the voice replay attack link.

[0112] In one embodiment, the device identifier can be a device model string. The voice playback attack simulation system maintains a hardware frequency response function library, which stores frequency response curve data or filter coefficients for various common recording and target acquisition devices. During a query, the corresponding frequency response data is directly read based on the identifier (including the recording device identifier and the target acquisition device identifier).

[0113] In another embodiment, if the frequency response of the required device cannot be found in the hardware frequency response function library, a reference frequency response can be selected according to the device category (such as "high-end mobile phone" or "cheap microphone"), and a certain random perturbation can be superimposed to obtain a frequency response that differs from the reference frequency response, which can then be used as the queried frequency response.

[0114] For acoustic algorithm processing parameters, the speech playback attack simulation system samples the control parameters of the pickup algorithm used by the recording device to determine the algorithm parameters of the pickup algorithm used by the recording device based on the sampled control parameters. The control parameters include at least one of the following: noise suppression level, target gain value, and echo cancellation depth.

[0115] In one embodiment, the control parameters can be high-level configuration items, such as noise suppression levels divided into "low," "medium," and "high." The speech playback attack simulation system internally maintains a mapping table that maps the "high" level to specific algorithm parameters, such as a larger spectral subtraction over-subtraction factor or a more aggressive gain compression threshold. By changing the control parameters, the processing effects of devices from different brands and with different settings can be simulated. Similarly, a similar approach can be used to sample the control parameters of the pickup algorithm used by the target acquisition device to determine its algorithm parameters.

[0116] Through this parameterized configuration and instantiation mechanism, the speech replay attack simulation system can flexibly combine acoustic scenarios under various conditions, thereby generating a large number of diverse simulated replay speech, effectively solving the problem of incomplete training data coverage in traditional methods.

[0117] In practical applications, the physical cavity structure of devices (including recording devices and target acquisition devices) (such as the microphone's sound inlet, the speaker's resonance cavity, and gaps in the device's casing) can also produce specific resonance or leakage effects on sound, causing local peaks or dips in the frequency spectrum. To more precisely simulate this effect and obtain more realistic speech reproduction, this specification further incorporates multiple filter banks into the aforementioned simulation process, such as... Figure 3 As shown.

[0118] Figure 3 This is a schematic diagram of the multi-segment filter bank structure for simulating the cavity effect of the device provided in this specification.

[0119] like Figure 3 As shown, a multi-segment filter bank can be composed of multiple bandpass or bandstop filters connected in parallel or cascaded, with each filter simulating a certain resonant mode or leakage path in the cavity structure.

[0120] During the recording phase, before the speech, processed by the room impulse response of the recording equipment, is sequentially processed by the frequency response of the recording equipment and the algorithm parameters of the pickup algorithm used by the recording equipment, the speech playback attack simulation system can construct a multi-segment filter bank of the recording equipment in the simulation environment. This multi-segment filter bank is used to simulate the influence of the physical cavity structure of the recording equipment on the sound.

[0121] Subsequently, the speech replay attack simulation system can use the constructed multi-segment filter bank of the recording device to filter the frequency response convolution result of the recording device, obtaining the cavity-filtered speech of the recording device. Then, using the algorithm parameters of the sound pickup algorithm used by the recording device, the cavity-filtered speech of the recording device is processed to obtain the filtered speech.

[0122] In one embodiment, the multi-segment filter bank can be a set of second-order IIR filters whose center frequency, Q value, and gain can be set according to the resonant characteristics of a typical microphone cavity. For example, simulating a pinhole resonance might produce a narrowband peak at 4 kHz. By introducing this subtle spectral distortion, the simulated speech more closely resembles the physical characteristics of real hardware.

[0123] During the playback phase, similarly, before the speech processed by the room impulse response of the target acquisition device is processed by the frequency response of the target acquisition device and the algorithm parameters of the pickup algorithm used by the target acquisition device, the speech replay attack simulation system can construct a multi-segment filter bank of the target acquisition device in the simulation environment. This multi-segment filter bank is used to simulate the influence of the physical cavity structure of the target acquisition device on the sound.

[0124] Next, using the constructed multi-segment filter bank of the target acquisition device, the frequency response convolution result of the target acquisition device is filtered to obtain the cavity-filtered speech of the target acquisition device. Then, using the algorithm parameters of the sound pickup algorithm used by the target acquisition device, the cavity-filtered speech of the target acquisition device is processed to obtain the simulated playback speech.

[0125] In one embodiment, for small Bluetooth speakers, cavity resonance may produce a noticeable booming sound in the low-frequency range (e.g., 100Hz-200Hz). Multi-band filter banks can simulate this low-frequency resonance, making the simulated speech more realistic in both sound and frequency spectrum.

[0126] By inserting multiple filter banks after frequency response filtering and before algorithm processing, this manual can more accurately reproduce the acoustic fingerprint of the device's physical structure, further improving the realism of the simulation data, thereby helping the speech playback detection model learn deeper discriminative features.

[0127] It should be noted that the above-described content can be divided into the training sample construction process and the model training process. In this specification, the entities executing these two processes can be different. For example, the training sample construction process can be implemented by the aforementioned speech replay attack simulation system, while the model training process can be implemented by the deployed training cluster equipment. The entire process is roughly as follows: Figure 4 As shown.

[0128] Figure 4 This diagram illustrates the process of constructing training samples for the voice replay attack simulation system provided in this manual and transmitting them to the training cluster device to complete model training.

[0129] exist Figure 4The speech replay attack simulation system can construct simulated replay speech covering various combinations of environments, devices, and algorithms. Through the network connection between the speech replay attack simulation system and the training cluster device, various training samples constructed based on the simulated replay speech are transmitted to the training cluster device. Thus, the training cluster device uses the various training samples to complete the training process of the speech replay detection model.

[0130] Based on the training samples constructed using the above-described training sample construction method, an embodiment of this specification also provides a speech playback detection method applied to a target acquisition device.

[0131] Furthermore, the above description actually refers to a scenario where real speech is recorded using a recording device and then played back to the target acquisition device using the same recording device. However, in practical applications, there are also cases where the device used to record the real speech and the device used to play back the recorded speech are different devices.

[0132] For example, an attacker records a user's real voice using a recording device in an environment, and then sends the recorded voice to a playback device used by another attacker via wired or wireless transmission. The other attacker then uses the playback device to play the recorded voice to the user's smart glasses, thereby achieving a voice replay attack.

[0133] In this case, the acoustic parameters of the device in this specification may also include the frequency response of the playback device. Correspondingly, the recorded speech played into the space where the target acquisition device is located (i.e., the filtered speech mentioned above) can actually be obtained by processing the frequency response convolution result of the recording device using the algorithm parameters of the sound pickup algorithm employed by the recording device to obtain the algorithm-processed speech, and then using the frequency response of the playback device to perform convolution processing on the algorithm-processed speech. The specific process can be referred to in the following formula:

[0134] In the above formula, This is used to represent the frequency response of the playback device. The explanations of other formula terms are as described above and will not be elaborated here.

[0135] Furthermore, in practical applications, playback devices may also employ audio pickup algorithms. Therefore, the speech replay attack simulation system can also simulate the process of the playback device's audio pickup algorithm processing the received speech and playing it back. That is, the algorithm parameters of the playback device's audio pickup algorithm can be used to process the speech processed by the aforementioned algorithm, and the processed speech can be convolved with the playback device's frequency response to obtain the filtered speech played back to the target acquisition device.

[0136] Furthermore, the influence of the physical cavity structure of the playback device on the sound can also be considered, thereby constructing a multi-segment filter bank of the playback device in the simulation environment to perform speech filtering through the multi-segment filter bank. This process can be referred to the above content and will not be described in detail here.

[0137] It should also be noted that in practical applications, there may be situations where a recording device plays recorded audio in space, and the playback device picks up the recorded audio. This is similar to the situation where a recording device plays recorded audio to a target acquisition device in space. Therefore, if this situation is considered, the audio replay attack simulation system can refer to the process of a recording device playing recorded audio to a target acquisition device in space to simulate the audio replay attack chain under this condition. The specific process will not be described in detail here.

[0138] Finally, the above content only uses the room impulse response as the environmental acoustic parameter, the device frequency response as the device acoustic parameter, and the pickup algorithm parameters as the acoustic algorithm processing parameters as an example to introduce the sample construction method for the speech playback detection model provided in this manual.

[0139] In practical applications, environmental acoustic parameters, device acoustic parameters, and acoustic algorithm processing parameters can take other forms. For example, environmental acoustic parameters can also include: multipath channel impulse response (in acoustics, sound often travels from a sound source to a receiver not along a single straight path, but through multiple reflections from objects such as walls and furniture, forming multiple paths (multipath). The multipath channel impulse response describes the sequence of "arrival time (delay) and energy attenuation (gain)" for these different paths); device acoustic parameters can also include: finite impulse response coefficients (essentially a digital filter used to simulate the filtering characteristics of a device); and acoustic algorithm processing parameters can also include: audio codec operators (a set of algorithms used to compress and decompress audio data, and the encoding and decoding process introduces specific distortion characteristics such as quantization noise, band cutoff, and pre-echo).

[0140] Other forms of environmental acoustic parameters, device acoustic parameters, and acoustic algorithm processing parameters will not be listed here. Any parameter that satisfies the requirement of cascading the target's real speech according to the order of use of acoustic parameters in the speech replay attack chain, as described in this manual, is acceptable.

[0141] Figure 5 This is a schematic diagram of the playback detection method provided in this manual, such as... Figure 5 As shown, the method includes the following steps: S500: Speech acquisition is performed in the space where the target acquisition device is located to obtain the speech to be detected.

[0142] The target acquisition device uses its built-in microphone array to collect the user's voice commands or authentication voice, obtaining the voice to be detected. The voice to be detected here refers to the voice processed by the target acquisition device using its built-in sound pickup algorithm. The voice collected by the target acquisition device is affected by the spatial environment in which the target acquisition device is located and the hardware characteristics of the target acquisition device.

[0143] The replay detection method provided in this manual can be applied in various scenarios. For example, in a scenario where a user makes a voice payment using a portable device, the user can speak into the portable device, which then captures the voice to verify the user's identity and completes the payment after confirming successful verification. Another example is in a login scenario, where the target acquisition device can capture the user's voice and perform an initial verification. After successful initial verification, the device verifies the login password or verification code entered by the user, thus performing a second verification. Only after successful second verification can the user's interface be displayed for business operations.

[0144] Other application scenarios will not be detailed here.

[0145] S502: Input the speech to be detected into a pre-trained speech replay detection model, so as to detect whether the speech to be detected is a replay attack speech according to the speech replay detection model.

[0146] The target acquisition device can input the speech to be detected into a pre-trained speech replay detection model. This speech replay detection model is trained using training samples constructed according to the aforementioned method and has the ability to identify diverse replay attacks. The model analyzes the input speech, extracts its features, and outputs the detection result. The detection result is usually a binary label ("real person" or "replay"), or a confidence score representing the probability of replay.

[0147] If the target acquisition device is a device with strong computing power (such as a laptop or a high-performance smartphone), a lightweight speech playback detection model can be deployed on the target acquisition device. In this way, the target acquisition device can directly input the obtained speech to be detected into the locally deployed lightweight speech playback detection model to obtain the detection result.

[0148] If the local computing power of the target acquisition device cannot support the local deployment of the speech playback detection model, the target acquisition device (such as portable devices like smart glasses) can send the speech to be detected to the remotely deployed speech playback detection model via network connection to obtain the detection results.

[0149] Furthermore, if the detection results determine that the collected voice is genuine human voice, then the user is considered authenticated and the user's voice commands are executed. If the detection results determine that the collected voice is replaying audio, then the user is considered unauthenticated and a security alert is issued.

[0150] Using this method, the target acquisition device can quickly and accurately identify voice replay attacks locally or through the cloud, protecting users' personal information and property security.

[0151] Based on the same concept as the above method, this specification also provides a voice replay attack simulation system.

[0152] Figure 6 This is a schematic diagram of the module structure of the voice replay attack simulation system provided in this manual. Figure 6 As shown, the system includes: a link construction module 600, a configuration module 602, a simulation module 604, and a data encapsulation module 606.

[0153] The link construction module 600 is used to construct a simulated voice playback attack link. This module is responsible for initializing and managing the various acoustic parameters involved in the simulated voice playback attack link. In one embodiment, the link construction module 600 is also responsible for maintaining the device frequency response library, RIR database, and algorithm parameter mapping table, and supports dynamic sampling and updating of parameters.

[0154] Configuration module 602 is used to acquire the target's real speech and configure various acoustic parameters in the simulated speech replay attack chain. Specifically, this configuration module can parse the configuration request input by the user in the speech replay attack simulation system to determine the user's parameter configuration requirements, and then configure the various acoustic parameters according to these requirements. Of course, the entire training sample construction process can be automated; in this case, the configuration module also automatically completes the parameter sampling process and configures the various acoustic parameters in the speech replay attack chain using the sampled parameters.

[0155] Simulation module 604 is used to cascade the target real speech according to the order in which the acoustic parameters are used in the speech replay attack chain, to obtain simulated replay speech. This module may internally include a processing unit from the recording stage. Figure 6 The recording processing unit and the playback processing unit (in the middle) Figure 6 The playback processing unit in the middle performs the aforementioned RIR convolution, frequency response filtering, cavity filtering and algorithm processing operations respectively.

[0156] The data encapsulation module 606 is used to encapsulate simulated playback speech into training samples. This module labels the generated simulated playback speech with the "playback" tag and packages it together with real speech samples to form a standard training dataset format (such as TFRecord, HDF5, etc.) for input into the speech playback detection model to be trained.

[0157] Figure 7 This is a schematic structural diagram of a device provided in this manual. Please refer to it. Figure 7 At the hardware level, the device includes a processor 702, an internal bus 704, a network interface 706, memory 708, and non-volatile memory 710, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 702 reads the corresponding computer program from the non-volatile memory 710 into memory 708 and then runs it. Of course, in addition to software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0158] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0159] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0160] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0161] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0162] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0163] The above are merely embodiments of this specification and are not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A sample construction method for a speech playback detection model, the method comprising: A simulated voice replay attack chain is constructed. The voice replay attack chain is used to simulate the replay attack process of recording real voice with a recording device and playing the recorded voice back to the target acquisition device by configuring various acoustic parameters. The acoustic parameters include: device acoustic parameters, environmental acoustic parameters, and acoustic algorithm processing parameters. The device acoustic parameters are used to characterize the acoustic characteristics of the device involved in the voice replay attack chain. The environmental acoustic parameters are used to characterize the acoustic characteristics of the space where the device is located. The acoustic algorithm processing parameters are used to represent the algorithm parameters used by the device to process the voice internally. Acquire the target's real speech and configure various acoustic parameters in the speech replay attack chain; The target real speech is cascaded according to the order in which the acoustic parameters are used in the speech replay attack chain to obtain simulated replay speech. Based on the simulated replayed speech, training samples are constructed, which are used to train the speech replay detection model to be trained.

2. The method of claim 1, the environmental acoustic parameters comprising: The room impulse response of the space where the recording equipment is located, and the room impulse response of the space where the target acquisition equipment is located; The acoustic parameters of the device include: the frequency response of the recording device; The acoustic algorithm processing parameters include: the algorithm parameters of the sound pickup algorithm used by the recording device; The target real speech is cascaded according to the order in which the acoustic parameters are used in the speech replay attack chain to obtain simulated replay speech, including: The target real speech is convolved using the room impulse response of the space where the recording equipment is located to obtain the impulse convolved speech of the recording equipment. The impulse convolved speech of the recording equipment is convolved using the frequency response of the recording equipment to obtain the frequency response convolved result of the recording equipment. The frequency response convolved result of the recording equipment is then processed using the algorithm parameters of the sound pickup algorithm used by the recording equipment to obtain the filtered speech. The filtered speech is convolved using the room impulse response of the space where the target acquisition device is located to obtain the impulse convolutional speech of the target acquisition device, and the simulated playback speech is obtained based on the impulse convolutional speech of the target acquisition device.

3. The method of claim 2, the device acoustic parameters further comprising: Frequency response of the target acquisition device; The acoustic algorithm processing parameters also include: the algorithm parameters of the sound pickup algorithm used by the target acquisition device; Based on the pulse convolutional speech from the target acquisition device, simulated replayed speech is obtained, including: Using the frequency response of the target acquisition device, the pulse convolutional speech of the target acquisition device is convolved to obtain the frequency response convolution result of the playback device. Then, using the algorithm parameters of the sound pickup algorithm used by the target acquisition device, the frequency response convolution result of the target acquisition device is processed to obtain the simulated playback speech.

4. The method of claim 2, wherein the frequency response of the recording device includes: The frequency response of the recording unit and the frequency response of the playback unit in the recording device; Using the frequency response of the recording equipment, the pulse convolutional speech of the recording equipment is convolved to obtain the frequency response convolution result of the recording equipment. Then, using the algorithm parameters of the sound pickup algorithm used by the recording equipment, the frequency response convolution result of the recording equipment is processed to obtain filtered speech, including: Using the frequency response of the recording unit in the recording device, the pulse convolution speech of the recording device is convolved to obtain the frequency response convolution result of the recording unit in the recording device. Using the algorithm parameters of the sound pickup algorithm used by the recording device, the frequency response convolution result of the recording unit in the recording device is processed to obtain the speech after algorithm processing by the recording device. The filtered speech is obtained by convolving the speech processed by the recording device with the frequency response of the playback unit in the recording device.

5. The method of claim 2, the device acoustic parameters further comprising: Frequency response of the playback device; Using the algorithm parameters of the sound pickup algorithm employed by the recording equipment, the frequency response convolution result of the recording equipment is processed to obtain filtered speech, including: The frequency response convolution result of the recording device is processed using the algorithm parameters of the sound pickup algorithm used by the recording device to obtain the processed speech. The frequency response of the playback device is used to perform convolution processing on the speech processed by the algorithm to obtain filtered speech.

6. The method of claim 2, wherein the target real speech is convolved using the room impulse response of the space where the recording device is located, comprising: The target real speech is mixed with the simulated environmental noise of the space where the recording device is located to obtain the first mixed speech; The first mixed speech is convolved using the room impulse response of the space where the recording equipment is located.

7. The method as described in claim 2, wherein the filtered speech is convolved using the room impulse response of the space where the target acquisition device is located, comprising: The filtered speech is mixed with the simulated environmental noise of the space where the target acquisition device is located to obtain a second mixed speech; The second mixed speech is convolved using the room impulse response of the space where the target acquisition device is located.

8. The method as described in claim 2, wherein configuring various acoustic parameters in the voice replay attack chain includes: The spatial parameters of the space where the recording equipment is located are sampled to determine the room impulse response of the space where the recording equipment is located, and the spatial parameters of the space where the target acquisition device is located are sampled to determine the room impulse response of the space where the target acquisition device is located. The spatial parameters include: room size, wall sound absorption coefficient, and reverberation time. The identifiers of the devices involved in the voice replay attack chain are sampled, and the frequency response of the devices involved in the voice replay attack chain is queried based on the sampled device identifiers. The control parameters of the pickup algorithm used by the device involved in the speech replay attack link are sampled to determine the algorithm parameters of the pickup algorithm used by the device involved in the speech replay attack link based on the sampled control parameters. The control parameters include at least one of noise suppression level, gain target value and echo cancellation depth.

9. The method as described in claim 2, wherein the frequency response convolution result of the recording device is processed using the algorithm parameters of the sound pickup algorithm employed by the recording device to obtain filtered speech, comprising: The multi-segment filter bank of the constructed recording device is used to filter the frequency response convolution result of the recording device to obtain the cavity filtered speech of the recording device. The multi-segment filter bank of the recording device is used to simulate the influence of the physical cavity structure of the recording device on the sound. The cavity-filtered speech of the recording device is processed using the algorithm parameters of the sound pickup algorithm employed by the recording device to obtain filtered speech.

10. The method as described in claim 3, wherein the frequency response convolution result of the target acquisition device is processed using the algorithm parameters of the sound pickup algorithm employed by the target acquisition device to obtain simulated replayed speech, comprising: Using the constructed multi-segment filter bank of the target acquisition device, the frequency response convolution result of the target acquisition device is filtered to obtain the cavity-filtered speech of the target acquisition device. The multi-segment filter bank of the target acquisition device is used to simulate the influence of the physical cavity structure of the target acquisition device on the sound. Using the algorithm parameters of the sound pickup algorithm employed by the target acquisition device, the cavity-filtered speech of the target acquisition device is processed to obtain simulated playback speech.

11. A replay detection method, applied to a target acquisition device, comprising: Voice is collected in the space where the target acquisition device is located to obtain the voice to be detected. The speech to be detected is input into a pre-trained speech replay detection model to detect whether the speech to be detected is a replay attack speech based on the speech replay detection model. The speech playback detection model is obtained by training the training samples constructed using the method described in any one of claims 1 to 10.

12. The method of claim 11, the target collection device comprising: Smart glasses.

13. A voice replay attack emulation system, the system comprising: Link building module, configuration module, simulation module, data encapsulation module; The link construction module is used to construct a simulated voice replay attack link. The voice replay attack link is used to simulate the replay attack process of recording real voice with a recording device and playing the recorded voice back to the target acquisition device by configuring various acoustic parameters. The acoustic parameters include: device acoustic parameters, environmental acoustic parameters, and acoustic algorithm processing parameters. The device acoustic parameters are used to characterize the acoustic characteristics of the device involved in the voice replay attack link. The environmental acoustic parameters are used to characterize the acoustic characteristics of the space where the device is located. The acoustic algorithm processing parameters are used to represent the algorithm parameters used by the device to process voice internally. The configuration module is used to acquire the target's real speech and configure various acoustic parameters in the speech replay attack chain; The simulation module is used to perform concatenated processing on the target real speech according to the order in which the acoustic parameters are used in the speech replay attack chain, so as to obtain simulated replay speech. The data encapsulation module is used to encapsulate the simulated replayed speech into training samples, which are used to train the speech replay detection model to be trained.

14. An electronic device comprising: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1 to 12 by executing the executable instructions.

15. A computer-readable storage medium having stored thereon computer instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 12.

16. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as claimed in any one of claims 1 to 12.

Images