A method and system for voice liveness detection based on near-field magnetic induction

Abstract

The present invention discloses a method and system for voice liveness detection based on near-field magnetic induction. The method first collects sound data and magnetometer data to detect the distance to the sound source; then preprocesses the magnetometer data; extracts the unique magnetic field change pattern during speech from the preprocessed magnetometer data; and finally determines the authenticity of the voice sample, distinguishing whether it was produced by a real person or a speaker. The present invention utilizes two sub-models based on a convolutional neural network (CNN) and a feature fusion model based on self-attention to extract effective and robust features. This method utilizes the smartphone's inherent magnetometer and microphone, and does not require active signal sensing or the integration of additional hardware. This method does not impose additional burden on the user and maintains its effectiveness under different environmental conditions, setting a new standard for voice liveness detection technology.

Description

Technical Field

[0001] The present invention belongs to the field of cyberspace security technology and relates to a voice liveness detection method and system, and specifically to a voice liveness detection method and system based on near-field magnetic induction. Background Art

[0002] Voice authentication is increasingly being adopted on modern smartphones for sensitive operations, ranging from secure logins to mobile banking transactions. While these interactive systems offer convenience and user-friendliness, the inherent public exposure of human speech introduces significant vulnerabilities. Due to the readily accessible nature of voice data, these systems are particularly susceptible to spoofing attacks, where malicious actors can record, manipulate, and replay voice samples to gain unauthorized access to secure services. These vulnerabilities not only compromise personal privacy but also pose significant risks of financial loss and unauthorized access to sensitive information.

[0003] Research on defending against spoofing attacks has driven advances in voice liveness detection technology, which aims to identify whether a voice originates from a genuine human or an artificial reproduction. Research in this area spans a variety of devices, including smartphones, smart speakers, and wearables. Voice liveness detection on smart speakers uses microphone arrays or specialized sensors (such as wireless sensors and millimeter-wave radar), or tailored liveness detection methods focus on the unique characteristics of wearable devices. Voice liveness detection methods on smartphones can be categorized into three key areas, each leveraging different characteristics of voice interaction. First, methods based on sound field features analyze the acoustic energy generated as sound propagates through air, but require the user to maintain a fixed posture for accurate results. Second, methods based on human characteristics target the biological mechanisms of sound production, but can be uncomfortable due to the need to transmit high-frequency sounds or rely on additional sensors. Third, methods based on speaker features focus on detecting anomalies in speaker output, but struggle to protect against advanced attacks that closely mimic the human voice and require specific user actions, compromising convenience. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a voice liveness detection method and system based on near-field magnetic induction based on the difference in magnetic field changes generated when a person and a speaker make sounds.

[0005] The technical solution adopted by the method of the present invention is: a voice liveness detection method based on near-field magnetic induction, comprising the following steps:

[0006] Step 1: Collect sound data and magnetometer data to detect the distance of the sound source;

[0007] Step 2: Preprocess the magnetometer data;

[0008] Step 3: Extract the unique magnetic field variation pattern during vocalization from the pre-processed magnetometer data;

[0009] Extract feature representations in the time domain and frequency domain respectively, and fuse the time domain feature representation and the frequency domain feature representation to obtain fused features;

[0010] Step 4: Determine the authenticity of the voice sample, distinguishing whether it is spoken by a real person or a speaker.

[0011] Preferably, in step 1, the built-in microphone and magnetometer of the smartphone are used to collect sound data and magnetometer data respectively, perform sound source distance detection, and measure the distance between the sound source and the smartphone; the distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation, and the distances d1 and d2 from the sound source to the two microphones are:

[0012]

[0013] Where E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone.

[0014] Preferably, the specific implementation of step 2 includes the following sub-steps:

[0015] Step 2.1: Denoise the magnetometer data;

[0016] Step 2.2: Use voice activity detection to identify speech segments in the denoised magnetometer data, and segment the denoised magnetometer data according to the speech segments.

[0017] As a preference, in step 2.1, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is used to eliminate noise caused by human motion and other interferences; then the three-axis magnetometer data m=(m x , m y , m z ) is aggregated to obtain the net magnetometer data m ′ ;

[0018]

[0019] Among them, m x (t), m y (t), m y (t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.

[0020] Preferably, in step 2.2, the start point and the end point of each segment are adjusted to obtain a magnetometer data segment containing N sample points, where N is a preset value.

[0021] Preferably, in step 3, a first convolutional neural network model is used to extract time domain feature representation, and a second convolutional neural network model is used to extract frequency domain feature representation;

[0022] The first convolutional neural network model consists of 6 neural network layers, of which the first layer is Conv1D+BN+ReLU, the second layer is Conv1D+BN+ReLU, the third layer is Conv1D+BN+ReLU, the fourth layer is Pooling, the fifth layer is Flatten, and the sixth layer is FC+ReLU;

[0023] The second convolutional neural network model consists of 6 neural network layers, of which the first layer is Conv2D+ReLU, the second layer is Pooling+BN, the third layer is Conv2D+ReLU, the fourth layer is Pooling+BN, the fifth layer is Flatten, and the sixth layer is FC+ReLU;

[0024] Among them, Conv1D represents a one-dimensional convolutional layer, Conv2D represents a two-dimensional convolutional layer, ReLU is a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer;

[0025] After concatenating the time domain feature representation and the frequency domain feature representation, a feature vector is obtained.

[0026] Preferably, in step 3, a feature fusion model based on a self-attention mechanism is used to fuse the time domain feature representation and the frequency domain feature representation to obtain a fused feature;

[0027] The feature fusion model based on the self-attention mechanism uses the SE block as the self-attention mechanism to learn a weight vector in the range of [0, 1]. The SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers. The obtained weight vector is used to scale the spliced ​​time domain feature representation and frequency domain feature representation to obtain a fused feature.

[0028] Preferably, a two-class classifier model is used to determine the authenticity of the speech sample and distinguish whether it is uttered by a real person or a speaker;

[0029] The two-class classifier model consists of three neural network layers, including two fully connected layers and a sigmoid layer. The output of the sigmoid layer represents the probability that the speech sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human; otherwise, it is considered to come from a speaker.

[0030] The technical solution adopted by the system of the present invention is: a voice liveness detection system based on near-field magnetic induction, comprising:

[0031] one or more processors;

[0032] A storage device is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the voice liveness detection method based on near-field magnetic induction.

[0033] Compared with the prior art, the beneficial effects of the present invention include:

[0034] (1) The present invention uses the inherent magnetometer and microphone of the smartphone, does not require active signal sensing or integration of additional hardware, and can achieve voice liveness detection without actively sending high-frequency signals or integrating additional hardware.

[0035] (2) The present invention does not bring additional burden to users, has few operational restrictions, and maintains its effectiveness under different environmental conditions, setting a new standard for voice liveness detection technology.

[0036] (3) The present invention uses the difference in magnetic field changes generated when a person speaks and a speaker to perform voice liveness detection, effectively resisting deception attacks.

[0037] (4) The present invention utilizes two sub-models based on convolutional neural networks (CNNs) and a feature fusion model based on self-attention to extract effective and robust features, and uses supervised contrastive learning to achieve user independence, device independence, and content independence. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The technical solution of the present invention is further illustrated below using embodiments and specific implementation methods. In addition, some drawings are also used in the process of illustrating the technical solution. For those skilled in the art, other drawings and the intention of the present invention can be obtained based on these drawings without making any creative efforts.

[0039] Figure 1 Schematic diagram of a method flow in an embodiment of the present invention.

[0040] Figure 2 This is a structural diagram of the first and second convolutional neural network models of an embodiment of the present invention.

[0041] Figure 3 This is a structural diagram of a feature fusion model based on the self-attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0043] Please see Figure 1 The present invention provides a method for voice liveness detection based on near-field magnetic induction, comprising the following steps:

[0044] Step 1: The smartphone collects sound data and magnetometer data simultaneously, uses the sound data for voice authentication, and uses the magnetometer data for voice liveness detection. Taking into account the inherent attenuation characteristics of magnetic signals, this embodiment performs sound source distance detection to measure the distance between the sound source and the smartphone.

[0045] In one embodiment, the distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation, and the distances d1 and d2 from the sound source to the two microphones are:

[0046]

[0047] Wherein, E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone. The goal of this embodiment is to ensure that the distance of the sound source is within a predefined threshold determined through experiments.

[0048] Step 2: Preprocess the magnetometer data;

[0049] In one embodiment, the specific implementation of step 2 includes the following sub-steps:

[0050] Step 2.1: Denoise the magnetometer data to eliminate noise and reduce the influence of the Earth's magnetic field;

[0051] In one embodiment, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is first applied to eliminate noise caused by human motion and other interferences; considering the influence of the geomagnetic field, the collected three-axis magnetometer data m = (m x , m y , m z ) has geographic spatial dependence. To mitigate the influence of location, the data are aggregated on three axes to obtain the net magnetometer data m ′ , the net magnetometer data m at each moment ′ (t) can be calculated by the following formula:

[0052]

[0053] Among them, m x (t), m y (t), m y(t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.

[0054] Step 2.2: Use voice activity detection to identify speech segments in the denoised magnetometer data, and segment the denoised magnetometer data according to the speech segments.

[0055] In one embodiment, since the magnetic field changes caused by the speaker are small, this embodiment uses speech segmentation to assist in the segmentation of the net magnetometer data. First, voice activity detection is used to identify speech segments. Since the magnetometer data and speech data are synchronized, segmentation of the net magnetometer data is obtained. To ensure that each segment covers the entire speech, the start and end points of each segment are adjusted to obtain a magnetometer data segment containing 100 sample points.

[0056] Step 3: Extract the unique magnetic field variation pattern during vocalization from the pre-processed magnetometer data;

[0057] In one embodiment, the specific implementation of step 3 includes the following sub-steps:

[0058] Step 3.1: Extract the envelope of the net magnetometer data, use short-time Fourier transform to extract the two-dimensional spectrum of amplitude and phase features, use the first convolutional neural network model to extract the time domain feature representation, and use the second convolutional neural network model to extract the frequency domain feature representation, see Figure 2 ;

[0059] In one embodiment, the first convolutional neural network model structure is shown in Table 1 below:

[0060] Table 1

[0061]

[0062]

[0063] In one embodiment, the second convolutional neural network model structure is shown in Table 2 below:

[0064] Table 2

[0065] Layer number Layer Type Output shape 1 Conv2D+ReLU 15×67×16 2 Pooling+BN 7×32×16 3 Conv2D+ReLU 5×31×32 4 Pooling+BN 2×15×32 5 Flatten 960 6 FC+ReLU 64

[0066] Among them, Conv1D represents a one-dimensional convolution layer, Conv2D represents a two-dimensional convolution layer, ReLU represents a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer. In this embodiment, the feature representations extracted by the two sub-models are concatenated to obtain a 128-dimensional feature vector.

[0067] Step 3.2: Use the feature fusion model based on the self-attention mechanism to recalibrate the feature representation, selectively emphasize the more important features to obtain the final fused features; use supervised contrastive learning to achieve user-independence, device-independence, and content-independence.

[0068] Please see Figure 3 In one embodiment, a feature fusion model based on a self-attention mechanism uses an SE block as a weight vector with a learning range of [0,1] for the self-attention mechanism; an SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers; the output features of the sub-model ("the concatenated time domain feature representation and the frequency domain feature representation") are scaled using the obtained weight vector to obtain a fused feature, and higher weights are given to features with greater information content to obtain a 128-dimensional feature vector.

[0069] Step 4: Using a two-class classifier model, determine the authenticity of the voice sample and set a pre-set threshold to distinguish whether it is spoken by a real person or a speaker.

[0070] In one embodiment, the two-class classifier model consists of three neural network layers, including two fully connected layers and one sigmoid layer; the classifier is trained using binary cross entropy as the loss function. Since voice liveness detection is a two-classification task, the output of the sigmoid layer represents the probability that the voice sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human, otherwise it is considered to come from a speaker; in this embodiment, the threshold is set to 0.5.

[0071] The structure of the two-class classifier model is shown in Table 3 below:

[0072] Table 3

[0073] Layer number Layer Type Output shape 1 FC 64 2 FC 1 3 Sigmoid 1

[0074] This embodiment also provides a voice liveness detection system based on near-field magnetic induction, including:

[0075] one or more processors;

[0076] A storage device is used to store one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement the voice liveness detection method based on near-field magnetic induction.

[0077] It should be understood that the embodiments described above are only some of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features of the various embodiments or individual embodiments provided by the present invention may be arbitrarily combined with each other to form a feasible technical solution. Such combination is not restricted by the order of steps and / or structural composition mode, but must be based on the ability of ordinary technicians in this field to implement it. When the combination of technical solutions is mutually inconsistent or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0078] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of protection of the patent of the present invention. Under the guidance of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested by the present invention shall be based on the attached claims.

Claims

1. A method for voice liveness detection based on near-field magnetic induction, characterized in that: The following steps are involved: Step 1: Use the smartphone’s built-in microphone and magnetometer to collect sound data and magnetometer data respectively to detect the distance of the sound source; Step 2: Preprocess the magnetometer data; The specific implementation of step 2 includes the following sub-steps: Step 2.1: Denoise the magnetometer data; Step 2.2: Use voice activity detection to identify the voice segments of the denoised magnetometer data, and segment the denoised magnetometer data according to the voice segments. Step 3: Extract the magnetic field change pattern during vocalization from the preprocessed magnetometer data; The convolutional neural network model is used to extract feature representations in the time domain and frequency domain respectively, and the fusion feature is obtained by fusing the time domain feature representation and the frequency domain feature representation; Among them, the feature fusion model based on the self-attention mechanism is used to fuse the time domain feature representation and the frequency domain feature representation to obtain the fusion feature; The feature fusion model based on the self-attention mechanism uses the SE block as the self-attention mechanism to learn a weight vector in the range of [0, 1]. The SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers. The obtained weight vector is used to scale the spliced ​​time domain feature representation and frequency domain feature representation to obtain a fused feature. Step 4: Determine the authenticity of the voice sample, distinguishing whether it is spoken by a real person or a speaker.

2. The method for voice liveness detection based on near-field magnetic induction according to claim 1, wherein: In step 1, the built-in microphone and magnetometer of the smartphone are used to collect sound data and magnetometer data, respectively, to detect the sound source distance and measure the distance between the sound source and the smartphone. The distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation. The distances d1 and d2 from the sound source to the two microphones are: Where E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone.

3. The method for voice liveness detection based on near-field magnetic induction according to claim 1, wherein: In step 2.1, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is used to eliminate the noise caused by human motion and other interferences; then the three-axis magnetometer data m = (m x , m y , m z ) is aggregated to obtain the net magnetometer data m ′ ; Among them, m x (t), m y (t), m y (t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.

4. The method for voice liveness detection based on near-field magnetic induction according to claim 1, wherein: In step 2.2, the start and end points of each segment are adjusted to obtain a magnetometer data segment containing N sample points, where N is a preset value.

5. The method for voice liveness detection based on near-field magnetic induction according to claim 1, wherein: In step 3, the first convolutional neural network model is used to extract the time domain feature representation, and the second convolutional neural network model is used to extract the frequency domain feature representation; The first convolutional neural network model consists of 6 neural network layers, of which the first layer is Conv1D+BN+ReLU, the second layer is Conv1D+BN+ReLU, the third layer is Conv1D+BN+ReLU, the fourth layer is Pooling, the fifth layer is Flatten, and the sixth layer is FC+ReLU; The second convolutional neural network model consists of 6 neural network layers, of which the first layer is Conv2D+ReLU, the second layer is Pooling+BN, the third layer is Conv2D+ReLU, the fourth layer is Pooling+BN, the fifth layer is Flatten, and the sixth layer is FC+ReLU; Among them, Conv1D represents a one-dimensional convolutional layer, Conv2D represents a two-dimensional convolutional layer, ReLU is a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer; After concatenating the time domain feature representation and the frequency domain feature representation, a feature vector is obtained.

6. The method for voice liveness detection based on near-field magnetic induction according to any one of claims 1 to 5, characterized in that: A two-class classifier model is used to determine the authenticity of the speech sample and distinguish whether it is produced by a real person or a speaker; The two-class classifier model consists of three neural network layers, including two fully connected layers and a sigmoid layer. The output of the sigmoid layer represents the probability that the speech sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human; otherwise, it is considered to come from a speaker.

7. A voice liveness detection system based on near-field magnetic induction, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the voice liveness detection method based on near-field magnetic induction as described in any one of claims 1 to 6.

Images