Voice living body detection method and system based on near-field magnetic induction
Through the built-in microphone and magnetometer of smartphones combined with convolutional neural network and self-attention mechanism, the near-field magnetic induction characteristics are extracted, which solves the problem of existing speech live detection being vulnerable to fraudulent attacks, and achieves efficient and user-friendly speech live detection. Resist the risks of unauthorized access and economic losses.
Patent Information
- Application Number
- CN202410875604.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-02
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-07-02
AI Technical Summary
Existing voice live detection technologies are vulnerable to fraudulent attacks, especially due to the vulnerability caused by the disclosure of human voice, which is difficult to effectively resist the risks of unauthorized access and economic losses.
Using a near-field magnetic induction method, data is collected using the built-in microphone and magnetometer of the smartphone, features are extracted through convolutional neural network and self-attention mechanism, and sounds emitted by real humans and speakers are distinguished, and voice live detection is achieved.
It does not require additional hardware and user operation restrictions, effectively resists spoofing attacks, provides user-friendly and efficient voice live detection in different environments, and achieves user-independent, device-independent and content-independent detection effects.
Smart Images

Figure CN118960939A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of cyberspace security technology, and relates to a voice liveness detection method and system, and specifically to a voice liveness detection method and system based on near-field magnetic induction. Background Art
[0002] Voice authentication is increasingly being adopted on modern smartphones to perform sensitive operations, ranging from secure logins to mobile banking transactions. Despite the convenience and user-friendliness of interactive systems, the inherent public exposure of human voice introduces significant vulnerabilities. Due to the easily accessible nature of voice data, these systems are particularly susceptible to spoofing attacks, where malicious actors can record, manipulate, and replay voice samples to gain unauthorized access to secure services. These vulnerabilities not only compromise individual privacy, but also pose a risk of significant financial losses and unauthorized access to sensitive information.
[0003] Research on defense against spoofing attacks has led to advances in voice liveness detection, which aims to identify whether a voice comes from a real human or is artificially reproduced. Research in this area covers a variety of devices, such as smartphones, smart speakers, and wearables. Voice liveness detection on smart speakers is done using microphone arrays or specialized sensors such as wireless sensors and millimeter-wave radars, or liveness detection methods are tailored to the unique characteristics of wearable devices. Voice liveness detection methods on smartphones can be divided into three key areas, each of which exploits different voice interaction characteristics. First, methods based on sound field features analyze the acoustic energy generated when sound propagates in the air, but require users to maintain a fixed posture to obtain accurate results. Second, methods based on human features target the biological mechanism of sound production, but may cause discomfort due to the need to send high-frequency sounds or rely on additional sensors. Third, methods based on speaker features focus on detecting anomalies in speaker output, but are difficult to resist advanced attacks that closely mimic human voices and require specific user operations, compromising convenience. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides a voice liveness detection method and system based on near-field magnetic induction based on the difference in magnetic field changes generated when a person and a speaker make sounds.
[0005] The technical solution adopted by the method of the present invention is: a voice liveness detection method based on near-field magnetic induction, comprising the following steps:
[0006] Step 1: Collect sound data and magnetometer data to detect the distance of the sound source;
[0007] Step 2: Preprocess the magnetometer data;
[0008] Step 3: Extract the unique magnetic field change pattern during vocalization from the pre-processed magnetometer data;
[0009] Extract feature representations in the time domain and frequency domain respectively, and fuse the time domain feature representation and the frequency domain feature representation to obtain fused features;
[0010] Step 4: Determine the authenticity of the voice sample, whether it is spoken by a real person or a speaker.
[0011] Preferably, in step 1, the built-in microphone and magnetometer of the smartphone are used to collect sound data and magnetometer data respectively, to detect the distance of the sound source, and to measure the distance between the sound source and the smartphone; the distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation, and the distances d1 and d2 from the sound source to the two microphones are:
[0012]
[0013] Where E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone.
[0014] Preferably, the specific implementation of step 2 includes the following sub-steps:
[0015] Step 2.1: De-noise the magnetometer data;
[0016] Step 2.2: Use voice activity detection to identify the voice segments of the denoised magnetometer data, and segment the denoised magnetometer data according to the voice segment segments.
[0017] Preferably, in step 2.1, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is used to eliminate noise caused by human motion and other interferences; then the three-axis magnetometer data m=(m x , m y , m z ) to obtain the net magnetometer data m ′ ;
[0018]
[0019] Among them, m x (t), m y (t), m y (t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.
[0020] Preferably, in step 2.2, the start point and the end point of each segment are adjusted to obtain a magnetometer data segment containing N sample points, wherein N is a preset value.
[0021] Preferably, in step 3, a first convolutional neural network model is used to extract a time domain feature representation, and a second convolutional neural network model is used to extract a frequency domain feature representation;
[0022] The first convolutional neural network model consists of 6 neural network layers, wherein the first layer is Conv1D+BN+ReLU, the second layer is Conv1D+BN+ReLU, the third layer is Conv1D+BN+ReLU, the fourth layer is Pooling, the fifth layer is Flatten, and the sixth layer is FC+ReLU;
[0023] The second convolutional neural network model consists of 6 neural network layers, wherein the first layer is Conv2D+ReLU, the second layer is Pooling+BN, the third layer is Conv2D+ReLU, the fourth layer is Pooling+BN, the fifth layer is Flatten, and the sixth layer is FC+ReLU;
[0024] Among them, Conv1D represents a one-dimensional convolution layer, Conv2D represents a two-dimensional convolution layer, ReLU is a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer;
[0025] After concatenating the time domain feature representation and the frequency domain feature representation, a feature vector is obtained.
[0026] Preferably, in step 3, a feature fusion model based on a self-attention mechanism is used to fuse the time domain feature representation and the frequency domain feature representation to obtain a fused feature;
[0027] The feature fusion model based on the self-attention mechanism uses the SE block as the self-attention mechanism and learns a weight vector with a range of [0,1]. The SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers. The obtained weight vector is used to scale the spliced time domain feature representation and frequency domain feature representation to obtain a fused feature.
[0028] Preferably, a two-class classifier model is used to determine the authenticity of the speech sample and distinguish whether it is uttered by a real person or a speaker;
[0029] The two-class classifier model consists of three neural network layers, including two fully connected layers and one sigmoid layer; the output of the sigmoid layer represents the probability that the speech sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human, otherwise it is considered to come from a speaker.
[0030] The technical solution adopted by the system of the present invention is: a voice liveness detection system based on near-field magnetic induction, comprising:
[0031] one or more processors;
[0032] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the voice liveness detection method based on near-field magnetic induction.
[0033] Compared with the prior art, the beneficial effects of the present invention include:
[0034] (1) The present invention uses the inherent magnetometer and microphone on the smartphone, does not require active signal sensing or integration of additional hardware, and can achieve voice liveness detection without actively sending high-frequency signals or integrating additional hardware.
[0035] (2) The present invention does not bring additional burden to users, has few operational restrictions, and maintains its effectiveness under different environmental conditions, setting a new standard for voice liveness detection technology.
[0036] (3) The present invention uses the difference in magnetic field changes generated when a person and a speaker speak to perform voice liveness detection, effectively resisting deception attacks.
[0037] (4) The present invention utilizes two sub-models based on convolutional neural network (CNN) and a feature fusion model based on self-attention to extract effective and robust features, and utilizes supervised contrastive learning to achieve user independence, device independence, and content independence. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.
[0039] Figure 1 The figure is a schematic diagram of a method flow of an embodiment of the present invention.
[0040] Figure 2 This is a structural diagram of the first and second convolutional neural network models of an embodiment of the present invention.
[0041] Figure 3 This is a structural diagram of a feature fusion model based on the self-attention mechanism in an embodiment of the present invention. DETAILED DESCRIPTION
[0042] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.
[0043] Please see Figure 1 The present embodiment provides a method for voice liveness detection based on near-field magnetic induction, comprising the following steps:
[0044] Step 1: The smartphone collects sound data and magnetometer data simultaneously, uses the sound data for voice authentication, and uses the magnetometer data for voice liveness detection; considering the inherent attenuation characteristics of magnetic signals, this embodiment performs sound source distance detection to measure the distance between the sound source and the smartphone.
[0045] In one implementation, the distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation, and the distances d1 and d2 from the sound source to the two microphones are:
[0046]
[0047] In the formula, E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone. The goal of this embodiment is to ensure that the distance of the sound source is within a predefined threshold determined by experiments.
[0048] Step 2: Preprocess the magnetometer data;
[0049] In one embodiment, the specific implementation of step 2 includes the following sub-steps:
[0050] Step 2.1: De-noise the magnetometer data to eliminate noise and reduce the influence of the geomagnetic field;
[0051] In one embodiment, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is first applied to eliminate the noise caused by human motion and other interferences; considering the influence of the geomagnetic field, the collected three-axis magnetometer data m = (m x , m y , m z ) has geospatial dependence. To mitigate the influence of location, the data are aggregated on three axes to obtain the net magnetometer data m ′ , the net magnetometer data m at each moment ′ (t) can be calculated by the following formula:
[0052]
[0053] Among them, m x (t), m y (t), m y(t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.
[0054] Step 2.2: Use voice activity detection to identify the voice segments of the denoised magnetometer data, and segment the denoised magnetometer data according to the voice segment segments.
[0055] In one embodiment, since the magnetic field change caused by the speaker is tiny, this embodiment uses speech segmentation to assist in the segmentation of net magnetometer data; first, voice activity detection is used to identify voice segments, and since the magnetometer data and voice data are synchronized, segmentation of the net magnetometer data is obtained; in order to ensure that each segment covers the entire voice, the start and end points of each segment are adjusted to obtain a magnetometer data segment containing 100 sample points.
[0056] Step 3: Extract the unique magnetic field change pattern during vocalization from the pre-processed magnetometer data;
[0057] In one implementation, the specific implementation of step 3 includes the following sub-steps:
[0058] Step 3.1: Extract the envelope of the net magnetometer data, use short-time Fourier transform to extract the two-dimensional spectrum of amplitude and phase features, use the first convolutional neural network model to extract the time domain feature representation, and use the second convolutional neural network model to extract the frequency domain feature representation, see Figure 2 ;
[0059] In one embodiment, the first convolutional neural network model structure is shown in Table 1 below:
[0060] Table 1
[0061]
[0062]
[0063] In one embodiment, the second convolutional neural network model structure is shown in Table 2 below:
[0064] Table 2
[0065] Layer Number Layer Type Output shape 1 Conv2D+ReLU 15×67×16 2 Pooling+BN 7×32×16 3 Conv2D+ReLU 5×31×32 4 Pooling+BN 2×15×32 5 Flatten 960 6 FC+ReLU 64
[0066] Among them, Conv1D represents a one-dimensional convolution layer, Conv2D represents a two-dimensional convolution layer, ReLU is a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer; this embodiment concatenates the feature representations extracted by the two sub-models to obtain a 128-dimensional feature vector.
[0067] Step 3.2: Use the feature fusion model based on the self-attention mechanism to recalibrate the feature representation, selectively emphasize the more important features to obtain the final fused features; use supervised contrastive learning to achieve user-independence, device-independence, and content-independence.
[0068] Please see Figure 3 In one embodiment, a feature fusion model based on a self-attention mechanism uses an SE block as a weight vector with a learning range of [0,1] for the self-attention mechanism; an SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers; the output features of the sub-model ("the concatenated time domain feature representation and the frequency domain feature representation") are scaled using the obtained weight vector to obtain fused features, and higher weights are given to features with greater information content to obtain a 128-dimensional feature vector.
[0069] Step 4: Using a two-class classifier model, determine the authenticity of the voice sample and set a pre-set threshold to distinguish whether it is uttered by a real person or a speaker.
[0070] In one embodiment, the two-class classifier model consists of three neural network layers, including two fully connected layers and one sigmoid layer; the classifier is trained using binary cross entropy as the loss function. Since voice liveness detection is a two-classification task, the output of the sigmoid layer represents the probability that the voice sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human, otherwise it is considered to be coming from a speaker; in this embodiment, the threshold is set to 0.5.
[0071] The structure of the two-class classifier model is shown in Table 3 below:
[0072] Table 3
[0073] Layer Number Layer Type Output shape 1 FC 64 2 FC 1 3 Sigmoid 1
[0074] This embodiment also provides a voice liveness detection system based on near-field magnetic induction, including:
[0075] one or more processors;
[0076] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the voice liveness detection method based on near-field magnetic induction.
[0077] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.
[0078] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.
Claims
1. A voice liveness detection method based on near-field magnetic induction, characterized in that: The following steps are involved: Step 1: Collect sound data and magnetometer data to detect the distance of the sound source; Step 2: Preprocess the magnetometer data; Step 3: Extract the unique magnetic field change pattern during vocalization from the pre-processed magnetometer data; Extract feature representations in the time domain and frequency domain respectively, and fuse the time domain feature representation and the frequency domain feature representation to obtain fused features; Step 4: Determine the authenticity of the voice sample, whether it is spoken by a real person or a speaker.
2. The method for voice liveness detection based on near-field magnetic induction according to claim 1, characterized in that: In step 1, the built-in microphone and magnetometer of the smartphone are used to collect sound data and magnetometer data respectively, perform sound source distance detection, and measure the distance between the sound source and the smartphone; the distance difference Δd between the sound source and the two microphones is calculated using generalized cross-correlation, and the distances d1 and d2 from the sound source to the two microphones are: Where E1 represents the energy of the signal received by the first microphone, and E2 represents the energy of the signal received by the second microphone.
3. The method for voice liveness detection based on near-field magnetic induction according to claim 1, characterized in that: The specific implementation of step 2 includes the following sub-steps: Step 2.1: De-noise the magnetometer data; Step 2.2: Use voice activity detection to identify the voice segments of the denoised magnetometer data, and segment the denoised magnetometer data according to the voice segment segments.
4. The method for voice liveness detection based on near-field magnetic induction according to claim 3 is characterized in that: In step 2.1, a Butterworth high-pass filter with a cutoff frequency of 5 Hz is used to eliminate the noise caused by human motion and other interferences; then the three-axis magnetometer data m = (m x , m y , m z ) to obtain the net magnetometer data m ′ ; Among them, m x (t), m y (t), m y (t) represents the x-, y-, and z-axis data collected by the three-axis magnetometer at time t.
5. The method for voice liveness detection based on near-field magnetic induction according to claim 3 is characterized in that: In step 2.2, the start and end points of each segment are adjusted to obtain a magnetometer data segment containing N sample points, where N is a preset value.
6. The method for voice liveness detection based on near-field magnetic induction according to claim 1, characterized in that: In step 3, the first convolutional neural network model is used to extract the time domain feature representation, and the second convolutional neural network model is used to extract the frequency domain feature representation; The first convolutional neural network model consists of 6 neural network layers, wherein the first layer is Conv1D+BN+ReLU, the second layer is Conv1D+BN+ReLU, the third layer is Conv1D+BN+ReLU, the fourth layer is Pooling, the fifth layer is Flatten, and the sixth layer is FC+ReLU; The second convolutional neural network model consists of 6 neural network layers, wherein the first layer is Conv2D+ReLU, the second layer is Pooling+BN, the third layer is Conv2D+ReLU, the fourth layer is Pooling+BN, the fifth layer is Flatten, and the sixth layer is FC+ReLU; Among them, Conv1D represents a one-dimensional convolution layer, Conv2D represents a two-dimensional convolution layer, ReLU is a linear rectification activation function, BN represents a batch normalization layer, Pooling represents a pooling layer, Flatten represents a Flatten layer, and FC represents a fully connected layer; After concatenating the time domain feature representation and the frequency domain feature representation, a feature vector is obtained.
7. The method for voice liveness detection based on near-field magnetic induction according to claim 1, characterized in that: In step 3, a feature fusion model based on a self-attention mechanism is used to fuse the time domain feature representation and the frequency domain feature representation to obtain a fused feature; The feature fusion model based on the self-attention mechanism uses the SE block as the self-attention mechanism and learns a weight vector with a range of [0,1]. The SE block consists of three neural network layers, including a global average pooling layer and two fully connected layers. The obtained weight vector is used to scale the spliced time domain feature representation and frequency domain feature representation to obtain a fused feature.
8. The method for voice liveness detection based on near-field magnetic induction according to any one of claims 1 to 7, characterized in that: A two-class classifier model is used to determine the authenticity of the speech sample and distinguish whether it is produced by a real person or a speaker; The two-class classifier model consists of three neural network layers, including two fully connected layers and one sigmoid layer; the output of the sigmoid layer represents the probability that the speech sample belongs to a human. If the probability exceeds a set threshold, the sample is classified as coming from a real human, otherwise it is considered to come from a speaker.
9. A voice liveness detection system based on near-field magnetic induction, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the voice liveness detection method based on near-field magnetic induction as described in any one of claims 1 to 8.
Citation Information
Patent Citations
Three-dimensional electromagnetic field model generation method based on physical layout
CN105824995A
Voice recognition method and device, storage medium and electronic equipment
CN110459204A
119 alarm system based on voice recognition technology
CN115620704A
Real machine user detection method based on mobile equipment magnetic field sensor and IMU (Inertial Measurement Unit) sensor
CN116244621A
Relay device and operating method of the relay device
KR102308196B1
