Voice analysis identity authentication method based on sound field reconstruction

Abstract

The application discloses a speech analysis identity authentication method based on sound field reconstruction, which comprises distance sensing, sound field reconstruction, sound field extraction, and model training and inference. The distance sensing is achieved by emitting a chirp signal through a loudspeaker and receiving the signal, and the distance from the user to the mobile phone is obtained by using a cross-correlation method. The sound field reconstruction is achieved by establishing a distance-dependent impulse response database, obtaining the corresponding transfer function according to the measured distance, and then reconstructing the sound field at the verification position to the sound field at the registration position. The sound field extraction is achieved by separately processing the signals of the two sound channels of the speech signal after the sound field reconstruction, and extracting the field texture. The model training and inference are achieved by using the registration field texture and the reconstructed field texture in the verification stage to construct a speech authentication model. The method can effectively solve the problem that the original sound field method is sensitive to distance when verifying the speaker, and the user does not need to maintain a fixed distance during verification as during registration.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence voice assistant security technology, specifically relating to a voice analysis identity authentication method based on sound field reconstruction. Background Technology

[0002] Voice authentication systems verify users by extracting their voice features for feature training and matching. This is a widely used method of identity verification. With low hardware requirements, low cost, and high user-friendliness, it has gradually become a mainstream personal authentication method and is widely used on various smart devices.

[0003] Existing voice authentication systems based on sound field and field pattern are one such example. Sound field features are independent of the spoken content; they not only preserve the sound information of the speech signal itself but also incorporate relevant physiological identity information such as the user's vocal tract, mouth, head, and torso. Therefore, this system can detect voice spoofing attacks based on sound field and field pattern information, and is effectively applied in the fields of speaker authentication and voice recognition.

[0004] However, voice authentication systems based on sound field and field texture are highly sensitive to the distance between the user and the microphone. Significant differences in sound field characteristics occur when the microphone is placed close to the mouth or slightly further away. In practical applications, this system requires a fixed distance between the user and the microphone for each use, which greatly affects its convenience and user-friendliness. Summary of the Invention

[0005] To address the shortcomings of the aforementioned background technology, this invention provides a voice analysis authentication method based on sound field reconstruction. This method improves upon existing voice authentication systems based on sound field and field texture, effectively solves the problem of sound field sensitivity to distance, and enhances the robustness and user-friendliness of the system.

[0006] This invention is achieved using the following technical solution:

[0007] Step 1: Distance Perception

[0008] Step 1.1: Use the phone's speaker to emit a ranging signal and use the phone's bottom microphone to receive the echo signal (i.e., the signal that returns after the ranging signal passes through a person);

[0009] Step 1.2: Use a Butterworth bandpass filter to remove background noise from the echo signal;

[0010] Step 1.3: Locate the direct path and echo path using the time-domain image of the echo signal. Let Δt represent the time difference between the echo path and the direct path. The distance between the user and the mobile phone can be determined by... The calculation shows that c is the speed of light.

[0011] Step 2: Sound Field Reconstruction

[0012] Step 2.1: Generate a constant pulse s(t) using a stable sound source such as a high-fidelity speaker, and receive the signal at different distances from the user using the top and bottom microphones of the smartphone, denoted as s. k (d,t),k∈[1,2], let h be... k (d,t),k∈[1,2] represents the impulse responses of the top and bottom microphones, which can be expressed by the formula s k (d,t)= h k (d,t)*s(t) is deconvolved to obtain the signal, with a 1cm interval between each two received signals, and h is saved at different distances. k (d,t) are stored to form a distance-related impulse response database, where d is the distance and t is the time;

[0013] Step 2.2: When a user registers the sound field, the distance de between the sound source (user) and the mobile phone is obtained through distance perception. By querying the impulse response database, the impulse response h during the registration phase can be obtained. k (de,t); Similarly, during user authentication, the distance dv between the user and the mobile phone can be obtained, and the transfer function h during authentication can be obtained by querying the impulse response database. k (dv,t);

[0014] Step 2.3: Combine the received voice signal from the user during the verification phase with the impulse response during verification and the inverse impulse response during registration. By performing convolutions sequentially, the reconstructed signal x can be obtained. ′ (t), the specific formula is as follows:

[0015]

[0016] Step 3: Sound Field Extraction

[0017] Step 3.1: Remove noise and silent segments from the reconstructed signal;

[0018] Step 3.2: Perform short-time Fourier transform processing on the two channel signals respectively; use the formula The sound field at frequency f is represented by the logarithm of the sound pressure level ratio between the two channels of the reconstructed signal. Then, the sound field in a given frame is expressed as follows:

[0019] S(p1,p2)=[Sr(p1,p2,f1),Sr(p1,p2,f2),...,Sr(p1,p2,fn)]

[0020] Where p1 and p2 are the positions of the two microphones, f is the frequency, and n is the total number of frequency dimensions;

[0021] For the sound field of the reconstructed signal, the n-dimensional sound field feature vector SFF(p1,p2) is obtained by long-time average normalization:

[0022]

[0023] Where L is the number of frames in the reconstructed signal; S i (p1,p2) represents the sound field in the i-th frame;

[0024] Step 4: Model Training and Inference

[0025] Step 4.1: User registration. A Gaussian Mixture Model (GMM) is used to fit the user's sound field feature vector SFF(p1,p2) to form the corresponding speaker model. The registered user emits several speech signals. An n-dimensional sound field feature vector SFF(p1,p2) is extracted from the registered user's speech signals. A Gaussian Mixture Model can be used to model this vector. For SFF(p1,p2), the mixing density p(SFF|λ) of the registered user s is... s ) is defined as

[0026]

[0027] Where, λ s This represents the speaker model obtained by modeling registered user s using a Gaussian mixture model; M represents the number of Gaussian components. This represents the j-th Gaussian component of registered user s. The mixed weights; the j-th Gaussian component of registered user s The calculation formula is as follows:

[0028]

[0029] Where μ j It is the average vector of the n-dimensional sound field eigenvectors; Let be the covariance matrix of the n-dimensional sound field eigenvectors;

[0030] The model parameters are estimated and converged using the Iterative Expectation-Maximization (EM) algorithm.

[0031] Step 4.2: In the verification phase, the trained Gaussian mixture model is used to process the n-dimensional sound field feature vector of the speaker's reconstructed signal to obtain the similarity between the speaker and the registered user. If the similarity exceeds a predefined threshold, the speaker's identity authentication passes; otherwise, the speaker's identity authentication fails and is rejected.

[0032] The beneficial effects of this invention are:

[0033] This invention proposes a voice analysis authentication method based on sound field reconstruction. This method uses sound wave ranging and impulse response to reconstruct the sound field from the verification stage to the registration stage, ensuring the consistency between the registration and verification stages. This eliminates the need for users to maintain a fixed distance. This invention does not require any additional hardware costs and can complete the sound field reconstruction using two existing microphones and speakers. It solves the distance sensitivity problem of the original sound field and field texture voice authentication system and can be effectively applied to scenarios such as liveness detection that originally required a fixed location. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating a specific implementation of the speech analysis-based identity authentication method based on sound field reconstruction of the present invention;

[0035] Figure 2 This is a physical representation diagram of the direct path and the echo path;

[0036] Figure 3 These are sample images of the ranging signal and the received signal;

[0037] Figure 4 It is an impulse response diagram at a certain distance;

[0038] Figure 5 This is a flowchart of sound field reconstruction;

[0039] Figure 6 This is a comparison diagram of the sound field between the reconstructed signal and the registered signal. Detailed Implementation

[0040] To better understand the technical solution of the present invention, the present invention will now be further described in conjunction with the accompanying drawings:

[0041] Figure 1 The flowchart shown is a process for the method of the present invention. Before implementation, it is necessary to construct an impulse response database. In actual implementation, there are two cases: registration and verification. During registration, distance perception, sound field extraction, and speaker model training are required. During verification, distance perception, sound field reconstruction, sound field extraction, and model inference are required.

[0042] Construction of the impulse response database:

[0043] Step 1: Acquisition of Impulse Response. Many methods exist for acquiring microphone impulse responses from existing datasets. Some transient sounds, such as clapping and gunshots, are widely used as sound sources due to their convenience and effectiveness. However, maintaining the consistency of the impulses generated by clapping and gunshots is impossible, while the impulses generated by stable sound sources such as high-fidelity loudspeakers are more constant. This invention uses a JBL loudspeaker as the sound source, emitting an exponential sinusoidal scanning signal, the non-periodic deconvolution form of which is...

[0044]

[0045] It can eliminate artifacts caused by noise, the nonlinear characteristics of the speaker, and time variance. The specific scanning signal is selected from 0 to 48 kHz, lasting 4 seconds. s(t) is a constant pulse, ω1 and ω2 are the start and end frequencies of s(t), T is the period, and e is the natural logarithm.

[0046] Step 2: Constructing a database of distance-related transfer functions. Signals received using the top and bottom microphones of a smartphone at different user phone distances (1cm interval between adjacent measurements) are denoted as s. k (d,t),k∈[1,2].

[0047] Set h k (d,t),k∈[1,2] represents the impulse responses of the top and bottom microphones, which can be expressed by the formula s ′ k (d,t)=h k (d,t)*s(t) is deconvolved to obtain h at different distances. k (d,t) are stored to build a database of distance-related transfer functions. Figure 4 The data represents the pulse signal received at a certain distance, including the direct sound, early reflection, and reverberation tail. This invention only captures the first 3ms to exclude the effects of reflection and reverberation caused by different spatial sizes.

[0048] During the registration phase:

[0049] Step 1: Distance Perception

[0050] Step 1.1: Design of the transmitted signal. Figure 2 This is the main application scenario of this invention. When a smartphone is held horizontally in front of a person's mouth, the received signal will have two paths: a direct path and an echo path. The signal design for echo ranging should meet several conditions: first, it should be easily distinguishable from the direct path; second, for ease of use, it should be imperceptible to the user; and third, it should balance short duration and robustness. This invention uses an audio signal consisting of five monophonic chirped signals, each 0.25ms long and 12kHz in frequency, spaced 10ms apart, as the transmitted signal. Since the distance between the user and the phone is typically 5-35cm in actual use, to ensure the reliability of distance measurement, the chirped signal of the direct path must not interfere with the echo path. Therefore, the earliest microphone echo reflected back from the user must be later than the latest arrival time of the direct path. The following inequality must be satisfied: The echo path De is at least 5cm, and the direct path Dd is typically less than 1.5cm, thus the chirp duration T can be derived to be approximately 0.25ms. To ensure timely hardware response and stable transmission of all pulses, this invention sets an interval for the chirp signal; a 10ms interval is sufficient to separate different echoes, and the ranging results from 5 chirps help reduce errors. Simultaneously, this invention uses a Hanning window to reshape the chirp envelope, increasing the peak-to-sidelobe ratio of the chirp, thereby improving the signal-to-noise ratio of the echo. (See Appendix...) Figure 3 The two images above show the shape of the signal.

[0051] Step 1.2: Echo Ranging. The chirped signal emitted by the bottom microphone of the phone is received and passed through an 11-13kHz Butterworth bandpass filter to remove background noise. The received signal contains a direct path and an echo path of approximately 12kHz. Since the echo path is slower than the direct path, the cross-correlation peak position after the direct path peak is the echo path. Figure 3 The bottom image effectively distinguishes between the direct path and the echo path. Considering that the echo energy is much weaker than the direct path energy, this invention improves the cross-correlation method. It selects the points corresponding to the maximum and minimum values ​​of the direct path image, and then randomly selects two points within a 0.05ms range around these two points. The average time corresponding to these six points is calculated to determine the direct path time. The echo path is also determined using the same method, using six points and calculating the average time corresponding to these six selected points as the echo path time to reduce the impact of residual noise. Let Δt represent the time difference between the echo path and the direct path, expressed by the formula... The distance between the user and the mobile phone during the registration phase can be calculated, where c is the speed of light.

[0052] Step 2: Sound Field Reconstruction

[0053] Step 2.1: Generate a constant pulse s(t) using a stable sound source, and receive the signal at different distances using the top and bottom microphones of the mobile phone, denoted as s. k (d,t),k∈[1,2], let h be... k (d,t),k∈[1,2] are the impulse responses of the top and bottom microphones, given by formula s k (d,t)=h k The received signal is obtained by deconvolution of (d,t)*s(t), with each interval of 1cm. The h value at different distances is saved. k (d,t) forms a distance-related impulse response database, where d is the distance and t is the time.

[0054] Step 2.2: When a user registers the sound field, the distance de between the user and the mobile phone is obtained through distance perception. The impulse response database is then queried to obtain the impulse response h for the registration phase.k (de,t);

[0055] Step 2.3: Combine the received user-generated audio signal x(t) during the verification phase with the impulse response during verification and the inverse impulse response during registration. By performing convolutions sequentially, the reconstructed signal x can be obtained. ′ (t), the specific formula is as follows:

[0056]

[0057] Step 3: Sound Field Extraction. The audio signal received during user registration is obtained using the top and bottom microphones of the phone. After noise removal and silent segment removal, the sound field information is extracted.

[0058] Step 3.1: Noise Removal. Noise affects audio signals acquired at low signal-to-noise ratios, weakens the sound field, and leads to a decrease in verification performance. To perform noise reduction without compromising the original spatiotemporal information of the audio, this invention proposes a method based on separate noise reduction for each channel. Specifically, before the user speaks, two microphones are used to sense their respective environments for 0.5 seconds, sampling the ambient noise to display the real-time noise situation. When the user's voice is detected, the NoiseReduce toolbox is used to perform fast noise reduction through spectrum gating.

[0059] Step 3.2: Silence Removal. Silence segments in audio clips can affect sound field modeling. This invention uses the existing VAD (Voice Active Detection) method to remove silence segments.

[0060] Step 3.3: Sound Field Modeling. This invention uses existing sound field modeling methods to establish the registered human sound field in the processed two-channel audio data. A short-time Fourier transform (SFT) is performed on the reconstructed signal, where each frame lasts 25ms and the overlap between consecutive frames is 12.5ms. A Hanning window is used to reduce spectral leakage. p1 and p2 represent the positions of the two microphones, and S(p,f) represents the sound pressure level at a specific frequency in a certain channel. The formula is used... The sound field at frequency f is represented by the logarithm of the sound pressure level ratio between the two channels of the reconstructed signal. Then, the sound field in a given frame is expressed as follows:

[0061] S(p1,p2)=[Sr(p1,p2,f1),Sr(p1,p2,f2),...,Sr(p1,p2,fn)]

[0062] Where p1 and p2 are the positions of the two microphones, f is the frequency, and n is the total number of frequency dimensions; for the reconstructed sound field, the n-dimensional sound field feature vector SFF(p1,p2) is obtained by long-time average normalization:

[0063]

[0064] L is the number of frames in the reconstructed signal; S i (p1, p2) represents the sound field in the i-th frame. This invention uses a frequency resolution of 24kHz / 512 = 46.8Hz. Since typical applications such as online chat and telephone use have sampling rates not exceeding 16kHz, unless otherwise specified, this invention selects the first 171 dimensions out of 512 dimensions.

[0065] Step 4: Training the voice authentication model based on the sound field. A Gaussian mixture model (GMM) is used to fit the user's sound field feature vector SFF(p1,p2). The GMM can fit the user's sound field vector well, forming the corresponding speaker model. The registered user emits several speech signals, and the n-dimensional sound field feature vector SFF(p1,p2) is extracted from the registered user's speech signals. A Gaussian mixture model can be used to model this. For the n-dimensional SFF(p1,p2), the mixing density p(SFF|λ) of the registered user s is... s ) is defined as

[0066]

[0067] Where, λ s This represents the speaker model obtained by modeling registered user s using a Gaussian mixture model; M represents the number of Gaussian components. This represents the j-th Gaussian component of registered user s. The formula for calculating the j-th Gaussian component of registered user s, given the mixed weights, is as follows:

[0068]

[0069] Where μ i It is the average vector of the n-dimensional sound field eigenvectors; The covariance matrix of the n-dimensional sound field eigenvectors is estimated and converged using the Iterative Expectation-Maximization (EM) algorithm.

[0070] During the verification phase:

[0071] Step 1: Distance Awareness. A distance-aware method used during the repeated authentication phase to obtain the distance (dv) between the user and the phone during verification.

[0072] Step 2: Sound Field Reconstruction. Due to the mismatch between the registration and verification phases, the sound field for the verification phase needs to be reconstructed using the distances *de* and *dv* between the user and the phone obtained from the two phases, along with a database of distance-related transfer functions. During user verification, the distance *dv* between the speaker and the phone can be obtained, and the transfer function *h* at the time of verification can be retrieved from the impulse response database. k (dv,t). For example... Figure 5 As shown, the reconstructed signal can be obtained by sequentially convolving the verification phase signal with the inverse transfer function during verification and the transfer function during registration.

[0073]

[0074] from Figure 6 Figure (d) clearly shows that the sound field of the reconstructed signal is basically the same as that of the registered signal, indicating that the surface sound field reconstruction is effective.

[0075] Step 3: Use the same method as in Step 2 of the registration phase to reconstruct the sound field signal x. ′ (t) Extract the sound field.

[0076] Step 4: Process the sound field obtained in Step 3 of the verification phase using the trained Gaussian mixture model to obtain the similarity score, and perform speech authentication based on the similarity score. If the similarity score exceeds a predefined threshold, the speaker's identity authentication passes; otherwise, the speaker's identity authentication fails and is rejected.

Claims

1. A speech analysis-based identity authentication method based on sound field reconstruction, characterized in that, The method includes the following steps: Step 1: Distance Perception Step 1.1: Use the phone's speaker to emit a ranging signal, and use the phone's bottom microphone to receive the echo signal that is reflected back by the user after the ranging signal is emitted; Step 1.2: Use a Butterworth bandpass filter to remove background noise from the echo signal; Step 1.3: Locate the direct path and echo path using the time-domain image of the echo signal. Let Δt represent the time difference between the echo path and the direct path. The distance between the user and the mobile phone can be determined by... The calculation shows that c is the speed of light; Step 2: Sound Field Reconstruction Step 2.1: Generate a constant pulse s(t) using a stable sound source, and receive the signal at different distances from the user using the top and bottom microphones of the mobile phone, denoted as s. k (d,t),k∈[1,2], let h be... k (d,t),k∈[1,2] are the impulse responses of the top and bottom microphones, given by formula s k (d,t)=h k (d,t)*s(t) is deconvolved to obtain the signal, with a 1cm interval between each two receptions, and h is saved at different distances. k (d,t) forms a distance-related impulse response database, where d is the distance and t is the time. Step 2.2: When a user registers the sound field, the distance de between the user and the mobile phone is obtained through distance perception. The impulse response database is then queried to obtain the impulse response h for the registration phase. k (de,t); Similarly, during user authentication, the distance dv between the user and the mobile phone is obtained, and the impulse response h at the time of authentication is obtained by querying the impulse response database. k (dv,t); Step 2.3: Combine the received user-generated audio signal x(t) during the verification phase with the impulse response during verification and the inverse impulse response during registration. By performing convolutions sequentially, the reconstructed signal x can be obtained. ′ (t), the specific formula is as follows: Step 3: Sound Field Extraction Step 3.1: Remove noise and silent segments from the reconstructed signal; Step 3.2: Perform short-time Fourier transform processing on the reconstructed signal; using the formula The sound field at frequency f is represented by the logarithm of the sound pressure level ratio between the two channels of the reconstructed signal. Then, the sound field in a given frame is expressed as follows: S(p1,p2)=[Sr(p1,p2,f1),Sr(p1,p2,f2),...,Sr(p1,p2,fn)] Where p1 and p2 are the positions of the two microphones, f is the frequency, and n is the total number of frequency dimensions; For the sound field of the reconstructed signal, the n-dimensional sound field feature vector SFF(p1,p2) is obtained by long-time average normalization: Where L is the number of frames in the reconstructed signal; S i (p1,p2) represents the sound field in the i-th frame; Step 4: Model Training and Identity Inference Step 4.1: User registration. A Gaussian mixture model is used to fit the user's sound field feature vector SFF(p1,p2) to form the corresponding speaker model. The registered user emits several speech signals. An n-dimensional sound field feature vector SFF(p1,p2) is extracted from the registered user's speech signals and modeled using a Gaussian mixture model. For SFF(p1,p2), the mixture density p(SFF|λ) of the registered user s is... s ) is defined as Where, λ s This represents the speaker model obtained by modeling registered user s using a Gaussian mixture model; M represents the number of Gaussian components. This represents the j-th Gaussian component of registered user s. The mixed weights; the j-th Gaussian component of registered user s The calculation formula is as follows: Where μ j It is the average vector of the n-dimensional sound field eigenvectors; Let be the covariance matrix of the n-dimensional sound field eigenvectors; Then, the iterative expectation-maximization algorithm is used to estimate and converge the model parameters; Step 4.2: In the verification phase, the n-dimensional sound field feature vector of the speaker's reconstructed signal is processed using the trained Gaussian mixture model to obtain the similarity between the speaker and the registered user. If the similarity exceeds a preset threshold, the speaker's identity authentication is successful; otherwise, the speaker's identity authentication fails and will be rejected.

2. The speech analysis authentication method based on sound field reconstruction according to claim 1, characterized in that, In step 1.1, the ranging signal consists of five mono chirp signals with a duration of 0.25ms and a frequency of 12kHz, with an interval of 10ms between adjacent chirp signals.

3. The speech analysis authentication method based on sound field reconstruction according to claim 1, characterized in that, In step 1.3, the position of the cross-correlation peak after the direct path peak is the echo path. Select the points corresponding to the maximum and minimum values ​​in the image of the direct path, and then randomly select two points within a range of 0.05ms to the left and right of these two points. Calculate the average time corresponding to these 6 points to determine the time of the direct path. The echo path is also determined using the same method to select 6 points, and the average time corresponding to the selected 6 points is calculated as the time of the echo path.

4. The speech analysis authentication method based on sound field reconstruction according to claim 1, characterized in that, In step 2.1, the constant pulse s(t) is selected as an exponential sinusoidal scanning signal, whose aperiodic deconvolution form is: Where ω1 and ω2 are the start and end frequencies of s(t), T is the period, and e is the natural logarithm.

5. The speech analysis authentication method based on sound field reconstruction according to claim 1, characterized in that, In step 3.1, the noise reduction method is to reduce noise for each mono channel separately. That is, before the user speaks, the two microphones sense their respective environments for 0.5 seconds, that is, to sample the ambient noise. When the user's voice is detected, the NoiseReduce toolbox is used to perform fast noise reduction through spectrum gating. Use existing speech liveness detection algorithms to eliminate silent segments.

6. The speech analysis authentication method based on sound field reconstruction according to claim 1, characterized in that, When recording, the user must face the phone's microphone directly.

Images