Voiceprint recognition method and device under surrounding type signal domain and man-machine interaction equipment

Abstract

The invention relates to a voiceprint recognition method and device under an encircling type signal domain and man-machine interaction equipment, and belongs to the technical field of wearable device.The method comprises the steps that voice signals of a target user are collected, and physiological signals at least including wrist movement acceleration, skin surface vibration frequency and blood oxygen saturation are synchronously obtained; carrying out environmental noise separation processing on the voice signal, and extracting a fundamental frequency jitter rate and a voiceprint feature vector of a formant migration track contained in the voice signal; performing time-frequency transformation on the physiological signal, and performing feature fusion on the physiological signal and a voiceprint feature vector to generate a feature map containing a user identity identifier and a physiological state parameter; and event judgment is carried out on the voice feature segment and the physiological feature segment in the feature map, so that the voiceprint recognition precision is improved in a complex environment, and meanwhile, the real-time early warning effect of user identity determination and physiological abnormalities such as respiratory disorder is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable devices, and in particular to a voiceprint recognition method, device, and human-computer interaction device in an embracing signal domain. Background Art

[0002] Existing voiceprint recognition technologies typically rely on a single voice signal, making them susceptible to interference in complex environmental noise or when the user is in motion. They also lack the ability to distinguish between forged recordings and other security issues. Furthermore, the physiological monitoring functions of traditional wearable devices are independent of their identity authentication systems, resulting in low data utilization and poor dynamic adaptability. For example, while smart bracelets can collect heart rate and movement data, they cannot be integrated with voice features for dynamic identity verification. Existing noise suppression methods do not account for the coupled noise generated by vibrations from contact between the device and the skin. Summary of the Invention

[0003] The main purpose of the present invention is to provide a voiceprint recognition method, device and human-computer interaction equipment in an embracing signal domain, so as to improve the voiceprint recognition accuracy in complex environments while achieving the effect of user identity determination and real-time early warning of physiological abnormalities such as respiratory disorders.

[0004] To achieve the above objectives, the present invention provides a voiceprint recognition method in an embracing signal domain, comprising the following steps: Collect the target user's voice signal and simultaneously obtain physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; performing environmental noise separation processing on the speech signal to extract voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; The physiological signal is subjected to time-frequency transformation and feature fusion is performed with the voiceprint feature vector to generate a feature map containing the user identity and physiological state parameters; Event judgment is performed on the speech feature segments and the physiological feature segments in the feature map.

[0005] Furthermore, the step of collecting the target user's voice signal includes: Multi-angle voice signals are acquired through a micro-microphone array distributed in a ring in an embracing wearable device. The directional beam of the microphone array dynamically adjusts the focus area with the acceleration of the wrist movement.

[0006] Furthermore, the step of synchronously acquiring physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation includes: Utilizing the piezoelectric sensor array embedded in the wraparound wearable device to capture the skin surface vibration frequency band distribution information in real time, the skin surface vibration frequency is obtained; Dynamically correcting the time-frequency aliasing noise in the skin surface vibration frequency caused by limb displacement using the output value of the motion accelerometer to obtain the wrist motion acceleration; At the same time, a dual-wavelength optical module is used to sample blood oxygen saturation.

[0007] Furthermore, the step of performing environmental noise separation processing on the speech signal and extracting the voiceprint feature vector of the fundamental frequency jitter rate and the formant migration trajectory contained in the speech signal includes: A dynamic beamforming algorithm is used in combination with wrist acceleration parameters to adjust the sound field coverage of the microphone array and suppress the ambient noise component in the direction other than the wearer's direction. Separating the low-frequency modulation signal corresponding to the respiratory rhythm in the speech signal by a preset decomposition algorithm, and extracting the fundamental frequency jitter rate of the fundamental frequency trajectory; Capturing vocal tract morphological changes and combining the fluctuation trend of the blood oxygen saturation to generate the formant migration trajectory with physiological relevance; The fundamental frequency jitter rate and the formant migration trajectory are vectorized to obtain the voiceprint feature vector.

[0008] Furthermore, the physiological signal is subjected to time-frequency transformation and feature fusion is performed with the voiceprint feature vector to generate a feature map containing the user identity and physiological state parameters, including: A two-stream convolutional network is used to process the physiological signals separately. The first-stream network extracts speech physiological features that are coupled with the speech signal, and the second-stream network extracts independent physiological features that are unrelated to speech. The speech physiological features establish dynamic association weights with the voiceprint feature vector in the frequency domain through a cross-modal attention mechanism. A weight matrix is ​​constructed based on blood oxygen saturation. The update period of the weight matrix is ​​automatically adjusted according to the user's current wrist motion acceleration. When the motion acceleration exceeds a set threshold, the matrix update rate is reduced. The weight matrix is ​​used to map and fuse the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the voiceprint feature vector; The voiceprint feature vector is input into a generative adversarial network, wherein the generator of the generative adversarial network dynamically enhances the identity-sensitive component according to the physiological state parameters, and the discriminator synchronously detects physiological abnormal events and identity forgery features, and generates a coding map by alternately optimizing the identity discrimination loss function and the physiological abnormality detection loss function. The coding basis of the coding map is uniquely determined by the user identity identifier, and the mutation area in the coding map reflects the spatiotemporal distribution information of real-time physiological abnormal events. The coding map dimension dynamically increases the number of feature channels according to the continuous wearing time of the device.

[0009] Furthermore, the step of performing event judgment on the speech feature segment and the physiological feature segment in the feature map includes: Detecting the morphological mutation boundary of the physiological feature segment and performing time sequence matching with the fluctuation range of the fundamental frequency jitter rate in the voiceprint feature segment; When the deviation between the resonance peak migration trajectory and the preset identity template exceeds a dynamic adjustment threshold, a secondary verification process is triggered in combination with the blood oxygen saturation change rate. The dynamic adjustment threshold is automatically updated according to the user's historical physiological parameter baseline; Based on the correlation between skin vibration frequency energy distribution and wrist movement acceleration, the voiceprint feature artifact interference caused by limb movements is eliminated; The vocal cord vibration spectrum characteristics are coupled with the upper respiratory tract mechanical parameters for analysis, and the output includes the judgment results of the event type and abnormal tracing path.

[0010] Furthermore, a weight matrix is ​​constructed based on blood oxygen saturation, and the update period of the weight matrix is ​​automatically adjusted according to the user's current wrist motion acceleration. When the motion acceleration exceeds a set threshold, the matrix update rate is reduced. The time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the voiceprint feature vector are mapped and fused using the weight matrix, including: A weight matrix is ​​constructed based on the trend of blood oxygen saturation changes, wherein the axial dimensions of the weight matrix correspond to the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the amplitude change rate of the voiceprint feature vector; Dynamically adjust the matrix update period based on the user's current wrist acceleration amplitude range. When the acceleration exceeds the preset motion threshold, reduce the matrix update rate to 1 / 3 to 1 / 2 of the original rate. The instantaneous slope of the resonance peak migration trajectory is superimposed with the main frequency of the skin vibration frequency, and then the superposition result is convolved with the phase information of the voiceprint feature vector to generate the voiceprint feature vector with physiological state constraints.

[0011] Furthermore, the step of constructing a weight matrix based on the blood oxygen saturation change trend includes: The basic weight coefficient is set according to the change direction of the mean value of the blood oxygen saturation within a preset time. The upward trend corresponds to increasing the fusion weight of the skin vibration frequency, and the downward trend corresponds to increasing the discrimination weight of the resonance peak trajectory. The real-time value of wrist acceleration is linearly mapped to the weight update frequency. For every 1m / s² increase in acceleration, the matrix update interval is extended by 0.5 seconds. When the energy attenuation rates of the blood oxygen saturation and the skin vibration frequency fluctuate in opposite directions, a fixed proportional attenuation is applied to the voiceprint feature vector.

[0012] The present invention proposes a voiceprint recognition device in an embracing signal domain, comprising: An acquisition unit, configured to acquire a target user's voice signal and simultaneously acquire physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; a vector unit, configured to perform environmental noise separation processing on the speech signal and extract voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; A graph unit is used to perform feature fusion of the physiological signal after time-frequency transformation with the voiceprint feature vector to generate a feature graph containing the user identity and physiological state parameters; Identification unit, used to perform event judgment on the voice feature segment and physiological feature segment in the feature map The present invention also provides a human-computer interaction device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the voiceprint recognition method in the above-mentioned embracing signal domain are implemented.

[0013] The voiceprint recognition method, device, and human-computer interaction device in the embracing signal domain provided by the present invention have the following beneficial effects: (1) Through the dynamic beamforming of the surround microphone array and the adaptive filtering of the skin vibration frequency, the environmental noise and device-coupled vibration interference are effectively eliminated, so that the voiceprint feature extraction can still maintain high accuracy in motion scenes; (2) Using generative adversarial networks to construct a spatiotemporal joint coding map, deeply binding voiceprint features with physiological parameters such as blood oxygen and skin vibration; (3) Based on the weight matrix of motion acceleration and the self-expanding feature map of wearing time, the device automatically optimizes the allocation of computing resources in different states such as stillness and motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 1 is a flow chart of a voiceprint recognition method in an embracing signal domain according to an embodiment of the present invention; Figure 2 This is a structural block diagram of a voiceprint recognition device in an embracing signal domain according to an embodiment of the present invention; Figure 3 It is a schematic block diagram of the structure of a human-computer interaction device according to an embodiment of the present invention.

[0015] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] Reference Figure 1 , which is a flow chart of a voiceprint recognition method in an embracing signal domain proposed by the present invention. The method includes the following steps: S1, collecting the target user's voice signal and simultaneously obtaining physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; In step S1, a step of collecting a voice signal of a target user is included, which includes: Multi-angle voice signals are acquired through a micro-microphone array distributed in a ring in an embracing wearable device. The directional beam of the microphone array dynamically adjusts the focus area with the acceleration of the wrist movement.

[0018] And, the step of synchronously acquiring physiological signals including at least wrist motion acceleration, skin surface vibration frequency and blood oxygen saturation includes: Utilizing the piezoelectric sensor array embedded in the wraparound wearable device to capture the skin surface vibration frequency band distribution information in real time, the skin surface vibration frequency is obtained; Dynamically correcting the time-frequency aliasing noise in the skin surface vibration frequency caused by limb displacement using the output value of the motion accelerometer to obtain the wrist motion acceleration; At the same time, a dual-wavelength optical module is used to sample blood oxygen saturation.

[0019] In the embodiment of step S1, the human-computer interaction device (specifically, a smart wristband) has an integrated circular array of six miniature microphones on the inside of its wristband, with adjacent microphones spaced 8 mm apart to form a 360-degree voice capture loop. When worn by the user, the directional beam of the microphone array is based on the X / Y / Z-axis acceleration values ​​fed back in real time by the wrist motion accelerometer. The beam focus area adjustment is calculated using the following formula:

[0020] Among them, a x , a y , a z is the triaxial acceleration component, when the acceleration amplitude When the speed exceeds 2m / s², the beam width is automatically narrowed to 60° to suppress ambient noise in the direction of movement (such as wind noise when running).

[0021] The skin surface vibration frequency is collected by arranging a 4×4 piezoelectric sensor array (TDK PVDF thin film sensors with a sampling rate of 1kHz) on the contact surface of the wristband, covering the pulse area. In the process of implementing time-frequency aliasing noise correction, when the accelerometer detects wrist displacement (such as waving), the skin surface vibration frequency is extracted from the frequency band signal to establish the acceleration amplitude. Vibration frequency of the skin surface The linear regression model is: , and obtain the corrected wrist motion acceleration ax, ay, az. For the dual-wavelength optical module to sample blood oxygen saturation, red light AC (660nm) and infrared light DC (880nm) are emitted alternately at 200Hz, and the reflected light intensity is received by the photodiode, and the ratio is calculated. , according to the pre-calibrated curve S p O2=110 25R obtains blood oxygen value S p O2.

[0022] S2, performing environmental noise separation processing on the speech signal, and extracting voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; In step S2, a dynamic beamforming algorithm is used in combination with wrist motion acceleration parameters to adjust the sound field coverage of the microphone array to suppress the ambient noise component in the direction other than the wearer; a preset decomposition algorithm is used to separate the low-frequency modulation signal corresponding to the respiratory rhythm in the voice signal, and the fundamental frequency jitter rate of the fundamental frequency trajectory is extracted; the changes in the vocal tract morphology are captured, and the resonance peak migration trajectory with physiological relevance is generated in combination with the fluctuation trend of the blood oxygen saturation; the fundamental frequency jitter rate and the resonance peak migration trajectory are vectorized to obtain the voiceprint feature vector.

[0023] In the embodiment of step S2, the wrist triaxial acceleration data is input , calculate the magnitude of the resultant acceleration vector , and then, according to the acceleration vector magnitude Dynamically adjust the beam width of the microphone array. The dynamic adjustment range of the beam width is as follows:

[0024] Finally, the generalized sidelobe canceller (GSC) algorithm is used to suppress out-of-beam noise, and the main lobe direction is always pointed to the estimated position of the mouth, thereby suppressing the ambient noise component in the direction other than the wearer.

[0025] In the process of separating the low-frequency modulation signal corresponding to the respiratory rhythm in the speech signal using the decomposition algorithm, the improved variational mode decomposition (VMD) is performed on the denoised speech signal. The mode number K is set to 3 and the penalty factor α is set to 2000. The 50-300 Hz baseband, the respiratory modulation band (0.2-2 Hz) and the residual noise band are separated. The envelope signal of the respiratory modulation band is extracted, and the peak offset of the cross-correlation function between it and the baseband signal is calculated, which is defined as the fundamental frequency jitter rate J. F0 :

[0026] in, is the cross-correlation peak position of the i-th frame, is the historical average.

[0027] In the process of capturing the changes in vocal tract morphology and generating the physiologically relevant formant migration trajectory in combination with the fluctuation trend of the blood oxygen saturation, time-varying linear predictive coding (TV-LPC) is used to analyze the speech signal, and the formant parameters (F1-F3) are updated every 10ms. Then, the blood oxygen saturation (SpO2) and the formant migration trajectory are established. The association model:

[0028] Among them, k=1, 2, and 3 correspond to three resonance peaks, α=0.15 and β=0.08 are the physiological coupling coefficients obtained through user calibration. is the resonance peak segment, is the resonance peak migration trajectory, is the resonance peak segment associated with the fluctuation trend, d is the distance of capturing the vocal tract, and t is the time of capturing the vocal tract.

[0029] In the process of vectorizing the fundamental frequency jitter rate and the formant migration trajectory to obtain the voiceprint feature vector, the fundamental frequency jitter rate J F0 Perform Z-score standardization and calculate the mean, variance, and skewness statistics according to the time window to transfer the resonance peak trajectory. Use PCA transformation to reduce the dimension to 10-dimensional principal components, and concatenate the two to generate a 28-dimensional voiceprint feature vector:

[0030] S3, performing feature fusion on the physiological signal after time-frequency transformation and the voiceprint feature vector to generate a feature map containing the user identity and physiological state parameters; In the embodiment of step S3, a dual-stream convolutional network is used to process the physiological signals separately, the first-stream network extracts voice physiological features that are coupled with the voice signal, and the second-stream network extracts independent physiological features that are unrelated to the voice, wherein the voice physiological features establish dynamic correlation weights in the frequency domain with the voiceprint feature vector through a cross-modal attention mechanism; a weight matrix is ​​constructed based on blood oxygen saturation, and the update period of the weight matrix is ​​automatically adjusted according to the user's current wrist motion acceleration. When the motion acceleration exceeds the set threshold, the matrix update rate is reduced, and the time-varying parameters of the resonance peak migration trajectory and the skin vibration frequency are adjusted by the weight matrix. The frequency domain energy distribution and the voiceprint feature vector are mapped and fused; the voiceprint feature vector is input into a generative adversarial network, wherein the generator of the generative adversarial network dynamically enhances the identity sensitive component according to the physiological state parameters, and the discriminator synchronously detects physiological abnormal events and identity forgery features, and generates a coding map by alternately optimizing the identity discrimination loss function and the physiological abnormality detection loss function. The coding basis of the coding map is uniquely determined by the user identity identifier, and the mutation area in the coding map reflects the spatiotemporal distribution information of real-time physiological abnormal events, and the coding map dimension dynamically increases the number of feature channels according to the continuous wearing time of the device.

[0031] Specifically, a weight matrix is ​​constructed based on the changing trend of blood oxygen saturation, and the axial dimensions of the weight matrix correspond to the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the amplitude change rate of the voiceprint feature vector; the matrix update period is dynamically adjusted according to the amplitude range of the user's current wrist motion acceleration. When the acceleration exceeds the preset motion threshold, the matrix update rate is reduced to the range of 1 / 3 to 1 / 2 of the original rate; the instantaneous slope of the resonance peak migration trajectory and the main frequency of the skin vibration frequency are superimposed, and then the superposition result is convolved with the phase information of the voiceprint feature vector to generate the voiceprint feature vector with physiological state constraints.

[0032] As mentioned above, the basic weight coefficient is set according to the direction of change of the mean value of the blood oxygen saturation within the preset time. The upward trend corresponds to increasing the fusion weight of the skin vibration frequency, and the downward trend corresponds to increasing the discrimination weight of the resonance peak trajectory; the real-time numerical value of the wrist movement acceleration is linearly mapped to the weight update frequency, and the matrix update interval is extended by 0.5 seconds for every increase of 1m / s² in acceleration; when the energy attenuation rate of the blood oxygen saturation and the skin vibration frequency fluctuate in the opposite direction, a fixed proportional attenuation is applied to the voiceprint feature vector.

[0033] In the specific implementation of step S3, the dual-stream convolutional network is composed of the following branches: a speech-related physiological feature stream and an independent physiological feature stream, wherein the speech-related physiological feature stream is a physiological signal after input time-frequency transformation (such as skin vibration frequency STFT spectrum, blood oxygen time series waveform), and a three-layer causal convolution (kernel size [3,5,3], number of channels [16,32,64]) is used to extract coupling features related to speech fundamental frequency jitter through a cross-modal attention layer and a voiceprint feature vector. Interaction:

[0034] Where Q is the physiological feature, K and V are the voiceprint features, and d k = 64 is the scaling factor. The independent physiological feature streams take the same physiological signals as input, but introduce channel masks (e.g., masking frequency bands related to speech) into the convolutional layer. The output includes physiological indicators such as heart rate variability and skin impedance that are not related to identity.

[0035] In the process of building and fusion of the dynamic weight matrix mentioned above, the blood oxygen driven weight is first initialized to define the three-dimensional weight matrix , where the T-axis is the instantaneous slope of the corresponding resonance peak migration trajectory (every 10ms sampling point), the F-axis is the 1 / 3 octave energy of the corresponding skin vibration frequency (center frequency 50Hz, 160Hz, 500Hz), and the c-axis is the amplitude change rate of the corresponding voiceprint feature vector (calculated separately according to 28-dimensional features). In response to the blood oxygen trend, when the 5-second sliding mean of blood oxygen saturation increases by more than 1%, the F-axis weight is increased to 1.8 times. When the blood oxygen decreases for 3 consecutive seconds, the T-axis weight coefficient is increased according to Dynamic enhancement.

[0036] The update mechanism of motion acceleration constraint is the update cycle With acceleration The mapping relationship is:

[0037] Then the fusion process is carried out, firstly performing weighted superposition on the instantaneous slope of the resonance peak S(t) and the skin main frequency energy E(f): , get the first voiceprint feature , and then add the superposition result to the voiceprint phase information Perform time domain convolution to obtain the second voiceprint feature .

[0038] The above-mentioned generative adversarial network includes a generator and a discriminator, wherein, The generator is the enhanced second voiceprint feature , real-time physiological state parameters are used as input to the adversarial network. The structure of the adversarial network generated based on the input of the adversarial network is a 5-layer fully connected network (hidden layer dimensions [256, 128, 64, 32, 16]), which outputs the identity sensitive component .

[0039] The discriminator creates a loss function through the identity discrimination branch (binary classification) and the physiological anomaly detection branch (multi-classification), and then generates an encoding map. The specific process of generating the encoding map is: (1) Identity-sensitive components Generate 16-dimensional identity basis vectors through orthogonalization; (2) The 128-dimensional mutation vector triggered by the abnormal event is superimposed on the base to obtain the abnormal event coding map.

[0040] S4, performing event judgment on the speech feature segments and the physiological feature segments in the feature map.

[0041] In an embodiment of step S4, the morphological mutation boundary of the physiological feature segment is detected, and time-matched with the fluctuation range of the fundamental frequency jitter rate in the voiceprint feature segment; when the deviation of the resonance peak migration trajectory from the preset identity template exceeds the dynamic adjustment threshold, the secondary verification process is triggered in combination with the blood oxygen saturation change rate, and the dynamic adjustment threshold is automatically updated according to the user's historical physiological parameter baseline; based on the correlation between the skin vibration frequency energy distribution and the wrist movement acceleration, the voiceprint feature artifact interference caused by limb movements is eliminated; the vocal cord vibration spectrum characteristics and the upper respiratory tract mechanical parameters are coupled and analyzed, and the judgment result including the event type and the abnormal tracing path is output.

[0042] In the specific embodiment of step S4, the process of morphological mutation boundary detection is to perform the Canny edge detection algorithm on the physiological feature segment (skin vibration energy, blood oxygen change gradient), locate the energy jump point, and then define the mutation boundary condition so that the energy difference in adjacent 50ms windows exceeds 3 times the standard deviation of the historical baseline. In the process of time series matching with the fluctuation range of the fundamental frequency jitter rate in the voiceprint feature segment, the fundamental frequency jitter rate J in the voiceprint feature segment is extracted. F0 The dynamic time warping (DTW) algorithm is used to align the timing deviation between the physiological mutation point and the peak of the fundamental frequency fluctuation within the fluctuation range (i.e., the maximum value minus the minimum value). When the alignment error is less than 50ms, it is marked as a valid correlation event. The Canny edge detection algorithm and the dynamic time warping (DTW) algorithm use existing technologies and are not described in detail in this embodiment.

[0043] When the deviation between the resonance peak migration trajectory and the preset identity template exceeds the dynamic adjustment threshold, the secondary verification process is triggered in combination with the blood oxygen saturation change rate. The preset identity template is used to include the mean and fluctuation range of the resonance peak trajectory when the user registered. The cosine similarity between the current resonance peak migration trajectory and the template is calculated in real time, and the threshold is dynamically adjusted. When the cosine similarity is less than the dynamic adjustment threshold and the blood oxygen decrease rate is greater than 2% / s, the secondary verification is triggered.

[0044] The above process of eliminating the interference of voiceprint feature artifacts caused by body movements is to establish the vibration energy E v With acceleration a sum Compensation model:

[0045] Where, is the skin vibration frequency, when a sum >2m / s2 S When , a frequency domain mask is applied to the voiceprint feature vector (suppressing the high frequency components of 200-500Hz).

[0046] In the above process of coupling the vocal cord vibration spectrum characteristics with the upper respiratory tract mechanical parameters and outputting the judgment results including the event type and abnormal tracing path, the parameters include the vocal cord mass m=0.15g, stiffness k=1.2N / m, and the upper respiratory tract airflow disturbance parameter is determined by the blood oxygen saturation S p O2 and skin vibration frequency f v Make deduction:

[0047] Where, is the upper respiratory tract airflow disturbance parameter.

[0048] Reference Attachment Figure 2 This is a structural block diagram of a voiceprint recognition device in an embracing signal domain proposed by the present invention. The device includes: An acquisition unit, configured to acquire a target user's voice signal and simultaneously acquire physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; a vector unit, configured to perform environmental noise separation processing on the speech signal and extract voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; A graph unit is used to perform feature fusion of the physiological signal after time-frequency transformation with the voiceprint feature vector to generate a feature graph containing the user identity and physiological state parameters; The recognition unit is used to perform event judgment on the voice feature segments and physiological feature segments in the feature map.

[0049] Reference Figure 3 In the embodiment of the present invention, a human-computer interaction device is also provided. The human-computer interaction device can be a server, and its internal structure can be as follows: Figure 3 As shown. The human-computer interaction device includes a processor, a memory, a display screen, an input device, a network interface and a database connected via a system bus. The computer-designed processor is used to provide computing and control capabilities. The memory of the human-computer interaction device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the human-computer interaction device is used to store the corresponding data in this embodiment. The network interface of the human-computer interaction device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the above method is implemented.

[0050] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the human-computer interaction device to which the solution of the present invention is applied.

[0051] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A voiceprint recognition method in an embracing signal domain, characterized in that: The following steps are involved: Collect the target user's voice signal and simultaneously obtain physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; performing environmental noise separation processing on the speech signal to extract voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; The physiological signal is subjected to time-frequency transformation and feature fusion is performed with the voiceprint feature vector to generate a feature map containing the user identity and physiological state parameters; Event judgment is performed on the speech feature segments and the physiological feature segments in the feature map.

2. The voiceprint recognition method in the embracing signal domain according to claim 1, characterized in that: The steps of collecting the target user's voice signal include: Multi-angle voice signals are acquired through a micro-microphone array distributed in a ring in an embracing wearable device. The directional beam of the microphone array dynamically adjusts the focus area with the acceleration of the wrist movement.

3. The voiceprint recognition method in the embracing signal domain according to claim 1, characterized in that: The step of synchronously acquiring physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation includes: Utilizing the piezoelectric sensor array embedded in the wraparound wearable device to capture the skin surface vibration frequency band distribution information in real time, the skin surface vibration frequency is obtained; Dynamically correcting the time-frequency aliasing noise in the skin surface vibration frequency caused by limb displacement using the output value of the motion accelerometer to obtain the wrist motion acceleration; At the same time, a dual-wavelength optical module is used to sample blood oxygen saturation.

4. The voiceprint recognition method in the embracing signal domain according to claim 1, characterized in that: The step of performing environmental noise separation processing on the speech signal and extracting the voiceprint feature vector of the fundamental frequency jitter rate and the formant migration trajectory contained in the speech signal includes: A dynamic beamforming algorithm is used in combination with wrist acceleration parameters to adjust the sound field coverage of the microphone array and suppress the ambient noise component in the direction other than the wearer's direction. Separating the low-frequency modulation signal corresponding to the respiratory rhythm in the speech signal by a preset decomposition algorithm, and extracting the fundamental frequency jitter rate of the fundamental frequency trajectory; Capturing vocal tract morphological changes and combining the fluctuation trend of the blood oxygen saturation to generate the formant migration trajectory with physiological relevance; The fundamental frequency jitter rate and the formant migration trajectory are vectorized to obtain the voiceprint feature vector.

5. The voiceprint recognition method in the embracing signal domain according to claim 4, characterized in that: The step of fusing the physiological signal after time-frequency transformation with the voiceprint feature vector to generate a feature map containing the user identity and physiological state parameters includes: A two-stream convolutional network is used to process the physiological signals separately. The first-stream network extracts speech physiological features that are coupled with the speech signal, and the second-stream network extracts independent physiological features that are unrelated to speech. The speech physiological features establish dynamic association weights with the voiceprint feature vector in the frequency domain through a cross-modal attention mechanism. A weight matrix is ​​constructed based on blood oxygen saturation. The update period of the weight matrix is ​​automatically adjusted according to the user's current wrist motion acceleration. When the motion acceleration exceeds a set threshold, the matrix update rate is reduced. The weight matrix is ​​used to map and fuse the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the voiceprint feature vector; The voiceprint feature vector is input into a generative adversarial network, wherein the generator of the generative adversarial network dynamically enhances the identity-sensitive component according to the physiological state parameters, and the discriminator synchronously detects physiological abnormal events and identity forgery features, and generates a coding map by alternately optimizing the identity discrimination loss function and the physiological abnormality detection loss function. The coding basis of the coding map is uniquely determined by the user identity identifier, and the mutation area in the coding map reflects the spatiotemporal distribution information of real-time physiological abnormal events. The coding map dimension dynamically increases the number of feature channels according to the continuous wearing time of the device.

6. The voiceprint recognition method in the embracing signal domain according to claim 5, characterized in that: The step of performing event judgment on the speech feature segment and the physiological feature segment in the feature map includes: Detecting the morphological mutation boundary of the physiological feature segment and performing time sequence matching with the fluctuation range of the fundamental frequency jitter rate in the voiceprint feature segment; When the deviation between the resonance peak migration trajectory and the preset identity template exceeds a dynamic adjustment threshold, a secondary verification process is triggered in combination with the blood oxygen saturation change rate. The dynamic adjustment threshold is automatically updated according to the user's historical physiological parameter baseline; Based on the correlation between skin vibration frequency energy distribution and wrist movement acceleration, the voiceprint feature artifact interference caused by limb movements is eliminated; The vocal cord vibration spectrum characteristics are coupled with the upper respiratory tract mechanical parameters for analysis, and the output includes the judgment results of the event type and abnormal tracing path.

7. The voiceprint recognition method in the embracing signal domain according to claim 5, characterized in that: A weight matrix is ​​constructed based on blood oxygen saturation, and the update period of the weight matrix is ​​automatically adjusted according to the user's current wrist motion acceleration. When the motion acceleration exceeds a set threshold, the matrix update rate is reduced. The steps of mapping and fusing the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the voiceprint feature vector using the weight matrix include: A weight matrix is ​​constructed based on the trend of blood oxygen saturation changes, wherein the axial dimensions of the weight matrix correspond to the time-varying parameters of the resonance peak migration trajectory, the frequency domain energy distribution of the skin vibration frequency, and the amplitude change rate of the voiceprint feature vector; Dynamically adjust the matrix update period based on the user's current wrist acceleration amplitude range. When the acceleration exceeds the preset motion threshold, reduce the matrix update rate to 1 / 3 to 1 / 2 of the original rate. The instantaneous slope of the resonance peak migration trajectory is superimposed with the main frequency of the skin vibration frequency, and then the superposition result is convolved with the phase information of the voiceprint feature vector to generate the voiceprint feature vector with physiological state constraints.

8. The voiceprint recognition method in the embracing signal domain according to claim 7, characterized in that: The steps of constructing a weight matrix based on the blood oxygen saturation change trend include: The basic weight coefficient is set according to the change direction of the mean value of the blood oxygen saturation within a preset time. The upward trend corresponds to increasing the fusion weight of the skin vibration frequency, and the downward trend corresponds to increasing the discrimination weight of the resonance peak trajectory. The real-time value of wrist acceleration is linearly mapped to the weight update frequency. For every 1m / s² increase in acceleration, the matrix update interval is extended by 0.5 seconds. When the energy attenuation rates of the blood oxygen saturation and the skin vibration frequency fluctuate in opposite directions, a fixed proportional attenuation is applied to the voiceprint feature vector.

9. A voiceprint recognition device in an embracing signal domain, characterized in that: include: An acquisition unit, configured to acquire a target user's voice signal and simultaneously acquire physiological signals including at least wrist motion acceleration, skin surface vibration frequency, and blood oxygen saturation; a vector unit, configured to perform environmental noise separation processing on the speech signal and extract voiceprint feature vectors of the fundamental frequency jitter rate and formant migration trajectory contained in the speech signal; A graph unit is used to perform feature fusion of the physiological signal after time-frequency transformation with the voiceprint feature vector to generate a feature graph containing the user identity and physiological state parameters; The recognition unit is used to perform event judgment on the voice feature segments and physiological feature segments in the feature map.

10. A human-computer interaction device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the voiceprint recognition method in the embracing signal domain described in any one of claims 1 to 8 are implemented.

Images