User identity authentication method and system based on breath sound

Abstract

The invention discloses a user identity authentication method and system based on breathing sound, and the method comprises the steps: capturing an in-ear breathing signal from the interior of an ear canal through an ear-wearing device, and carrying out the denoising of the in-ear breathing signal, and obtaining a breathing denoised signal; utilizing a Gaussian mixture model to respectively calculate the probability that each frame of breath denoising signal belongs to a breath event and a non-breath event to obtain a breath prior probability and a non-breath prior probability, and segmenting the breath denoising signal to obtain a breath segment signal; extracting a biometric identifier from the respiratory segment signal; inputting the biological feature identifier into a pre-trained triple neural network, and calculating the similarity between the biological feature identifier and a pre-stored biological feature template; performing user identity authentication according to the similarity; according to the method, the biological characteristics which are difficult to counterfeit are extracted from the breathing sound, spoofing attacks can be resisted to a high degree, and meanwhile the method has high resistance and adaptability to environmental noise and breathing behavior changes.

Description

Technical Field

[0001] The invention belongs to the field of identity recognition, and in particular relates to a user identity authentication method and system based on breathing sounds. Background Art

[0002] As the market size of ear-worn devices continues to expand, more and more sensors are integrated into earphones, which in turn leads to more and more applications and provides a new direction for identity authentication technology. Existing biometric technologies based on ear-worn devices achieve identity authentication by extracting biometric features such as the user's ear canal geometry, gait, voice characteristics, and tooth occlusion. However, these methods usually rely on the user's active behavior (such as walking, talking, or chewing) and require the joint use of transceiver sensors (such as speakers and microphones), which reduces the user experience.

[0003] As a common and unconscious physiological behavior, breathing provides new possibilities for continuous passive authentication. However, due to the weak breathing sounds and the potential safety risks of air conduction, such systems are vulnerable to audio playback attacks, and accurate authentication requires users to bring the acquisition device close to their mouth or nose, which has obvious deficiencies in security and ease of use. Therefore, there is an urgent need for a continuous passive user authentication system that has no other behavioral restrictions or usage restrictions on users while maintaining a high level of security. Summary of the invention

[0004] The present invention provides a user identity authentication method and system based on breathing sounds, which extracts difficult-to-forge biological features from breathing sounds, can resist deception attacks to a high degree, and have strong resistance and adaptability to environmental noise and changes in breathing behavior.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A first aspect of the present invention provides a user identity authentication method based on breathing sounds, comprising:

[0007] The ear breathing signal is captured from the ear canal by using an ear-worn device, and the ear breathing signal is denoised to obtain a breathing denoised signal;

[0008] The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain the respiratory prior probability and the non-respiratory prior probability. The likelihood ratio of the respiratory denoised signal of each frame is calculated by the respiratory prior probability and the non-respiratory prior probability. The respiratory denoised signal is segmented according to the likelihood ratio to obtain the respiratory segment signal.

[0009] A biometric identifier is extracted from a respiratory segment signal; the biometric identifier is input into a pre-trained ternary neural network to calculate the similarity between the biometric identifier and a pre-stored biometric template; and user identity authentication is performed based on the similarity.

[0010] Furthermore, the in-ear breathing signal is denoised to obtain a breathing denoised signal, specifically including:

[0011] A recognition window is set on the intra-ear breathing signal; after moving the recognition window on the intra-ear breathing signal according to the set jump length, the power spectrum density of the intra-ear breathing signal in the recognition window is calculated, and when the power spectrum density exceeds a preset density threshold, the intra-ear breathing signal in the recognition window is discarded to obtain a first-level noise reduction breathing signal;

[0012] The first-stage noise reduction breathing signal is input into a bandpass filter, and the first-stage noise reduction breathing signal is filtered within a set frequency range to obtain a second-stage noise reduction breathing signal;

[0013] The denoising parameters of the maximum overlap discrete wavelet transform model are optimized by genetic algorithm; the second-level denoised respiratory signal is input into the optimized maximum overlap discrete wavelet transform model, the second-level denoised respiratory signal is decomposed into multiple wavelet coefficients and scaling coefficients, the wavelet coefficients and scaling coefficients are thresholded and the signal is reconstructed to obtain the respiratory denoising signal.

[0014] Furthermore, the denoising parameters of the maximum overlap discrete wavelet transform model are optimized by genetic algorithm, including:

[0015] Define the denoising parameter set of the maximum overlap discrete wavelet transform model, expressed as:

[0016]

[0017] In the formula, For the The set of denoising parameters in the genetic algorithm iteration; For the Mother wavelet function in genetic algorithm iteration; For the Decomposition level in genetic algorithm iteration; For the Threshold function in genetic algorithm iteration; For the The threshold is selected in the genetic algorithm iteration; For the Threshold rescaling function in genetic algorithm iteration;

[0018] Encode the denoising parameter set as chromosomes, and randomly generate a set number of chromosomes to form an initial population;

[0019] Based on the maximum overlap discrete wavelet transform model of the chromosome corresponding denoising parameter set, the original noisy signal is converted into a reconstructed signal and the mean square error is calculated. The expression formula is:

[0020]

[0021]

[0022] In the formula, is the maximum overlap discrete wavelet transform model, To reconstruct the signal; is the original noisy signal; is the mean square error; is the number of original noisy signals; n is the sequence number of the original noisy signal;

[0023] According to the mean square error, the chromosomes are crossovered and mutated and the initial population of chromosomes is updated. The optimization process of denoising parameters in the maximum overlap discrete wavelet transform model is repeated until the termination condition is met and the optimized maximum overlap discrete wavelet transform model is output.

[0024] Furthermore, the Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain the respiratory prior probability and the non-respiratory prior probability, and the likelihood ratio of each frame of the respiratory denoised signal is calculated by the respiratory prior probability and the non-respiratory prior probability, specifically including:

[0025] Divide each frame of the breathing denoised signal into W sub-bands according to the frequency;

[0026] The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoising signal belongs to a respiratory event to obtain the respiratory prior probability, which is expressed as:

[0027]

[0028] In the formula, is the prior probability of breathing; For the The first The logarithmic energy of the sub-bands; For the Parameter sets for sub-bands; is the category label of the breathing denoised signal of each frame; For the The mean of the sub-bands; For the The variance of the sub-bands; is the ratio of pi; is the natural exponential function;

[0029] The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a non-respiratory event to obtain the non-respiratory prior probability, which is expressed as:

[0030]

[0031] In the formula, is the non-breathing prior probability;

[0032] The likelihood ratio of the sub-band in each frame of the respiratory denoised signal is calculated by the respiratory prior probability and the non-respiratory prior probability. The expression formula is:

[0033]

[0034] In the formula, For the The first The likelihood ratio of each sub-band;

[0035] According to the power cumulative distribution of each sub-band, a weight is assigned to the likelihood ratio of each sub-band, and the weighted sum of the likelihood ratios of each sub-band is obtained to obtain the likelihood ratio of each frame of the respiratory denoised signal. The expression formula is:

[0036]

[0037] In the formula, For the The weighting coefficients of the sub-bands; For the Likelihood ratio of the frame respiration denoised signal.

[0038] Furthermore, the respiratory denoising signal is segmented according to the likelihood ratio to obtain the respiratory segment signal, specifically including:

[0039] The adaptive threshold is calculated based on the mean and variance of the sub-band in each frame of the respiratory denoising signal. The expression formula is:

[0040]

[0041] In the formula, is the adaptive threshold, Indicates The threshold weight of each sub-band; Indicates The mean of the sub-bands in non-breathing mode; Indicates The mean of the sub-bands in breathing mode; Indicates The variance of each sub-band in non-respiratory mode; Expressed as The variance of each sub-band in the breathing mode;

[0042] The first The likelihood ratio of the frame respiratory denoising signal is compared with the adaptive threshold to determine whether each frame respiratory denoising signal belongs to a respiratory event or a non-respiratory event. The frames belonging to non-respiratory events are deleted from the respiratory denoising signal to obtain the respiratory segment signal.

[0043] Furthermore, the biometric identifier includes a body asymmetry identifier, an ear canal geometry identifier, and a respiratory tract descriptor; extracting the biometric identifier from the respiratory segment signal specifically includes:

[0044] Extract the start timestamp and end timestamp of the respiratory segment signal, and calculate the time difference of the respiratory segment signal. The expression formula is:

[0045]

[0046] In the formula, is the time difference of the respiratory segment signal; is the end time stamp of the respiratory segment signal; It is the starting time stamp of the respiratory segment signal;

[0047] If the time difference of the breathing segment signal exceeds the preset breathing time threshold , determine that the breathing segment signal is a valid breathing segment; otherwise, delete the breathing segment signal;

[0048] Get multiple consecutive valid breathing segments, calculate the time interval between the start timestamp of the current valid breathing segment and the end timestamp of the previous adjacent valid breathing segment, and record it as the first time interval. The expression formula is:

[0049]

[0050] In the formula, The starting time stamp of the current effective breathing segment; It is the end time stamp of the previous adjacent valid breathing segment; is the first time interval;

[0051] Calculate the time interval between the end timestamp of the current valid breathing segment and the end start timestamp of the next adjacent valid breathing segment, recorded as the second time interval, and the expression formula is:

[0052]

[0053] In the formula, is the second time interval; It is the end time stamp of the current effective breathing segment; It is the end point timestamp of the next adjacent valid breathing segment;

[0054] If the first time interval is greater than the second time interval, the effective breathing segment is determined to be an inhalation process; otherwise, the effective breathing segment is determined to be an exhalation process; the effective breathing segment of the inhalation process and the effective breathing segment of the adjacent exhalation process are combined into a single cycle of respiratory event features;

[0055] The respiratory event features are divided into left channel respiratory event features and right channel respiratory event features; the cross power spectrum density is calculated according to the left channel respiratory event features and the right channel respiratory event features; the body asymmetry identifier is obtained by calculating the cross power spectrum density and the auto-power spectrum of the right channel respiratory event features, and the expression formula is:

[0056]

[0057] In the formula, Identifier for body asymmetry; is the cross power spectral density; is the autopower spectrum of the respiratory event characteristics of the right channel;

[0058] Setting a number of target frequencies within the frequency range of the respiratory sound in the ear; calculating the power cumulative distribution characteristics of a single respiratory cycle according to the target frequencies;

[0059]

[0060] In the formula, It is the time-frequency diagram of respiratory event characteristics; is the starting frequency of the respiratory event feature; is the termination frequency characteristic of respiratory events; is the target frequency; It is the power accumulation distribution characteristic of a single breathing cycle;

[0061] The ear canal geometry identifier is composed of the power cumulative distribution characteristics corresponding to each target frequency;

[0062] Mel-frequency cepstral coefficients are extracted from respiratory event features, and the Mel-frequency cepstral coefficients and their first-order derivatives and second-order derivatives are used to form respiratory tract descriptors.

[0063] Furthermore, the triplet neural network includes three sub-network models; three groups of convolutional units and three fully connected layers are sequentially connected to form the sub-network model; the three groups of convolutional units are configured with sequentially connected two-dimensional convolutional blocks and maximum pooling layers;

[0064] The training process of the triplet neural network includes:

[0065] Acquire breathing audio training signals of multiple users from a database, multiply the audio waveform of the breathing audio training signal by a random factor to adjust the volume of the breathing sound; change the duration and speed of the breathing sound in the breathing audio training signal; add Gaussian noise to the breathing audio training signal, and move the breathing audio training signal along the time domain to obtain a breathing audio training sample;

[0066] Selecting a sample anchor point from the breathing audio training sample, setting the breathing audio training sample with the same user breathing sound as the sample anchor point as a positive sample, otherwise setting the breathing audio training sample as a negative sample;

[0067] Inputting the sample anchor point, the positive sample and the negative sample into the triplet neural network, the triplet neural network outputting the intra-class distance between the sample anchor point and the positive sample and the inter-class distance between the sample anchor point and the negative sample;

[0068] The training loss value is calculated based on the intra-class distance and inter-class distance. The expression formula is:

[0069]

[0070] In the formula, is the training loss value, is the sample anchor point, is a positive sample; is a negative sample; is the minimum distance between positive samples and negative samples; is the intra-class distance between the sample anchor point and the positive sample; is the inter-class distance between the sample anchor point and the negative sample;

[0071] The weight parameters of the triplet neural network are optimized according to the training loss value, and the training process of the triplet neural network is iterated repeatedly until the set number of iterations is reached to output the trained triplet neural network.

[0072] Furthermore, the biometric identifier is input into a pre-trained triplet neural network to calculate the similarity between the biometric identifier and a pre-stored biometric template; and user identity authentication is performed based on the similarity, specifically including:

[0073] The biometric identifier is input into the pre-trained triplet neural network to calculate the similarity between the biometric identifier and the pre-stored biometric template. The expression formula is:

[0074]

[0075] In the formula, is the similarity between the biometric identifier and the kth biometric template; is the sub-network model; Identifier for body asymmetry; is the ear canal geometry identifier; is the respiratory tract descriptor; is the body asymmetry identification feature in the kth biometric template; is the geometric identification feature of the auditory canal in the kth biometric template; Describe the respiratory tract features in the kth biometric template;

[0076] when When , it is determined that the biometric identifier belongs to the user identity corresponding to the kth biometric template; is the similarity threshold;

[0077] when When the biometric identifier is detected, it is determined that it belongs to an illegal user.

[0078] A second aspect of the present invention provides a user identity authentication system based on breath sounds, comprising:

[0079] The acquisition module is used to capture the in-ear breathing signal from the ear canal through the ear-worn device, and denoise the in-ear breathing signal to obtain a breathing denoised signal;

[0080] A segmentation module is used to use a Gaussian mixture model to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain a respiratory prior probability and a non-respiratory prior probability, and to calculate the likelihood ratio of each frame of the respiratory denoised signal by using the respiratory prior probability and the non-respiratory prior probability; and to segment the respiratory denoised signal according to the likelihood ratio to obtain a respiratory segment signal;

[0081] The recognition module is used to extract a biometric identifier from a respiratory segment signal; input the biometric identifier into a pre-trained ternary neural network, calculate the similarity between the biometric identifier and a pre-stored biometric template; and perform user identity authentication based on the similarity.

[0082] The third aspect of the present invention provides an electronic device, comprising a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the user identity authentication method described in the first aspect of the present invention.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] The present invention uses a Gaussian mixture model to respectively calculate the probability that each frame of respiratory denoised signal belongs to a respiratory event and a non-respiratory event to obtain a respiratory prior probability and a non-respiratory prior probability, and calculates the likelihood ratio of each frame of respiratory denoised signal through the respiratory prior probability and the non-respiratory prior probability; the respiratory denoised signal is segmented according to the likelihood ratio to obtain a respiratory segment signal; the present invention can effectively filter noise interference, reduce the impact of abnormal data, and enhance the robustness of the system in complex environments; it can accurately distinguish between respiratory and non-respiratory signals, adapt to the respiratory patterns of different users, and improve the accuracy of respiratory event detection.

[0085] The present invention extracts a biometric identifier from a respiratory segment signal; inputs the biometric identifier into a pre-trained ternary neural network, calculates the similarity between the biometric identifier and a pre-stored biometric template; performs user identity authentication based on the similarity; respiratory sounds as biometrics are difficult to forge, and combined with a deep learning ternary neural network, can accurately identify individual differences, effectively resist spoofing attacks, and significantly improve the security of identity authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 This is a flow chart of the user identity authentication method based on breathing sounds provided in Example 1.

[0087] Figure 2 is a flow chart of performing denoising processing on the in-ear breathing signal provided in Example 1;

[0088] Figure 3 is a flow chart of segmenting the respiratory denoising signal provided in Example 1;

[0089] Figure 4 is a flow chart of the training process of the triplet neural network provided in Example 1;

[0090] Figure 5 is a user identity authentication flow chart provided in Example 1;

[0091] Figure 6 It is a structural diagram of the triplet neural network provided in Example 1. DETAILED DESCRIPTION

[0092] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of the present invention.

[0093] Example 1

[0094] like Figure 1 As shown, this embodiment provides a user identity authentication method based on breathing sounds, including:

[0095] like Figure 6As shown, three groups of convolutional units and three fully connected layers are connected in sequence to form the sub-network model; a triplet neural network is constructed by the three sub-network models; the three groups of convolutional units are configured with two-dimensional convolutional blocks and maximum pooling layers connected in sequence; the two-dimensional convolutional block consists of a convolutional layer based on a ReLU activation function and a batch normalization layer; in this embodiment, the kernel size of the maximum pooling layer is 3x3, and the step size is 2.

[0096] like Figure 4 As shown, the training process of the triplet neural network includes:

[0097] Acquire breathing audio training signals of multiple users from a database, multiply the audio waveform of the breathing audio training signal by a random factor to adjust the volume of the breathing sound; change the duration and speed of the breathing sound in the breathing audio training signal; add Gaussian noise to the breathing audio training signal to shift the pitch of the breathing sound upward or downward; move the breathing audio training signal along the time domain to obtain a breathing audio training sample;

[0098] Selecting a sample anchor point from the breathing audio training sample, setting the breathing audio training sample with the same user breathing sound as the sample anchor point as a positive sample, otherwise setting the breathing audio training sample as a negative sample;

[0099] Inputting the sample anchor point, the positive sample and the negative sample into the triplet neural network, the triplet neural network outputting the intra-class distance between the sample anchor point and the positive sample and the inter-class distance between the sample anchor point and the negative sample;

[0100] The training loss value is calculated based on the intra-class distance and inter-class distance. The expression formula is:

[0101]

[0102] In the formula, is the training loss value, is the sample anchor point, is a positive sample; is a negative sample; is the minimum distance between positive samples and negative samples; is the intra-class distance between the sample anchor point and the positive sample; is the inter-class distance between the sample anchor point and the negative sample;

[0103] The Adam trainer is used to optimize the weight parameters of the triple neural network according to the training loss value, and the training process of the triple neural network is iterated repeatedly until the set number of iterations is reached to output the trained triple neural network; in this embodiment, the learning rate, batch size and number of iterations are set to 0.0001, 16 and 60 respectively.

[0104] This embodiment trains the model by constructing positive sample pairs and negative sample pairs and designing a specific loss function. During the training process, the model adjusts the feature space so that the breathing signal features of the same user are close to each other in the feature space (reducing the intra-class distance), while the breathing signal features of different users are far away from each other (increasing the inter-class distance). This optimization method enables the triplet neural network to more accurately identify the differences between different users, thereby improving the accuracy and reliability of identity authentication.

[0105] like Figure 2 As shown, the ear breathing signal is captured from the ear canal by an ear-worn device, and the ear breathing signal is denoised to obtain a breathing denoised signal, which specifically includes:

[0106] A recognition window is set on the intra-ear breathing signal; after moving the recognition window on the intra-ear breathing signal according to the set jump length, the power spectrum density of the intra-ear breathing signal in the recognition window is calculated, and when the power spectrum density exceeds a preset density threshold, the intra-ear breathing signal in the recognition window is discarded to obtain a first-level denoised breathing signal; in this embodiment, the window length of the recognition window is 10s and the jump length is 1s.

[0107] The first-stage noise reduction breathing signal is input into the bandpass filter, and the first-stage noise reduction breathing signal is filtered within the set frequency range to obtain the second-stage noise reduction breathing signal; since the noise generated by movement and heartbeat is mainly distributed below 150Hz, and the main frequency of breathing sound in the ear is 0 to 2KHz, the filtering range of the bandpass filter in this embodiment is set to 150 to 3000Hz.

[0108] Define the denoising parameter set of the maximum overlap discrete wavelet transform model, expressed as:

[0109]

[0110] In the formula, For the The set of denoising parameters in the genetic algorithm iteration; For the Mother wavelet function in genetic algorithm iteration; For the Decomposition level in genetic algorithm iteration; For the Threshold function in genetic algorithm iteration; For the The threshold is selected in the genetic algorithm iteration; For the Threshold rescaling function in genetic algorithm iteration;

[0111] Encode the denoising parameter set as chromosomes, and randomly generate a set number of chromosomes to form an initial population;

[0112] Based on the maximum overlap discrete wavelet transform model of the chromosome corresponding denoising parameter set, the original noisy signal is converted into a reconstructed signal and the mean square error is calculated. The expression formula is:

[0113]

[0114]

[0115] In the formula, is the maximum overlap discrete wavelet transform model, To reconstruct the signal; is the original noisy signal; is the mean square error; is the number of original noisy signals; n is the sequence number of the original noisy signal;

[0116] According to the mean square error, the chromosomes are crossovered and mutated and the initial population of chromosomes is updated. The optimization process of denoising parameters in the maximum overlap discrete wavelet transform model is repeated until the termination condition is met and the optimized maximum overlap discrete wavelet transform model is output.

[0117] The second-level denoised breathing signal is input into the optimized maximum overlap discrete wavelet transform model, the second-level denoised breathing signal is decomposed into multiple wavelet coefficients and scaling coefficients, the wavelet coefficients and scaling coefficients are thresholded and the signal is reconstructed to obtain the breathing denoised signal.

[0118] like Figure 3 As shown, the Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event to obtain the respiratory prior probability and the non-respiratory prior probability. The likelihood ratio of the respiratory denoised signal of each frame is calculated by the respiratory prior probability and the non-respiratory prior probability, which specifically includes:

[0119] Each frame of the respiration denoising signal is divided into W sub-bands according to the frequency; in this embodiment, W=4; the ranges of the four sub-bands are 0 to 500 Hz, 500 to 1000 Hz, 1000 to 1500 Hz and 1500 to 3000 Hz respectively.

[0120] The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoising signal belongs to a respiratory event to obtain the respiratory prior probability, which is expressed as:

[0121]

[0122] In the formula, is the prior probability of breathing; For the The first The logarithmic energy of the sub-bands; For the Parameter sets for sub-bands; is the category label of the breathing denoised signal of each frame; For the The mean of the sub-bands; For the The variance of the sub-bands; is the ratio of pi; is the natural exponential function;

[0123] The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a non-respiratory event to obtain the non-respiratory prior probability, which is expressed as:

[0124]

[0125] In the formula, is the non-breathing prior probability;

[0126] The likelihood ratio of the sub-band in each frame of the respiratory denoised signal is calculated by the respiratory prior probability and the non-respiratory prior probability. The expression formula is:

[0127]

[0128] In the formula, For the The first The likelihood ratio of each sub-band;

[0129] According to the power cumulative distribution of each sub-band, a weight is assigned to the likelihood ratio of each sub-band, and the weighted sum of the likelihood ratios of each sub-band is obtained to obtain the likelihood ratio of each frame of the respiratory denoised signal. The expression formula is:

[0130]

[0131] In the formula, For the The weighting coefficients of the sub-bands; For the Likelihood ratio of the frame respiration denoised signal.

[0132] The respiratory denoising signal is segmented according to the likelihood ratio to obtain the respiratory segment signal, specifically including:

[0133] The adaptive threshold is calculated based on the mean and variance of the sub-band in each frame of the respiratory denoising signal. The expression formula is:

[0134]

[0135] In the formula, is the adaptive threshold, Indicates The threshold weight of each sub-band; Indicates The mean of the sub-bands in non-breathing mode; Indicates The mean of the sub-bands in breathing mode; Indicates The variance of each sub-band in non-respiratory mode; Expressed as The variance of each sub-band in the breathing mode;

[0136] The first The likelihood ratio of the frame respiratory denoising signal is compared with the adaptive threshold to determine whether each frame respiratory denoising signal belongs to a respiratory event or a non-respiratory event. The frames belonging to non-respiratory events are deleted from the respiratory denoising signal to obtain the respiratory segment signal.

[0137] The biometric identifier includes a body asymmetry identifier, an ear canal geometry identifier, and a respiratory tract descriptor; the biometric identifier is extracted from the respiratory segment signal, specifically including:

[0138] Extract the start timestamp and end timestamp of the respiratory segment signal, and calculate the time difference of the respiratory segment signal. The expression formula is:

[0139]

[0140] In the formula, is the time difference of the respiratory segment signal; is the end time stamp of the respiratory segment signal; It is the starting time stamp of the respiratory segment signal;

[0141] If the time difference of the breathing segment signal exceeds the preset breathing time threshold , determine that the breathing segment signal is a valid breathing segment; otherwise, delete the breathing segment signal;

[0142] Get multiple consecutive valid breathing segments, calculate the time interval between the start timestamp of the current valid breathing segment and the end timestamp of the previous adjacent valid breathing segment, and record it as the first time interval. The expression formula is:

[0143]

[0144] In the formula, The starting time stamp of the current effective breathing segment; It is the end time stamp of the previous adjacent valid breathing segment; is the first time interval;

[0145] Calculate the time interval between the end timestamp of the current valid breathing segment and the end start timestamp of the next adjacent valid breathing segment, recorded as the second time interval, and the expression formula is:

[0146]

[0147] In the formula, is the second time interval; It is the end time stamp of the current effective breathing segment; It is the end point timestamp of the next adjacent valid breathing segment;

[0148] If the first time interval is greater than the second time interval, the effective breathing segment is determined to be an inhalation process; otherwise, the effective breathing segment is determined to be an exhalation process; the effective breathing segment of the inhalation process and the effective breathing segment of the adjacent exhalation process are combined into a single cycle of respiratory event features;

[0149] like Figure 5 As shown, the respiratory event features are divided into left channel respiratory event features and right channel respiratory event features; the cross power spectrum density is calculated according to the left channel respiratory event features and the right channel respiratory event features; the body asymmetry identifier is obtained by calculating the cross power spectrum density and the auto-power spectrum of the right channel respiratory event features, and the expression formula is:

[0150]

[0151] In the formula, Identifier for body asymmetry; is the cross power spectral density; is the autopower spectrum of the respiratory event characteristics of the right channel;

[0152] In this embodiment, the individual differences and asymmetries of human bones are utilized. The vibrations caused by breathing are transmitted to the left and right ear canals through the bones in the body. The left and right channel signals captured by the microphone are regarded as two independent signals with different bone conduction paths. The cross power spectral density (CPSD) is calculated separately to obtain the correlation within the frequency spectrum, thereby obtaining a body asymmetry identifier.

[0153] Setting a number of target frequencies within the frequency range of the respiratory sound in the ear; calculating the power cumulative distribution characteristics of a single respiratory cycle according to the target frequencies;

[0154]

[0155] In the formula, It is the time-frequency diagram of respiratory event characteristics; is the starting frequency of the respiratory event feature; is the termination frequency characteristic of respiratory events; is the target frequency; is the power cumulative distribution characteristic of a single breathing cycle; in this embodiment, the target frequency Settings are 500Hz, 1000Hz, 1500Hz and 3000Hz;

[0156] The power cumulative distribution characteristics corresponding to each target frequency form an ear canal geometry identifier; this embodiment utilizes the differences in ear canal structure between individuals, and the impedance effect of the occlusion effect on breath sounds varies from person to person. In order to characterize the differences caused by the occlusion effect between users, this embodiment calculates the power cumulative distribution of a single breathing cycle, and then obtains the ear canal geometry identifier.

[0157] Mel frequency cepstral coefficients are extracted from the respiratory event features, and the Mel frequency cepstral coefficients and their first-order derivatives and second-order derivatives are used to form respiratory descriptors. In this embodiment, during breathing, high-speed airflow collides with the respiratory tract and causes resonance, and respiratory descriptors are extracted based on significant individual differences in resonance frequency.

[0158] The biometric identifier is input into a pre-trained triplet neural network, and the similarity between the biometric identifier and the pre-stored biometric template is calculated; user identity authentication is performed based on the similarity, specifically including:

[0159] The biometric identifier is input into the pre-trained triplet neural network to calculate the similarity between the biometric identifier and the pre-stored biometric template. The expression formula is:

[0160]

[0161] In the formula, is the similarity between the biometric identifier and the kth biometric template; is the sub-network model; Identifier for body asymmetry; is the ear canal geometry identifier; is the respiratory tract descriptor; is the body asymmetry identification feature in the kth biometric template; is the geometric identification feature of the auditory canal in the kth biometric template; Describe the respiratory tract features in the kth biometric template;

[0162] when When , it is determined that the biometric identifier belongs to the user identity corresponding to the kth biometric template; is the similarity threshold;

[0163] when When the biometric identifier is detected, it is determined that it belongs to an illegal user.

[0164] This embodiment first denoises the breathing signal in the ear to obtain a breathing denoised signal, then uses a Gaussian mixture model to calculate the probability that each frame of the breathing denoised signal belongs to a breathing event and a non-respiratory event to obtain a breathing prior probability and a non-respiratory prior probability, and then segments the breathing denoised signal to obtain a breathing segment signal, so that this embodiment has resistance to environmental noise and changes in breathing behavior and has strong adaptability; by extracting biometric features that are difficult to forge, it can resist deception attacks to a high degree.

[0165] Example 2

[0166] This embodiment provides a user identity authentication system based on breathing sounds. The system described in this embodiment can be applied to the method described in Example 1. The user identity authentication system includes:

[0167] The acquisition module is used to capture the in-ear breathing signal from the ear canal through the ear-worn device, and denoise the in-ear breathing signal to obtain a breathing denoised signal;

[0168] A segmentation module is used to use a Gaussian mixture model to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain a respiratory prior probability and a non-respiratory prior probability, and to calculate the likelihood ratio of each frame of the respiratory denoised signal by using the respiratory prior probability and the non-respiratory prior probability; and to segment the respiratory denoised signal according to the likelihood ratio to obtain a respiratory segment signal;

[0169] The recognition module is used to extract a biometric identifier from a respiratory segment signal; input the biometric identifier into a pre-trained ternary neural network, calculate the similarity between the biometric identifier and a pre-stored biometric template; and perform user identity authentication based on the similarity.

[0170] The acquisition module performs denoising on the in-ear respiratory signal to obtain a respiratory denoised signal, specifically comprising:

[0171] A recognition window is set on the intra-ear breathing signal; after moving the recognition window on the intra-ear breathing signal according to the set jump length, the power spectrum density of the intra-ear breathing signal in the recognition window is calculated, and when the power spectrum density exceeds a preset density threshold, the intra-ear breathing signal in the recognition window is discarded to obtain a first-level noise reduction breathing signal;

[0172] The first-stage noise reduction breathing signal is input into a bandpass filter, and the first-stage noise reduction breathing signal is filtered within a set frequency range to obtain a second-stage noise reduction breathing signal;

[0173] The denoising parameters of the maximum overlap discrete wavelet transform model are optimized by genetic algorithm; the second-level denoised respiratory signal is input into the optimized maximum overlap discrete wavelet transform model, the second-level denoised respiratory signal is decomposed into multiple wavelet coefficients and scaling coefficients, the wavelet coefficients and scaling coefficients are thresholded and the signal is reconstructed to obtain the respiratory denoising signal.

[0174] The biometric identifier includes a body asymmetry identifier, an ear canal geometry identifier and a respiratory tract descriptor; the recognition module extracts the biometric identifier from the respiratory segment signal, specifically including:

[0175] Extract the start timestamp and end timestamp of the respiratory segment signal, and calculate the time difference of the respiratory segment signal. The expression formula is:

[0176]

[0177] In the formula, is the time difference of the respiratory segment signal; is the end time stamp of the respiratory segment signal; It is the starting time stamp of the breathing segment signal;

[0178] If the time difference of the breathing segment signal exceeds the preset breathing time threshold , determine that the breathing segment signal is a valid breathing segment; otherwise, delete the breathing segment signal;

[0179] Get multiple consecutive valid breathing segments, calculate the time interval between the start timestamp of the current valid breathing segment and the end timestamp of the previous adjacent valid breathing segment, and record it as the first time interval. The expression formula is:

[0180]

[0181] In the formula, The starting time stamp of the current effective breathing segment; It is the end time stamp of the previous adjacent valid breathing segment; is the first time interval;

[0182] Calculate the time interval between the end timestamp of the current valid breathing segment and the end start timestamp of the next adjacent valid breathing segment, recorded as the second time interval, and the expression formula is:

[0183]

[0184] In the formula, is the second time interval; It is the end time stamp of the current effective breathing segment; It is the end point timestamp of the next adjacent valid breathing segment;

[0185] If the first time interval is greater than the second time interval, the effective breathing segment is determined to be an inhalation process; otherwise, the effective breathing segment is determined to be an exhalation process; the effective breathing segment of the inhalation process and the effective breathing segment of the adjacent exhalation process are combined into a single cycle of respiratory event features;

[0186] The respiratory event features are divided into left channel respiratory event features and right channel respiratory event features; the cross power spectrum density is calculated according to the left channel respiratory event features and the right channel respiratory event features; the body asymmetry identifier is obtained by calculating the cross power spectrum density and the auto-power spectrum of the right channel respiratory event features, and the expression formula is:

[0187]

[0188] In the formula, Identifier for body asymmetry; is the cross power spectral density; is the autopower spectrum of the respiratory event characteristics of the right channel;

[0189] Setting a number of target frequencies within the frequency range of the respiratory sound in the ear; calculating the power cumulative distribution characteristics of a single respiratory cycle according to the target frequencies;

[0190]

[0191] In the formula, It is the time-frequency diagram of respiratory event characteristics; is the starting frequency of the respiratory event feature; is the termination frequency characteristic of respiratory events; is the target frequency; It is the power accumulation distribution characteristic of a single breathing cycle;

[0192] The ear canal geometry identifier is composed of the power cumulative distribution characteristics corresponding to each target frequency;

[0193] Mel-frequency cepstral coefficients are extracted from respiratory event features, and the Mel-frequency cepstral coefficients and their first-order derivatives and second-order derivatives are used to form respiratory tract descriptors.

[0194] Example 3

[0195] This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the user identity authentication method described in Example 1.

[0196] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0197] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0198] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0200] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A user identity authentication method based on breathing sound, characterized in that: include: The ear breathing signal is captured from the ear canal by using an ear-worn device, and the ear breathing signal is denoised to obtain a breathing denoised signal; The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain the respiratory prior probability and the non-respiratory prior probability. The likelihood ratio of the respiratory denoised signal of each frame is calculated by the respiratory prior probability and the non-respiratory prior probability. The respiratory denoised signal is segmented according to the likelihood ratio to obtain the respiratory segment signal. A biometric identifier is extracted from a respiratory segment signal; the biometric identifier is input into a pre-trained ternary neural network to calculate the similarity between the biometric identifier and a pre-stored biometric template; and user identity authentication is performed based on the similarity.

2. The user identity authentication method based on breathing sound according to claim 1, characterized in that: The in-ear breathing signal is denoised to obtain a breathing denoised signal, specifically including: A recognition window is set on the intra-ear breathing signal; after moving the recognition window on the intra-ear breathing signal according to the set jump length, the power spectrum density of the intra-ear breathing signal in the recognition window is calculated, and when the power spectrum density exceeds a preset density threshold, the intra-ear breathing signal in the recognition window is discarded to obtain a first-level noise reduction breathing signal; The first-stage noise reduction breathing signal is input into a bandpass filter, and the first-stage noise reduction breathing signal is filtered within a set frequency range to obtain a second-stage noise reduction breathing signal; The denoising parameters of the maximum overlap discrete wavelet transform model are optimized by genetic algorithm; the second-level denoised respiratory signal is input into the optimized maximum overlap discrete wavelet transform model, the second-level denoised respiratory signal is decomposed into multiple wavelet coefficients and scaling coefficients, the wavelet coefficients and scaling coefficients are thresholded and the signal is reconstructed to obtain the respiratory denoising signal.

3. The user identity authentication method based on breathing sound according to claim 2, characterized in that: The denoising parameters of the maximum overlap discrete wavelet transform model are optimized by genetic algorithm, including: Define the denoising parameter set of the maximum overlap discrete wavelet transform model, expressed as: ； In the formula, For the The set of denoising parameters in the genetic algorithm iteration; For the Mother wavelet function in genetic algorithm iteration; For the Decomposition level in genetic algorithm iteration; For the Threshold function in genetic algorithm iteration; For the The threshold is selected in the genetic algorithm iteration; For the Threshold rescaling function in genetic algorithm iteration; Encode the denoising parameter set as chromosomes, and randomly generate a set number of chromosomes to form an initial population; Based on the maximum overlap discrete wavelet transform model of the chromosome corresponding denoising parameter set, the original noisy signal is converted into a reconstructed signal and the mean square error is calculated. The expression formula is: ； ； In the formula, is the maximum overlap discrete wavelet transform model, To reconstruct the signal; is the original noisy signal; is the mean square error; is the number of original noisy signals; n is the sequence number of the original noisy signal; According to the mean square error, the chromosomes are crossovered and mutated and the initial population of chromosomes is updated. The optimization process of denoising parameters in the maximum overlap discrete wavelet transform model is repeated until the termination condition is met and the optimized maximum overlap discrete wavelet transform model is output.

4. The user identity authentication method based on breathing sound according to claim 1, characterized in that: The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain the respiratory prior probability and the non-respiratory prior probability. The likelihood ratio of the respiratory denoised signal of each frame is calculated by the respiratory prior probability and the non-respiratory prior probability, including: Divide each frame of the breathing denoised signal into W sub-bands according to the frequency; The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoising signal belongs to a respiratory event to obtain the respiratory prior probability, which is expressed as: ； In the formula, is the prior probability of breathing; For the The first The logarithmic energy of the sub-bands; For the Parameter sets for sub-bands; is the category label of the breathing denoised signal of each frame; For the The mean of the sub-bands; For the The variance of the sub-bands; is the ratio of pi; is the natural exponential function; The Gaussian mixture model is used to calculate the probability that each frame of the respiratory denoised signal belongs to a non-respiratory event to obtain the non-respiratory prior probability, which is expressed as: ； In the formula, is the non-breathing prior probability; The likelihood ratio of the sub-band in each frame of the respiratory denoised signal is calculated by the respiratory prior probability and the non-respiratory prior probability. The expression formula is: ； In the formula, For the The first The likelihood ratio of each sub-band; According to the power cumulative distribution of each sub-band, a weight is assigned to the likelihood ratio of each sub-band, and the weighted sum of the likelihood ratios of each sub-band is obtained to obtain the likelihood ratio of each frame of the respiratory denoised signal. The expression formula is: ； In the formula, For the The weighting coefficients of the sub-bands; For the Likelihood ratio of the frame breathing denoised signal; W is the number of divided sub-bands.

5. The user identity authentication method based on breathing sound according to claim 4, characterized in that: The respiratory denoising signal is segmented according to the likelihood ratio to obtain the respiratory segment signal, specifically including: The adaptive threshold is calculated based on the mean and variance of the sub-band in each frame of the respiratory denoising signal. The expression formula is: ； In the formula, is the adaptive threshold, Indicates The threshold weight of each sub-band; Indicates The mean of the sub-bands in non-breathing mode; Indicates The mean of the sub-bands in breathing mode; Indicates The variance of each sub-band in non-respiratory mode; Expressed as The variance of the sub-bands in the breathing mode; W is the number of divided sub-bands; The first The likelihood ratio of the frame respiratory denoising signal is compared with the adaptive threshold to determine whether each frame respiratory denoising signal belongs to a respiratory event or a non-respiratory event. The frames belonging to non-respiratory events are deleted from the respiratory denoising signal to obtain the respiratory segment signal.

6. The user identity authentication method based on breathing sound according to claim 1, characterized in that: The biometric identifier includes a body asymmetry identifier, an ear canal geometry identifier, and a respiratory tract descriptor; the biometric identifier is extracted from the respiratory segment signal, specifically including: Extract the start timestamp and end timestamp of the respiratory segment signal, and calculate the time difference of the respiratory segment signal. The expression formula is: ； In the formula, is the time difference of the respiratory segment signal; is the end time stamp of the respiratory segment signal; It is the starting time stamp of the respiratory segment signal; If the time difference of the breathing segment signal exceeds the preset breathing time threshold , determine that the breathing segment signal is a valid breathing segment; otherwise, delete the breathing segment signal; Get multiple consecutive valid breathing segments, calculate the time interval between the start timestamp of the current valid breathing segment and the end timestamp of the previous adjacent valid breathing segment, and record it as the first time interval. The expression formula is: ； In the formula, The starting time stamp of the current effective breathing segment; It is the end time stamp of the previous adjacent valid breathing segment; is the first time interval; Calculate the time interval between the end timestamp of the current valid breathing segment and the end start timestamp of the next adjacent valid breathing segment, which is recorded as the second time interval. The expression formula is: ； In the formula, is the second time interval; It is the end time stamp of the current effective breathing segment; It is the end point timestamp of the next adjacent valid breathing segment; If the first time interval is greater than the second time interval, the effective breathing segment is determined to be an inhalation process; otherwise, the effective breathing segment is determined to be an exhalation process; the effective breathing segment of the inhalation process and the effective breathing segment of the adjacent exhalation process are combined into a single cycle of respiratory event features; The respiratory event features are divided into left channel respiratory event features and right channel respiratory event features; the cross power spectrum density is calculated according to the left channel respiratory event features and the right channel respiratory event features; the body asymmetry identifier is obtained by calculating the cross power spectrum density and the auto-power spectrum of the right channel respiratory event features, and the expression formula is: ； In the formula, Identifier for body asymmetry; is the cross power spectral density; is the autopower spectrum of the respiratory event characteristics of the right channel; Setting a number of target frequencies within the frequency range of the respiratory sound in the ear; calculating the power cumulative distribution characteristics of a single respiratory cycle according to the target frequencies; ； In the formula, It is the time-frequency diagram of respiratory event characteristics; is the starting frequency of the respiratory event feature; is the termination frequency characteristic of respiratory events; is the target frequency; It is the power accumulation distribution characteristic of a single breathing cycle; The ear canal geometry identifier is composed of the power cumulative distribution characteristics corresponding to each target frequency; Mel-frequency cepstral coefficients are extracted from respiratory event features, and the Mel-frequency cepstral coefficients and their first-order derivatives and second-order derivatives are used to form respiratory tract descriptors.

7. The user identity authentication method based on breathing sound according to claim 6, characterized in that: The triplet neural network includes three sub-network models; three groups of convolution units and three fully connected layers are sequentially connected to form the sub-network model; the three groups of convolution units are configured with sequentially connected two-dimensional convolution blocks and maximum pooling layers; The training process of the triplet neural network includes: Acquire breathing audio training signals of multiple users from a database, and multiply the audio waveform of the breathing audio training signal by a random factor to adjust the volume of the breathing sound; Changing the duration and speed of the breathing sound in the breathing audio training signal; adding Gaussian noise to the breathing audio training signal, and moving the breathing audio training signal along the time domain to obtain a breathing audio training sample; Selecting a sample anchor point from the breathing audio training sample, setting the breathing audio training sample with the same user breathing sound as the sample anchor point as a positive sample, otherwise setting the breathing audio training sample as a negative sample; Inputting the sample anchor point, the positive sample and the negative sample into the triplet neural network, the triplet neural network outputting the intra-class distance between the sample anchor point and the positive sample and the inter-class distance between the sample anchor point and the negative sample; The training loss value is calculated based on the intra-class distance and inter-class distance. The expression formula is: ； In the formula, is the training loss value, is the sample anchor point, is a positive sample; is a negative sample; is the minimum distance between positive samples and negative samples; is the intra-class distance between the sample anchor point and the positive sample; is the inter-class distance between the sample anchor point and the negative sample; The weight parameters of the triplet neural network are optimized according to the training loss value, and the training process of the triplet neural network is iterated repeatedly until the set number of iterations is reached to output the trained triplet neural network.

8. The user identity authentication method based on breathing sound according to claim 7, characterized in that: Inputting the biometric identifier into a pre-trained triplet neural network and calculating the similarity between the biometric identifier and a pre-stored biometric template; User identity authentication is performed based on similarity, including: The biometric identifier is input into the pre-trained triplet neural network to calculate the similarity between the biometric identifier and the pre-stored biometric template. The expression formula is: ； In the formula, is the similarity between the biometric identifier and the kth biometric template; is the sub-network model; Identifier for body asymmetry; is the ear canal geometry identifier; is the respiratory tract descriptor; is the body asymmetry identification feature in the kth biometric template; is the geometric identification feature of the auditory canal in the kth biometric template; Describe the respiratory tract features in the kth biometric template; when When , it is determined that the biometric identifier belongs to the user identity corresponding to the kth biometric template; is the similarity threshold; when When the biometric identifier is detected, it is determined that it belongs to an illegal user.

9. A user identity authentication system based on breathing sound, characterized in that: include: An acquisition module is used to capture the in-ear breathing signal from the ear canal through an ear-worn device, and denoise the in-ear breathing signal to obtain a breathing denoised signal; A segmentation module is used to use a Gaussian mixture model to calculate the probability that each frame of the respiratory denoised signal belongs to a respiratory event and a non-respiratory event, respectively, to obtain a respiratory prior probability and a non-respiratory prior probability, and to calculate the likelihood ratio of each frame of the respiratory denoised signal by using the respiratory prior probability and the non-respiratory prior probability; and to segment the respiratory denoised signal according to the likelihood ratio to obtain a respiratory segment signal; The recognition module is used to extract a biometric identifier from a respiratory segment signal; input the biometric identifier into a pre-trained ternary neural network, calculate the similarity between the biometric identifier and a pre-stored biometric template; and perform user identity authentication based on the similarity.

10. An electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; characterized in that: The processor is used to operate according to the instruction to execute the user identity authentication method described in any one of claims 1 to claim 8.

Images