Person authentication device using living body vibration signal, person authentication method, and program

Abstract

To authenticate a person on the basis of a living body vibration signal acquired using a piezoelectric sensor.SOLUTION: A living body vibration signal acquired by a piezoelectric sensor is short-time Fourier transformed to generate a spectrogram image, or an AR model reflection coefficient, a regression coefficient, and an autocorrelation function are calculated, the generated or calculated result is input to a person authentication part that has performed machine learning in advance, and thereby, a person who is a source of the living body vibration signal is authenticated.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technology for authenticating a person from a biological vibration signal acquired by a piezoelectric sensor. [Background technology]

[0002] The inventor of the present invention has been developing a technology for estimating an electrocardiogram (ECG) or a sleep state from a biovibration signal acquired by a piezoelectric sensor (Patent Document 1, Patent Document 2), when a new need has arisen to identify and identify the subject. Conventionally, biometric authentication technologies such as fingerprint authentication, iris authentication, and vein authentication have been proposed to identify individuals, but these do not sense unconsciously, and involve specific actions for authentication, such as the subject pressing a finger or palm against a sensor or moving the face into the camera's imaging frame. By measuring biovibration signals unconsciously and unrestrainedly using a piezoelectric sensor, it becomes possible to perform personal authentication without putting stress on the subject. As a related technology, Non-Patent Document 1 describes the possibility of identifying individuals from the frequency spectrum of biological vibrations obtained using a piezoelectric sensor installed on a chair, but does not propose any specific methods, so it is at the level of suggestion. Therefore, the inventor has invented a technology for authenticating a person based on their biological vibration signals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2022 / 176221 [Patent Document 2] WO2022 / 269936 [Non-patent literature]

[0004] [Non-Patent Document 1] A study on the analysis of biosignals obtained by piezoelectric sensors (Daiichi Institute of Technology Research Report No. 31 (2019) pp. 43-48) Summary of the Invention [Problem to be solved by the invention]

[0005] The objective is to provide a technology for authenticating people based on bio-vibration signals acquired by a sheet-shaped piezoelectric sensor. [Means for solving the problem]

[0006] As a person authentication device, A bio-vibration signal receiving unit that receives a human bio-vibration signal; A spectrogram image generating unit that generates a spectrogram image from the biological vibration signal; A person authentication unit that authenticates a person from the spectrogram image, The person authentication unit inputs an identifier of the person and the spectrogram image, performs machine learning to learn a relationship between the identifier and the spectrogram image in advance, and inputs the spectrogram image of a subject to output an identifier of that person. In addition, instead of the spectrogram image generation unit, an AR model coefficient / autocorrelation function analysis unit is provided that calculates the reflection coefficients, regression coefficients, and autocorrelation function of an autoregressive (AR) model from the biological vibration signal, and the reflection coefficients, regression coefficients, and autocorrelation function of the AR model calculated by the AR model coefficient / autocorrelation function analysis unit are used for machine learning and authentication in the person authentication unit. Effect of the Invention

[0007] The subject, who is the source of the signal, can be identified from the subject's bio-vibration signal acquired by the piezoelectric sensor, without restraining the subject or making the subject aware of the signal. For example, a piezoelectric sensor can be installed on a toilet seat in a public restroom to identify a user sitting on it without causing any stress to the user. In the case of a chair, a piezoelectric sensor can be placed under the buttocks, on the seat, etc., and the subject can be identified from the biosignals of the subject acquired by the piezoelectric sensor. Similarly, even when a subject is in bed or on a sofa, the subject can be identified from the bio-vibration signals acquired using a piezoelectric sensor. It is also possible to add the identifier to the bio-vibration signal, and to label which subject the vital information estimated from the bio-vibration signal belongs to. It can also be used as a substitute for biometric authentication methods such as fingerprint authentication, face authentication, iris authentication, and vein authentication. [Brief description of the drawings]

[0008] [Figure 1] Schematic block diagram of human authentication device (part 1) [Diagram 2] BCG signal and its short-time Fourier transform spectrogram, and input of the spectrogram [Diagram 3] Confusion matrix by cross-validation using short-time Fourier transform spectrogram images of BCG signals [Figure 4] Block diagram of human authentication device (part 2) [Diagram 5] Ensemble averaging of short-time Fourier transform spectrograms of BCG signals [Figure 6] Confusion matrix by cross-validation using ensemble average images of short-time Fourier transform spectrograms of BCG signals [Figure 7] Block diagram of human authentication device (part 3) [Figure 8] AR model reflection coefficients, regression coefficients, autocorrelation functions and their ensemble averages for BCG signals from a single subject [Figure 9] Five measurements of the ensemble average of the AR model reflection coefficients, regression coefficients, and autocorrelation functions of the BCG signal from the same subject [Figure 10] Confusion matrix by cross-validation using AR model reflection coefficients, regression coefficients, and autocorrelation function of BCG signals [Figure 11] Short-time Fourier transform spectrogram image of the ECG signal [Figure 12] Confusion matrix by cross-validation using short-time Fourier transform spectrogram images of ECG signals [Figure 13] Short-time Fourier transform spectrogram image of ECG signal and ensemble average spectrogram image obtained by segmentation [Figure 14] Confusion matrix by cross-validation using segmented ensemble average spectrogram images from short-time Fourier transform of ECG signals [Figure 15] AR model reflection coefficients, regression coefficients, autocorrelation functions and their ensemble averages for a single subject's ECG signal [Figure 16] Five measurements of the ensemble average of the AR model reflection coefficients, regression coefficients, and autocorrelation functions of ECG signals from the same subject [Figure 17] Confusion matrix by cross-validation using ensemble average of AR model reflection coefficients, regression coefficients, and autocorrelation functions of ECG signals [Figure 18] Schematic top view of a toilet seat with electrodes installed [Figure 19] Schematic side view of a chair with electrodes installed [Figure 20] Schematic side view of a bathtub with electrodes installed DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An overview of the first embodiment of the present invention will be described with reference to FIG. The human authentication system 1 is composed of a human authentication device 2 and a sensor unit 3, and inputs a bio-vibration signal detected by the sensor unit 3 or a bio-vibration signal acquired in advance and stored in memory into the human authentication device 2, and outputs an identifier of the person who is the source of the bio-vibration signal. The human authentication device 2 is the core of the present invention and is composed of a biological vibration signal receiving unit 21, a spectrogram image generating unit 22, and a human authentication unit 24. Each component of the person authentication device 2 is usually implemented on a computer such as a personal computer, a smartphone, or a tablet that is equipped with a CPU, memory, external memory, and communication means.

[0010] Each component will now be described in detail. The sensor unit 3 detects a person's biovibration and outputs a biovibration signal. For example, the sensor unit 3 is a sheet-shaped piezoelectric sensor that is installed on a mat on which a person lies and outputs a biovibration signal based on body movement, heartbeat (ballistocardiogram caused by the beating of the heart), respiratory movement, and also vocalization, which may include a signal caused by vibration based on the external environment, etc. The sensor unit 3 is not limited to a sheet-shaped piezoelectric sensor, but may be configured in combination with an electrocardiograph, a pulse wave meter, and the like. For example, electrodes can be placed on a toilet seat or the armrest of a chair, and when an elderly or other user's thigh touches the toilet seat, or their forearm or palm touches the armrest, the toilet seat or armrest becomes an electrode for an electrocardiograph, and an electrocardiogram can be obtained from the subject, making it possible to identify the subject. Furthermore, a photoplethysmograph can be placed on the toilet seat or armrest to obtain pulse waves from the buttocks, forearm, or palm, allowing the subject to be identified.

[0011] The biovibration signal receiving unit 21 receives the biovibration signal output by the sensor unit 3 connected via a communication means such as a cable or wirelessly. The received bio-vibration signal is stored in the internal memory of the person authentication device 2 or in an external memory (not shown) such as a USB memory.

[0012] The spectrogram image generating unit 22 performs preprocessing (not shown) on the biological vibration signal as necessary, performs a Short-Time Fourier Transform (STFT) and generates a frequency spectrogram image. The preprocessing performed as necessary is a process of passing the biological vibration signal through a high-pass filter of 0.7 Hz or more when it is desired to block signals caused by breathing contained in the biological vibration signal, and a differentiation process when it is desired to emphasize changes in the signal caused by the heartbeat, but is not limited to these. Then, a frequency band of 0.7 to 10 Hz, which is the frequency component of vibrations (ballistocardiogram) originating from the beating of the heart, which is useful for human authentication, is extracted. The frequency band is adjustable, and the wider the frequency band, the larger the image size. The ballistocardiogram waveform is a vibration caused by the pressure pulse wave caused by the beating of the heart propagating to the body surface, and the vibration pattern, vibration amplitude, and degree of vibration attenuation change depending on the heart rate, heart rate variability, body tissue composition, etc. of the person to be identified. These are reflected in the time-frequency distribution of the spectrogram.

[0013] The person authentication unit 24 first inputs a set of data including a spectrogram image and the person's identifier into a two-dimensional Convolutional Neural Network (CNN), and performs machine learning to learn the relationship between the spectrogram image of the biovibration signal of the person to be authenticated, obtained using a piezoelectric sensor, and the person's identifier. When the person authentication unit 24, which includes a machine learning model, receives a spectrogram image of the biological vibration signal acquired using a piezoelectric sensor, it outputs the person's identifier.

[0014] The second embodiment will be outlined with reference to Fig. 4. The difference from the first embodiment is that a post-processing unit 23 is added next to the spectrogram image generating unit 22. Components common to the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and their description will be omitted, and only different components will be described.

[0015] The post-processing unit 23, for example, performs an averaging process on the spectrogram image generated by the spectrogram image generating unit 22. Since the biological vibration signal may contain accidental disturbance signals and body movement signals, in order to reduce uncertainty, the post-processing unit 23 divides the spectrogram into segments and calculates an ensemble average. Furthermore, the post-processing unit 23 performs, for example, a histogram equalization process on the spectrogram image generated by the spectrogram image generating unit 22. The histogram equalization process is to adjust pixel values ​​so that the histogram of pixel values ​​becomes uniform overall. The person authentication unit 24 inputs the post-processed spectrogram image and performs machine learning and authentication.

[0016] An overview of the third embodiment will be explained using FIG. 7. What differs from the first embodiment is that an autoregressive (AR) model / autocorrelation function analysis unit 26 is added after the biological vibration signal receiving unit 21. The AR model is a stochastic process model in which the current data of a time series depends on a linear combination of past data, and when the model is applied to a heartbeat signal time series, it is expected that the AR order and AR regression coefficients will differ from person to person. In addition, in the autocorrelation function of the heartbeat signal time series, information on an individual's heartbeat cycle and heartbeat variability is reflected in the peak of the autocorrelation function and its decay pattern. Components common to the first embodiment shown in FIG. 1 are denoted by the same reference numerals, and their description will be omitted, and only different components will be described.

[0017] The AR model and autocorrelation function analysis unit 26 divides 30 seconds of BCG data (time series data obtained by cutting low-frequency components below 0.7 Hz from biological vibration signals and extracting components derived from heartbeats) into five segments with 10-second intervals, and calculates the ensemble average of the AR reflection coefficient (Ref Coef), AR regression coefficient (AR Coef), and autocorrelation function (ACF) for each segment time series. The AR reflection coefficient corresponds to the negative value of the partial autocorrelation function. The person authentication unit 24 inputs the AR reflection coefficient (Ref Coef), the AR regression coefficient (AR Coef), and the ensemble average of the autocorrelation function (ACF) to perform machine learning and authentication. EXAMPLES

[0018] First, using a piezoelectric sensor, bio-vibration signals from the buttocks of 13 subjects were obtained five times for each subject, each time for three minutes, at a sampling frequency of 100 Hz (top of Figure 2). The biovibration signal of the buttocks for any 30 seconds out of the 3 minutes was received by the biovibration signal receiving unit 21 and passed to the spectrogram image generating unit 22. In the spectrogram image generating unit 22, the biological vibration signals of a predetermined length of time (512 signals x 1 / 100 Hz = 5.12 seconds) were overlapped by 95% and a short-time Fourier transform (STFT) was performed to extract frequency components from 0.7 to 10 Hz (middle part of Figure 2). Next, a spectrogram image was generated (Figure 2, bottom). The size of the image corresponding to the frequency band of 0.7 to 10 Hz is 48 x 96, with 48 pixels vertically (frequency axis) and 96 pixels horizontally (time axis). Five spectrogram images were generated for each subject, and since there were 13 subjects, the total number of spectrogram images was 13 x 5 = 65.

[0019] In the person authentication unit 24, a learning model was generated by performing deep learning using CNN to determine the correspondence between the spectrogram image and the identifier of the person who is the source of the spectrogram image. To perform cross-validation using the leave-one-out method, machine learning was performed on 64 spectrogram images excluding one spectrogram image for validation.

[0020] When a spectrogram image of a subject, such as that shown in FIG. 2, was input into the person authentication unit 24, which includes a machine-learned model, the person's identifier was output. Figure 3 shows the confusion matrix when the subject's authentication was correct or not using leave-one-out cross-validation. The numbers in the outer box indicate the ID numbers of the subjects. The match rate was 0.831 and the Kappa coefficient was 0.817. EXAMPLES

[0021] Fig. 5 is an example according to the second embodiment. The spectrogram image generating unit 22 calculated a spectrogram of a 30-second biovibration signal (top row of Fig. 5) and extracted the 0.7 to 10 Hz components (second row from the top of Fig. 5). The post-processing unit 23 divided the spectrogram image into six segments (third row from the top of Fig. 5) and calculated the ensemble average of these spectrogram images (bottom row of Fig. 5). The person authentication unit 24 performed machine learning of the relationship between the ensemble-averaged spectrogram image and a personal identifier, and performed authentication by leave-one-out cross-validation. Figure 6 shows the confusion matrix when cross-validating whether the subject's authentication was correct or not. The agreement rate was 0.815 and the Kappa coefficient was 0.800. EXAMPLES

[0022] 8 shows an example according to the third embodiment. The AR model coefficient / autocorrelation function analysis unit 26 divided a 30-second BCG signal (top row in the figure) into five segments with a 10-second interval and a 5-second overlap, calculated the AR reflection coefficient (Ref Coef) (second row from the top in the figure), the AR regression coefficient (AR Coef) (third row from the top in the figure), and the autocorrelation function (ACF) (fourth row from the top in the figure) of each segment, and obtained the ensemble average of each (bottom row in the figure). The order of the AR model was set to 80, and the autocorrelation function was calculated after downsampling the BCG signal to 20 Hz, and the lag was set to 4 seconds (1 / 20 Hz x 80) in accordance with the order. Figure 9 shows an example of ensemble averages of the AR reflection coefficient (Ref Coef), AR regression coefficient (AR Coef), and autocorrelation function (ACF) calculated from an arbitrary 30-second BCG signal time series of each BCG data acquisition for one subject, taken five times at different dates and times. The person authentication unit 24 performed machine learning of the relationship between the ensemble-averaged AR reflection coefficient (Ref Coef), AR regression coefficient (AR Coef), autocorrelation function (ACF), and the person identifier, and performed authentication by leave-one-out cross-validation. Figure 10 shows the confusion matrix when cross-validating whether the subject's authentication was correct or not. The match rate was 0.831 and the Kappa coefficient was 0.817. EXAMPLES

[0023] Instead of the piezoelectric sensor 3 of Example 1, an electrocardiograph (not shown) was used, and ECGs were obtained from 13 subjects, five times for each subject, for three minutes each time, and person authentication was performed using the short-time Fourier transform spectrogram of the ECG signal for any 30 seconds. An ECG signal for any 30 seconds of the time series signal was received by the biological vibration signal receiving unit 21 and passed to the spectrogram image generating unit 22, and a spectrogram image was generated in the same manner as in Example 1. FIG. 11 shows spectrogram images in which frequency components from 0.7 to 10 Hz were extracted after short-time Fourier transform (STFT) of the ECG, and examples of images from five of the 13 subjects are shown. Figure 12 shows the confusion matrix obtained by machine learning the relationship between the spectrogram image shown in Figure 11 and the personal identifier, and then performing leave-one-out cross-validation to determine whether the subject's authentication was correct or not. The match rate was 0.815, and the Kappa coefficient was 0.800. EXAMPLES

[0024] FIG. 13 shows an example in which an electrocardiograph (not shown) was used instead of the piezoelectric sensor 3 of Example 2 to obtain ECGs from 13 subjects, five times for each subject, each time lasting three minutes, and the ECG signal (top row of FIG. 13) for any 30 seconds and its short-time Fourier transform spectrogram were calculated. The 0.7 to 10 Hz components of the spectrogram (second row from the top of the figure) were divided into six segments (third row from the top of the figure), and the ensemble average of these spectrograms (bottom row of the figure) was obtained. The person authentication unit 24 performed machine learning of the relationship between the ensemble average image of the spectrogram and the person identifier, and performed authentication by leave-one-out cross-validation. Figure 14 shows the confusion matrix when cross-validating whether the subject's authentication was correct or not. The match rate was 0.908 and the Kappa coefficient was 0.900. EXAMPLES

[0025] FIG. 15 shows an example in which an electrocardiograph (not shown) was used instead of the piezoelectric sensor 3 of Example 3 to obtain ECGs from 13 subjects, five times for each subject, each time lasting three minutes, and non-overlapping ECG signals of any 30-second intervals were extracted from the ECG signals (top row of FIG. 15), and the AR reflection coefficient, AR regression coefficient, and autocorrelation function of the ECG signals were calculated. Thirty seconds of ECG data were divided into five segments with a 10 second interval and a 5 second overlap, and the AR reflection coefficient (Ref Coef) (second row from the top of the figure), AR regression coefficient (AR Coef) (third row from the top of the figure), and autocorrelation function (ACF) (fourth row from the top of the figure) of each segment were calculated, and the ensemble average of each segment (bottom row of the figure) was obtained. FIG. 16 shows an example in which ECG data was acquired five times on different dates and times for one subject, and the ensemble averages of the AR reflection coefficient (Ref Coef), AR regression coefficient (AR Coef), and autocorrelation function (ACF) were calculated from the ECG signal time series for any 30 seconds in each acquisition. The person authentication unit 24 performed machine learning of the relationship between the ensemble averaged AR reflection coefficient (Ref Coef), AR regression coefficient (AR Coef), autocorrelation function (ACF) and the person identifier, and performed authentication by leave-one-out cross-validation. Figure 17 shows the confusion matrix when cross-validating whether the subject's authentication was correct or not. The match rate was 0.923 and the Kappa coefficient was 0.917. EXAMPLES

[0026] Figure 18 is a schematic top view showing an example of two electrodes placed on a toilet seat. When a subject sits on the toilet seat, these electrodes touch the subject's thighs, and an ECG can be obtained by measuring the potential difference between the two electrodes. EXAMPLES

[0027] Fig. 19 is a schematic side view showing an example of electrodes installed on the upper surface of the armrests of a chair. One electrode is installed on each of the upper surfaces of the left and right armrests. An ECG can be obtained by measuring the potential difference between the two electrodes that occurs when the subject's forearm or palm touches the electrodes. EXAMPLES

[0028] Figure 20 is a schematic side view showing an example of electrodes installed on the left and right side of the inside of a bathtub. One electrode is installed on each of the left and right sides of the inside of the bathtub. Since water is a conductor, the subject indirectly touches the electrodes. Therefore, an ECG can be obtained by measuring the potential difference between the two electrodes.

[0029] From the confusion matrices shown in Examples 1 to 3, it was confirmed that person authentication is possible by inputting the spectrogram image of the bio-vibration signal obtained by performing a short-time Fourier transform (STFT) on the bio-vibration signal acquired using a piezoelectric sensor, or the reflection coefficients, regression coefficients, and autocorrelation function of the AR model, into the person authentication unit 24 that has been machine-learned in advance. Moreover, from the confusion matrices shown in Examples 4 to 6, it was possible to confirm that it is possible to authenticate a person using an ECG acquired using an electrocardiograph instead of a piezoelectric sensor. Furthermore, as shown in Figures 18 to 20 of Examples 7 to 9, by placing two electrodes on a toilet seat, an armrest of a chair, or the inside of a bathtub that the subject regularly uses, and measuring the potential between the electrodes, an ECG can be obtained without restraining the subject or making him or her aware of it. [Explanation of symbols]

[0030] One-person authentication system 2 Person Authentication Device 21 Bio-vibration signal receiver 22 Spectrogram image generator 23 Post-processing section 24 Personnel Authentication Department 25 Pretreatment section 26 AR model coefficients and autocorrelation function analysis section 3 Sensor section 51 Toilet seat 52 electrodes 53 Chair 54 Bathtub 55 Water surface

Claims

1. As a person authentication device, A bio-vibration signal receiving unit that receives a human bio-vibration signal; A spectrogram image generating unit that generates a spectrogram image from the biological vibration signal; A person authentication unit that authenticates a person from the spectrogram image, The person authentication unit inputs the person's identifier and the spectrogram image, performs machine learning to determine the relationship between the identifier and the spectrogram image in advance, and inputs the spectrogram image of a subject to output the person's identifier.

2. A human authentication system comprising the human authentication device according to claim 1 and a sensor unit for detecting a human bio-vibration signal.

3. receiving a bio-vibration signal of a person; generating a spectrogram image from the biological vibration signal; A person authentication unit that authenticates a person from the spectrogram image, the person authentication unit inputs an identifier of the person and the spectrogram image, and performs machine learning in advance to determine a relationship between the identifier and the spectrogram image; A method for person authentication comprising the steps of inputting the spectrogram image of a subject and outputting an identifier for that person.

4. Computer, A bio-vibration signal receiving means for receiving a bio-vibration signal of a person; A means for extracting a useful signal from the biovibration signal; a spectrogram image generating means for generating a spectrogram image from said useful signal; A person authentication unit that authenticates a person from the spectrogram image, the person authentication unit is a means for inputting an identifier of the person and the spectrogram image and performing machine learning in advance on a relationship between the identifier and the spectrogram image; means for inputting said spectrogram image of a subject and outputting an identifier for that person; A human authentication program characterized by functioning as a

5. As a person authentication device, A bio-vibration signal receiving unit that receives a human bio-vibration signal; an AR model coefficient / autocorrelation function analysis unit that calculates an AR model reflection coefficient, a regression coefficient, and an autocorrelation function from the useful signal; a person authentication unit that authenticates a person based on the AR model reflection coefficients, regression coefficients, and an autocorrelation function; The person authentication unit inputs the person's identifier, the AR model reflection coefficients, the regression coefficients, and the autocorrelation function to machine-learn the relationship between the identifier, the AR model reflection coefficients, the regression coefficients, and the autocorrelation function in advance, and inputs the AR model reflection coefficients, the regression coefficients, and the autocorrelation function of a subject to output the person's identifier.

6. A human authentication system comprising the human authentication device according to claim 5 and a sensor unit for detecting a human biovibration signal.

7. receiving a bio-vibration signal of a person; Calculating an AR model reflection coefficient, a regression coefficient, and an autocorrelation function from the biological vibration signal; A person authentication unit that authenticates a person by calculating the AR model reflection coefficient, a regression coefficient, and an autocorrelation function, the person authentication unit inputs an identifier of the person, an AR model reflection coefficient, a regression coefficient, and an autocorrelation function, and performs machine learning in advance on a relationship between the identifier, the AR model reflection coefficient, the regression coefficient, and the autocorrelation function; A method for authenticating a person, comprising the step of inputting the AR model reflection coefficients, regression coefficients and autocorrelation function of a subject and outputting an identifier for that person.

8. Computer, A bio-vibration signal receiving means for receiving a bio-vibration signal of a person; A means for calculating an AR model reflection coefficient, a regression coefficient, and an autocorrelation function from the biological vibration signal; a person authentication unit that authenticates a person based on the AR model reflection coefficients, regression coefficients, and an autocorrelation function; The person authentication unit is a means for inputting an identifier of the person, the AR model reflection coefficients, the regression coefficients, and the autocorrelation function, and machine learning the relationship between the identifier, the AR model reflection coefficients, the regression coefficients, and the autocorrelation function in advance, and inputting the AR model reflection coefficients, the regression coefficients, and the autocorrelation function of a subject to output the identifier of the person. A human authentication program characterized by functioning as a

Images