Ultrasonic fingerprint identification method and system

Abstract

The invention provides an ultrasonic fingerprint identification method and system. The method comprises the following steps: acquiring acceleration time sequence data, ultrasonic waveform data and a fingerprint generation image sequence under a time step, and respectively preprocessing; performing feature extraction on the preprocessed acceleration time sequence data, ultrasonic waveform data and fingerprint generation image sequence, and performing multi-modal fusion to obtain a global feature distribution matrix; and inputting the global feature distribution matrix into a classification and identification network for fingerprint identification and classification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of terminal equipment, and in particular to an ultrasonic fingerprint identification method and system. BACKGROUND

[0002] Fingerprint identification is a biometric technology that analyzes the characteristics of human fingerprints, such as lines, patterns, and details, and converts them into mathematical models to identify identity. Fingerprint identification is the most mature and widely used biometric technology. In recent years, with the continuous innovation and improvement of technology, the application scenarios of fingerprint identification technology have been continuously expanded, and the recognition rate and speed have been greatly improved. Currently, fingerprint identification has become one of the default unlocking methods for smartphones and other mobile devices, and is widely used in identity authentication, payment, and data security fields.

[0003] Among the common fingerprint identification solutions on smartphones, ultrasonic fingerprint is favored by mainstream manufacturers due to its advantages of being unaffected by light, dirt, scratches, etc. Ultrasonic fingerprint identification technology uses the penetration and reflection principle of ultrasonic waves to obtain fingerprint information on the surface of the finger, thereby realizing fingerprint identification. In this process, the ultrasonic sensor emits ultrasonic signals, which penetrate the skin of the finger, are reflected and scattered, and are then received by the ultrasonic sensor. Then, using signal processing technology, the signals received by the sensor are analyzed, and the fingerprint lines and wrinkles on the surface of the finger are extracted for comparison to determine whether they match the specified fingerprint.

[0004] However, when a tempered film, protective film or other additional material is covered on most current smartphone screens, the performance of traditional optical or capacitive fingerprint identification technology significantly decreases, the recognition rate decreases, and the misjudgment rate increases. The reason is that the additional layer may cause signal attenuation, scattering or distortion, making the extracted fingerprint features not clear enough.

[0005] Although ultrasonic signals can penetrate the screen and additional material to collect the three-dimensional structure of the fingerprint, in complex environments, the noise influence of ultrasonic signals and the dynamic changes of the screen may cause instability of the collected signals. In addition, the characteristics of finger movement during the pressing process, such as acceleration and force, may also affect the collection effect of the fingerprint image. The existing ultrasonic fingerprint identification technology still has significant technical bottlenecks in practical application: Current fingerprint recognition technology mainly relies on the extraction and analysis of a single biological feature (such as fingerprint ridges or feature points), which has the limitations of high image quality sensitivity and easy loss of local features. When the screen is covered with a tempered film with a thickness of more than 0.3 mm or is in a high-frequency electromagnetic noise environment, the signal-to-noise ratio of the echo signal will drop sharply, resulting in blurred fingerprint images and loss of detailed features, which seriously affects the fingerprint recognition result. More seriously, the existing technology has a serious lack of sensing ability for dynamic pressing behavior, which cannot capture the acceleration change characteristics (such as sliding trajectory and tremor frequency) in the process of finger pressing, and also cannot distinguish the elastic response of real biological tissue from the mechanical properties of static fake materials such as silica gel film, forming a major safety hazard. In addition, although some schemes try to introduce multi-sensor data fusion, there are generally problems of time sequence misalignment and feature fragmentation, and only through simple weighted processing to superimpose data, multi-modal deep collaborative enhancement cannot be achieved. SUMMARY

[0006] The purpose of the present application is to overcome the shortcomings of the prior art, and to provide an ultrasonic fingerprint recognition method and system to improve the accuracy of recognizing fingerprint information in the case of a mobile phone screen covered with a tempered film and other additional layers.

[0007] The present application is implemented as follows. In a first aspect, the present application provides an ultrasonic fingerprint recognition method, the method comprising: Step S1, obtaining a time step The acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence are preprocessed respectively; Step S2, extracting features from the preprocessed acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence, and obtaining a global feature distribution matrix through multi-modal fusion; Step S3, inputting the global feature distribution matrix into a classification recognition network for fingerprint recognition classification.

[0008] Preferably, step S2 is specifically: Extracting one-dimensional features from the preprocessed acceleration time sequence data and ultrasonic waveform data to obtain an ultrasonic feature vector in the i-th time step , an acceleration feature vector ; Extracting features from the preprocessed fingerprint generation image sequence using a convolutional neural network (CNN) to obtain an image texture feature in the i-th time step ; then using one-dimensional expansion to obtain an image feature vector ; The ultrasonic feature vector , the acceleration feature vector , and the image feature vector The splicing is performed to obtain the i-th time step The comprehensive feature vector in the inner part Finally, the global feature distribution matrix is obtained .

[0009] Preferably, the classification recognition network comprises a transformer network and a discriminator. The transformer network is used for performing self-attention mechanism on the global feature distribution matrix to obtain a cross-modal time sequence dependency relationship as a real-time fingerprint feature. The discriminator adopts a metric learning strategy to perform similarity matching between the real-time fingerprint feature and a cross-modal time sequence dependency relationship feature in a pre-stored fingerprint template library to obtain a fingerprint matching result.

[0010] In a second aspect, the present application provides an ultrasonic fingerprint identification system, comprising: A data acquisition module is responsible for acquiring time steps The acceleration time sequence data, the ultrasonic waveform data and the fingerprint generation image sequence are preprocessed respectively. A feature extraction module is responsible for extracting features from the preprocessed acceleration time sequence data, the ultrasonic waveform data and the fingerprint generation image sequence, and obtaining a global feature distribution matrix through multi-modal fusion. An ultrasonic fingerprint identification module is responsible for inputting the global feature distribution matrix to a classification recognition network for fingerprint identification and classification.

[0011] The present application deeply fuses the above-mentioned extracted dynamic acceleration behavior features, ultrasonic waveform data (reflecting acoustic impedance changes and possibly containing micro-vibration information) and fingerprint image sequences in time sequence, which can effectively distinguish the biomechanical characteristics (such as elastic response of tissues and natural tremor) of real living body fingers from the mechanical characteristics of static fake materials such as silica gel film. The introduction of dynamic behavior features and cross-modal time sequence modeling greatly improve the resistance ability (security) of the system to fake attacks, which can automatically learn and model the complex dependency relationships between cross-modal (image, sound wave, motion) and between time steps. This multi-dimensional information that fuses static and dynamic, surface and subsurface, image and signal fully excavates the internal correlation of multi-source information, has more abundant biological identification information and stronger individual distinguishing ability (high discriminability) than traditional single fingerprint image or single modal feature.

[0012] The present application builds a multi-dimensional perception system through hardware-algorithm collaborative innovation, realizes a systematic breakthrough of under-screen fingerprint identification technology, and systematically solves the severe challenges faced by existing ultrasonic fingerprint identification technology in terms of screen additional layer penetration, dynamic behavior perception, anti-fake ability and effective fusion of multi-modal data. Finally, the fingerprint identification accuracy, robustness (stability) and security (anti-fake ability) are significantly improved in complex environments (such as covered with tempered film, with noise interference). The present application provides an effective technical solution for under-screen fingerprint identification, especially high-precision and high-security identity authentication in harsh application scenarios such as with protective film or thick screen. The present application can be widely applied in the field of under-screen fingerprint identification, and is especially suitable for high-precision fingerprint identification in the case of tempered film or thick screen. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described in the following are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0014] Figure 1 is the flow chart of the ultrasonic fingerprint identification method provided by the embodiment of the present application. DETAILED DESCRIPTION

[0015] The technical solutions in the embodiments of the present application will be described clearly and completely in combination with the drawings in the embodiments of the present application.

[0016] As shown in Figure 1 The present embodiment provides an ultrasonic fingerprint identification method, which is a multi-modal ultrasonic fingerprint identification method for improving the accuracy of fingerprint identification in the case of external factors such as tempered film and fingerprint film on the mobile phone. The method specifically includes the following steps: Step S1, data acquisition: S1-1 obtains time step downward acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence by using the ultrasonic fingerprint identification module. The ultrasonic fingerprint identification module is a sensor combination placed under the screen, including: an acceleration sensor, an array type sensor capable of emitting and receiving ultrasonic waves, a signal processing module for generating a fingerprint image according to the received ultrasonic waves; the signal processing module is connected with the array type sensor signal.

[0017] One embodiment provides that the acceleration time series data is captured in real time by an acceleration sensor integrated under the screen of the electronic device at a sampling frequency of no less than 1000 frames per second, and the acceleration sensor outputs dynamic signals containing timestamps and x, y, z axis acceleration values for quantifying the force of finger pressing, sliding trajectory and contact stability.

[0018] The embodiment constructs a dynamic behavior perception model based on a high-precision acceleration sensor (≥1000 Hz sampling rate) to quantize the pressing force, contact stability and motion trajectory features in real time.

[0019] One embodiment provides that the ultrasonic waveform data is collected by an array ultrasonic sensor, which is composed of multiple groups of ultrasonic transmitting and receiving units and embedded in the area under the screen, generates continuous ultrasonic waveform data by transmitting high-frequency ultrasonic pulses and receiving reflected echo signals, and the waveform amplitude reflects the acoustic impedance difference between fingerprint ridge lines and valley lines.

[0020] One embodiment provides that the process of obtaining the sequence of fingerprint generated images is: The ultrasonic waveform data received by the array ultrasonic sensor is processed by a signal processing module for signal enhancement, noise suppression and image reconstruction, and a dynamically updated sequence of fingerprint grayscale images is output; wherein the grayscale value range of each image is 0-255, the generation frequency is synchronized with the ultrasonic sampling rate, and the image real-time performance is ensured.

[0021] The application collects high-frequency ultrasonic waveform data by an array ultrasonic sensor, and processes the data by a signal processing module for signal enhancement, noise suppression and image reconstruction, which significantly improves the signal-to-noise ratio (SNR) of the echo signal after penetrating the additional layer, effectively alleviates the problems of signal attenuation, scattering and distortion, and thus generates a clearer and more detailed sequence of fingerprint grayscale images in the image reconstruction stage, thereby improving the recognition rate problem caused by image blur and detail loss under the cover layer in the traditional scheme.

[0022] The acceleration sensor, array ultrasonic sensor and signal processing module are synchronized by a hardware-level clock to ensure that the time alignment error of multi-modal data is less than 1 millisecond. Strict time alignment ensures that the data captured by different sensors (acceleration, ultrasonic transmission / reception, image generation) in the same time step strictly corresponds to the same state or action of the finger, eliminating the problem of time sequence misalignment caused by different sensor response delays.

[0023] S1-2 pre-processes the acceleration time series data, ultrasonic waveform data and sequence of fingerprint generated images respectively; specifically: The ultrasound waveform data, acceleration time series data, and fingerprint generation image sequence are time-stamped and synchronized, and the ultrasound waveform data, acceleration time series data, and fingerprint generation image sequence are normalized to eliminate dimensional differences. Specifically, the ultrasound signal intensity is mapped to the interval [0, 1], the acceleration amplitude is standardized by z-score, and the image pixel value is histogram equalized.

[0024] In step S2, the preprocessed acceleration time series data, ultrasound waveform data, and fingerprint generation image sequence are feature extracted, and a global feature distribution matrix is obtained through multi-modal fusion. Specifically: One-dimensional features are extracted from the preprocessed acceleration time series data and ultrasound waveform data to obtain an ultrasound feature vector and an acceleration feature vector in the ith time step. ; The feature extraction attributes of the ultrasound waveform are time domain features and frequency domain features. The time domain features include mean, standard deviation, skewness, kurtosis, and signal strength. The frequency domain features include bandwidth, power spectral density, and spectral entropy. In one embodiment, the ultrasound waveform data is analyzed based on a sliding window to extract mean, variance, kurtosis, and zero-crossing rate. Fast Fourier Transform (FFT) is used to calculate frequency domain power spectral density, main frequency band energy ratio, and spectral entropy as one-dimensional ultrasound features. The feature extraction attributes of the acceleration time series data can include maximum value, minimum value, average value, peak value, rectified average value, variance, standard deviation, kurtosis, skewness, root mean square, waveform factor, peak factor, pulse factor, margin factor, center frequency, mean square frequency, root mean square frequency, frequency variance, frequency standard deviation, spectral kurtosis mean, spectral kurtosis standard deviation, spectral kurtosis skewness, and spectral kurtosis kurtosis. In one embodiment, the acceleration time series data is analyzed using a time-frequency joint analysis method to extract the peak value, root mean square, waveform factor, and wavelet packet decomposition energy entropy of the acceleration signal, and the frequency domain center frequency and frequency band variance are calculated as one-dimensional acceleration features. The preprocessed fingerprint generation image sequence is extracted using a convolutional neural network (CNN) to obtain an image texture feature matrix in the ith time step , which contains ridge line direction, bifurcation point density, and local contrast information. Then, the image texture feature matrix is expanded in one dimension to obtain an image feature vector . The ultrasound feature vector , the acceleration feature vector , and the image feature vector are concatenated to obtain a comprehensive feature vector in the ith time step. , and finally obtain a global feature distribution matrix according to the time steps .

[0025] In step S3, the global feature distribution matrix is input into a classification recognition network for fingerprint recognition and classification.

[0026] The classification recognition network comprises a transformer network and a discriminator. The transformer network is used to obtain cross-modal time sequence dependency relationship through self-attention mechanism, as real-time fingerprint features.

[0027] The discriminator adopts a metric learning strategy to perform similarity matching between the real-time fingerprint features and cross-modal time sequence dependency relationship features in a pre-stored fingerprint template library, and obtain a fingerprint matching result.

[0028] The Euclidean distance between the real-time fingerprint features and the cross-modal time sequence dependency relationship features in the pre-stored fingerprint template library is calculated to obtain a similarity; if the similarity falls within a threshold range, it is considered that the fingerprint matching is successful, otherwise the fingerprint matching is failed.

[0029] The embodiment also provides an ultrasonic fingerprint recognition system, comprising: A data acquisition module is responsible for acquiring time steps downward acceleration time sequence data, ultrasonic waveform data, and a fingerprint generation image sequence, and respectively performing preprocessing; A feature extraction module is responsible for feature extraction on the preprocessed acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence, and obtaining a global feature distribution matrix through multi-modal fusion. An ultrasonic fingerprint recognition module is responsible for inputting the global feature distribution matrix into a classification recognition network for fingerprint recognition and classification.

[0030] The above describes the preferred embodiments of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, several improvements and refinements can be made, which are also considered within the protection scope of the present application.

Claims

1. An ultrasonic fingerprint identification method, characterized by, The method comprises: Acquisition time step The lower acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence are preprocessed respectively. The pre-processed acceleration time series data, ultrasonic waveform data, and fingerprint generation image sequence are subjected to feature extraction, and a global feature distribution matrix is obtained through multi-modal fusion; The global feature distribution matrix is input into a classification recognition network for fingerprint recognition classification.

2. The method of claim 1, wherein, The preprocessing specifically comprises: time series alignment of the ultrasonic waveform data, acceleration time series data, and fingerprint generation image sequence through a timestamp synchronization mechanism, and normalization processing of the ultrasonic waveform data, acceleration time series data, and fingerprint generation image sequence.

3. The method of claim 1, wherein, The acceleration time series data is collected by an acceleration sensor integrated below the screen of an electronic device at a sampling frequency of no less than 1000 frames / s in real time, and a dynamic signal containing a timestamp and x, y, and z axis acceleration values is output, which is used to quantify the force, sliding track, and contact stability of a finger press.

4. The method of claim 1, wherein, The ultrasonic waveform data is collected by an array type ultrasonic sensor, which is composed of multiple groups of ultrasonic emission and receiving units and is embedded in the area below the screen, and through emission of high-frequency ultrasonic pulses and reception of reflected echo signals, continuous ultrasonic waveform data is generated.

5. The method of claim 1, wherein, The acquisition process of the fingerprint generation image sequence is as follows: The ultrasonic waveform data received by the array type ultrasonic sensor is subjected to signal enhancement, noise suppression, and image reconstruction processing by a signal processing module, and a fingerprint grayscale image sequence is output; wherein the grayscale value range of each image is 0-255, and the generation frequency is synchronized with the ultrasonic sampling rate.

6. The method of claim 1, wherein, Step S2 specifically comprises: The one-dimensional features are extracted from the preprocessed acceleration time series data and ultrasonic waveform data to obtain an i-th time step ultrasonic feature vector in the inner acceleration feature vector ; The image sequence of the pretreated fingerprint is extracted by a convolutional neural network (CNN) to obtain image texture features at the i-th time step within the image Then the image texture features are obtained by one-dimensional expansion ; ultrasonic feature vector , acceleration feature vector , image feature vector , and the comprehensive feature vector in the i-th time step is obtained by splicing , and finally the global feature distribution matrix is obtained.

7. The method of claim 6, wherein, The image texture features include ridge line direction, bifurcation point density, and local contrast information.

8. The method of claim 1, wherein, The classification recognition network comprises a transformer network and a discriminator; The transformer network is used to obtain cross-modal time series dependency relationships through a self-attention mechanism on the global feature distribution matrix, as real-time fingerprint features; The discriminator adopts a metric learning strategy to perform similarity matching between the real-time fingerprint features and cross-modal time series dependency relationship features in a pre-stored fingerprint template library, and obtains a fingerprint matching result.

9. The method of claim 8, wherein, The similarity matching specifically comprises calculating the Euclidean distance between the real-time fingerprint features and the cross-modal time series dependency relationship features in the pre-stored fingerprint template library to obtain a similarity; if the similarity falls within a threshold range, it is considered that the fingerprint matching is successful, otherwise the fingerprint matching fails.

10. An ultrasonic fingerprint sensing system implementing the method of any of claims 1-9, characterized by It comprises: A data acquisition module is responsible for acquiring time steps The lower acceleration time sequence data, ultrasonic waveform data, and fingerprint generation image sequence are preprocessed respectively. A feature extraction module is responsible for feature extraction on the pre-processed acceleration time series data, ultrasonic waveform data, and fingerprint generation image sequence, and a global feature distribution matrix is obtained through multi-modal fusion; An ultrasonic fingerprint recognition module is responsible for inputting the global feature distribution matrix into a classification recognition network for fingerprint recognition classification.

Images