Fingerprint identification method and electronic device and medium thereof

Abstract

The present application relates to the field of intelligent terminal technology, and in particular to a fingerprint recognition method, medium and electronic device. The method of the present application includes obtaining the capacitance information of the hand to which the finger used by the user when performing fingerprint recognition on the electronic device is collected by the capacitance sensor of the electronic device, wherein the capacitance information includes at least one of the capacitance fingerprint image of the finger and the capacitance image of the base of the hand to which the finger belongs; and performing fingerprint recognition based on the obtained capacitance information. The fingerprint recognition method of the present application can collect the capacitance information of the user's fingers and the hand to which the fingers belong through the capacitance sensor, and collect the optical fingerprint image of the user's fingers through the optical fingerprint sensor, and then combine the capacitance information and the optical fingerprint image to perform fingerprint recognition authentication, thereby improving the efficiency of terminal fingerprint recognition.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent terminals, and in particular to a fingerprint recognition method, medium and electronic device. Background Art

[0002] As mobile phone screens tend to be full-screen, under-screen fingerprint recognition as a new biometric solution has gradually become a hot technology for major mobile phone manufacturers to study. Under-screen fingerprint recognition is performed by collecting the user's fingerprint image for recognition. The better the quality of the collected image, the clearer the fingerprint pattern in the image, and the higher the corresponding recognition rate. However, the fingerprint matching method is relatively simple, relying only on the collected fingerprint image for matching, and because the user's finger conditions are different each time the collection is performed, such as the dryness or wetness of the fingers, the quality of the collected fingerprint image is different, so the matching success rate will also be affected. In addition, during the fingerprint matching process, the newly collected fingerprints will be matched with all the fingerprints already recorded in the mobile phone in turn. Since the number of fingerprint templates already recorded in the mobile phone is large, the matching process is not only time-consuming but also invalid matches may occur. Summary of the invention

[0003] The present application relates to the field of intelligent terminal technology, and in particular to a fingerprint recognition method, medium and electronic device. The fingerprint recognition method of the present application can collect the capacitance information of the user's fingers and the hand to which the fingers belong through a capacitive sensor, and collect the optical fingerprint image of the user's fingers through an optical fingerprint sensor, and then combine the capacitance information and the optical fingerprint image to perform fingerprint recognition authentication, thereby improving the efficiency of terminal fingerprint recognition. The following is a detailed introduction to the content of the present application.

[0004] In a first aspect, an embodiment of the present application provides a fingerprint recognition method, including:

[0005] The capacitance information of the hand to which the finger used by the user for fingerprint recognition on the electronic device belongs is obtained through the capacitance sensor of the electronic device, wherein the capacitance information includes at least one of the capacitance fingerprint image of the finger and the capacitance image of the base of the hand to which the finger belongs; fingerprint recognition is performed based on the obtained capacitance information. That is, the capacitance fingerprint image of the finger is used as auxiliary information to perform fingerprint recognition; or the base of the hand to which the finger belongs is first used to determine whether the finger used is the left hand or the right hand, and then fingerprint recognition is performed; or the base of the hand to which the finger belongs and the capacitance fingerprint image of the finger are used to jointly determine whether the finger used belongs to the left hand or the right hand, and then fingerprint recognition is performed.

[0006] In a possible implementation of the first aspect, the performing fingerprint recognition based on the acquired capacitance information further includes:

[0007] The hand to which the finger used by the user belongs is determined to be the user's left hand or right hand according to the capacity value image of the tiger's mouth, or the hand to which the finger used by the user belongs to is determined to be the user's left hand or right hand according to the capacity value image of the tiger's mouth and the capacity value fingerprint image; the fingerprint of the finger used by the user is identified according to the determination result.

[0008] In a possible implementation of the first aspect, the identifying the fingerprint of the finger used by the user according to the judgment result includes:

[0009] In the case where it is determined that the hand to which the finger used by the user belongs is the user's left hand or right hand, the fingerprint feature information of the finger used by the user is matched with the stored fingerprint feature information of each finger of the determined left hand or right hand, and if the match is successful, it is determined that the fingerprint recognition has passed; in the case where it is not determined that the hand to which the finger used by the user belongs is the user's left hand or right hand, the fingerprint feature information of the finger used by the user is matched with the stored fingerprint feature information of each finger of the left hand and right hand, and if the match is successful, it is determined that the fingerprint recognition has passed. That is, after it is determined that the hand to which the finger used by the user belongs is the user's left hand or right hand, the fingerprint feature information of the finger used by the user is matched with the fingerprint feature information of each finger of the corresponding left hand or right hand stored in the electronic device. If the fingerprint feature information of the finger used by the user matches the fingerprint feature information of a finger of the corresponding left hand or right hand stored in the electronic device, it is considered that the fingerprint match is successful, and the fingerprint recognition has passed. If it is impossible to determine whether the finger used by the user belongs to the left hand or the right hand, the fingerprint feature information of the finger used by the user is matched with the fingerprint feature information of all fingers stored in the electronic device, and then if the fingerprint match is successful, it is determined that the fingerprint recognition is successful.

[0010] In a possible implementation of the first aspect, the fingerprint feature information of the finger includes:

[0011] Fingerprint features of the optical fingerprint image of the finger, wherein the optical fingerprint image is obtained by collecting the user's finger through the optical fingerprint sensor of the electronic device. That is, the optical fingerprint image of the user's finger is collected by the optical fingerprint sensor of the electronic device.

[0012] In a possible implementation of the first aspect, the fingerprint feature information of the finger includes:

[0013] Fingerprint features of the capacitance fingerprint image of the finger. That is, the fingerprint feature information of the finger includes the fingerprint features of the optical fingerprint image of the finger and / or the fingerprint features of the capacitance fingerprint image of the finger. In some embodiments of the present application, when performing fingerprint matching, at least one of the fingerprint features of the optical fingerprint image or the fingerprint features of the capacitance fingerprint image of the finger in the fingerprint feature information of the finger can be used.

[0014] In a possible implementation of the first aspect, the method further includes:

[0015] When matching the fingerprint feature information of the finger used by the user with the fingerprint feature information of each stored finger, the order of the fingerprint feature information of each finger during matching is determined according to the frequency of the fingerprint feature information of each finger, wherein the greater the frequency of the fingerprint feature information of the finger, the higher the order during matching; wherein, the greater the frequency of the fingerprint feature information of the finger, the more times the fingerprint feature information of the finger has been matched in the historical fingerprint recognition process. That is, the fingerprint feature information is stored in the fingerprint information library, and when performing fingerprint matching, the fingerprint feature information of each finger in the fingerprint information library is arranged in descending order according to the frequency. The greater the frequency, the more times the fingerprint feature information of the finger has been successfully matched in the previous fingerprint matching, and it will be matched first in the current fingerprint matching, thereby improving the efficiency of fingerprint matching.

[0016] In a possible implementation of the first aspect, the performing fingerprint recognition based on the acquired capacitance information includes:

[0017] The fingerprint of the user's finger is identified based on the acquired capacitance fingerprint image and optical fingerprint image, wherein the optical fingerprint image is obtained by collecting the user's finger through the optical fingerprint sensor of the electronic device. That is, fingerprint recognition and authentication are performed by combining the fingerprint features in the capacitance fingerprint image and the optical fingerprint image, thereby improving the accuracy of fingerprint recognition.

[0018] In a possible implementation of the first aspect, the identifying the fingerprint of the finger used by the user based on the acquired capacitance fingerprint image and the optical fingerprint image includes:

[0019] Feature extraction is performed on the capacitance fingerprint image and the optical fingerprint image respectively to obtain fingerprint feature information of the finger used by the user; the fingerprint feature information of the finger used by the user is matched with the fingerprint feature information of each finger that has been stored, and the fingerprint recognition is determined to be successful if the match is successful.

[0020] In a second aspect, the present application provides an electronic device, including:

[0021] A capacitive sensor, used to collect capacitance information of the hand to which the finger used by the user belongs when performing fingerprint recognition on the electronic device, wherein the capacitance information includes at least one of a capacitance fingerprint image of the finger and a capacitance image of the base of the hand to which the finger belongs;

[0022] An optical fingerprint sensor is used to collect an optical fingerprint image of the finger used by the user for fingerprint recognition on an electronic device;

[0023] a memory storing instructions; and

[0024] At least one processor is configured to access the memory and execute instructions in the memory to control the capacitive sensor and the optical fingerprint sensor to perform the fingerprint recognition method of the first aspect.

[0025] In a third aspect, an embodiment of the present application provides a readable medium of an electronic device, on which instructions are stored, and when the instructions are executed on the electronic device, the electronic device executes any of the methods described above.

[0026] In a fourth aspect, an embodiment of the present application provides an electronic device having the function of implementing the above-mentioned search method. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more units corresponding to the above-mentioned function. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 According to some embodiments of the present application, a schematic diagram of the hardware structure of a mobile phone 100 capable of realizing the functions of the mobile phone 100 is shown;

[0028] Figure 2 A schematic diagram of the relative positions of a mobile phone touch screen, a capacitive sensor, and an optical fingerprint sensor in the prior art;

[0029] Figure 3 This is a circuit diagram of the under-screen fingerprint recognition;

[0030] Figure 4 According to some embodiments of the present application, a schematic flow chart of a fingerprint recognition method is shown;

[0031] Figure 5 According to some embodiments of the present application, a schematic flow chart of a fingerprint recognition method is shown;

[0032] Figure 6 According to some embodiments of the present application, a schematic flow chart of a fingerprint recognition method is shown;

[0033] Figure 7 According to some embodiments of the present application, a schematic flow chart of a fingerprint recognition method is shown;

[0034] FIG8( a ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0035] FIG8( b ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0036] FIG8( c ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0037] FIG8( d ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0038] FIG8( e ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0039] FIG8( f ) shows a diagram of a human interaction interface according to some embodiments of the present application;

[0040] Fig. 9 According to some embodiments of the present application, a schematic diagram of a user operating a mobile phone 100 is shown;

[0041] FIG10( a ) shows a schematic diagram of an optical fingerprint image of a right hand finger in a fingerprint information database according to some embodiments of the present application;

[0042] FIG10( b ) shows a schematic diagram of an optical fingerprint image of a right hand finger in a fingerprint information database according to some embodiments of the present application;

[0043] FIG11( a ) is a schematic diagram showing a user performing fingerprint recognition using the index finger of his right hand according to some embodiments of the present application;

[0044] FIG11( b ) is a schematic diagram showing a user performing fingerprint recognition using the middle finger of his right hand according to some embodiments of the present application;

[0045] FIG12( a ) is a schematic diagram showing a user performing fingerprint recognition using the index finger of his right hand according to some embodiments of the present application;

[0046] FIG. 12( b ) shows a schematic diagram of user fingerprint feature information stored in the mobile phone 100 according to some embodiments of the present application;

[0047] FIG12( c ) is a schematic diagram showing a user performing fingerprint recognition using the index finger of his right hand according to some embodiments of the present application;

[0048] Fig.13 According to some embodiments of the present application, a schematic diagram of the software structure of the mobile phone 100 capable of implementing the mobile phone 100 is shown. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0050] Below, some terms used in the embodiments of the present application are explained to facilitate understanding by those skilled in the art.

[0051] In an embodiment of the present application, for the fingerprint recognition service of the electronic device, the recognition efficiency and accuracy are improved by introducing the capacitance information collected by the capacitance sensor in the touch screen of the electronic device. Among them, the capacitance information includes at least one of the capacitance fingerprint image of the finger used by the user when performing fingerprint recognition and the capacitance image of the base of the hand to which the finger belongs. Specifically, in some embodiments, the electronic device first determines whether the hand used for fingerprint recognition authentication by the user is the left hand or the right hand through at least one of the above capacitance information. After judging the left and right hands, the fingerprint feature information of the optical fingerprint image and the capacitance fingerprint image (or only the optical fingerprint image) of the finger pressing the screen of the user is matched with the fingerprint feature information of each finger of the judged left hand or right hand respectively. When the match is successful, the fingerprint recognition is passed and the fingerprint recognition service is completed. In some other embodiments, the electronic device does not first determine whether the hand used for fingerprint recognition authentication by the user is the left hand or the right hand, but matches the fingerprint feature information of the optical fingerprint image and the capacitance fingerprint image of the finger pressing the screen of the user with the fingerprint feature information of each finger of the two hands of the user respectively, and when the match is successful, the fingerprint recognition is passed and the fingerprint recognition service is completed. The technical solution of the present application introduces the user's capacitance information for fingerprint recognition, which improves the accuracy of fingerprint recognition, and pre-identifies the left and right hands, reducing the number of times the fingerprint template needs to be matched, thereby improving the fingerprint recognition rate.

[0052] It can be understood that the technical solution of the present application is applicable to various electronic devices with touch screens and fingerprint recognition services, including but not limited to mobile phones, tablet computers, smart printers, desktop computers, laptop computers, wearable devices (such as smart watches, bracelets, etc., or as part of smart watches, bracelets, etc.), etc. Exemplary embodiments of electronic devices include but are not limited to devices equipped with Or a portable terminal with other operating systems. The electronic device may also be other portable terminals, as long as it can collect the user's optical fingerprint information and capacitance value information.

[0053] It can be understood that the fingerprint recognition service of the present application can be various services on electronic devices that require fingerprint authentication, for example, it can include but is not limited to fingerprint unlocking, fingerprint payment authentication, fingerprint authentication application lock, fingerprint switching privacy space, etc.

[0054] For ease of explanation, the following takes the under-screen fingerprint recognition service on the mobile phone 100 as an example to illustrate the technical solution of the present application.

[0055] Reference Figure 1 , Figure 11 shows a schematic diagram of the structure of a mobile phone 100. The mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) connector 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0056] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0057] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0058] The processor can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. In some practical examples of the present application, the processor can be configured to execute relevant instructions to implement the fingerprint recognition technology of the present application.

[0059] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory.

[0060] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, and a subscriber identity module (SIM) interface.

[0061] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 140 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the mobile phone 100.

[0062] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0063] The USB connector 130 is a connector that complies with USB standard specifications and can be used to connect the mobile phone 100 and peripheral devices. Specifically, it can be a standard USB connector (such as a Type C connector), a Mini USB connector, a Micro USB connector, etc.

[0064] The charging management module 140 is used to receive charging input from a charger. While the charging management module 140 is charging the battery 142 , it can also power the electronic device through the power management module 141 .

[0065] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 424, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 444 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc.

[0066] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0067] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0068] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the mobile phone 100 .

[0069] The wireless communication module 160 can provide wireless communication solutions for application in the mobile phone 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0070] The mobile phone 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 140 may include one or more GPUs, which execute program instructions to generate or change display information.

[0071] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0072] In some embodiments of the present application, the display screen 194 is used to display the image interface and the user's commonly used applications (applications, APPs), and when the user performs a corresponding touch operation on the touch screen, a corresponding prompt message is displayed to prompt the user to perform the next operation. For example, when the user performs an under-screen fingerprint unlock, the fingerprint recognition area is displayed on the display screen 194, and a prompt message "Please verify the fingerprint in this area to unlock the screen" is displayed to the user.

[0073] The mobile phone 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0074] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone 100. The external memory card communicates with the processor 140 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0075] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone 100 by running instructions stored in the internal memory 424 and / or instructions stored in a memory provided in the processor. For example, the under-screen fingerprint recognition service of the present application is executed.

[0076] The mobile phone 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0077] The fingerprint sensor 180H is used to collect optical fingerprint images, capacitive fingerprint images, and thumb-mouth capacitive images. The mobile phone 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. It can be understood that in some embodiments, the fingerprint sensor 180H of the present application can include a capacitive sensor 102, an optical fingerprint sensor 103, etc. under the display screen 194.

[0078] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 491. In other embodiments, the touch sensor 180K can also be set on the surface of the mobile phone 100, which is different from the position of the display screen 194.

[0079] In some embodiments of the present application, when the mobile phone 100 detects a touch event of a user's finger in the fingerprint verification area (the touch event may specifically be the user's finger tapping the touch screen, etc.), the mobile phone 100 collects the capacitance information of the current user through the fingerprint sensor in the fingerprint verification area, and verifies the user's fingerprint based on the capacitance information (for example, matching it with a fingerprint pre-stored in the mobile phone 100); if the fingerprint match is successful, it responds to the event corresponding to the touch event (for example, unlocking the screen, etc.).

[0080] Generally, an electronic device can support multiple applications. For example, one or more of the following applications: camera application, instant messaging application, etc. Among them, there can be multiple instant messaging applications, such as WeChat, Tencent chat software (QQ), WhatsApp Messenger, Line, Kakao Talk, DingTalk, etc. Through the instant messaging application, users can send information such as text, voice, pictures, video files, and various other files to other contacts; or users can make voice and video calls with other contacts through the instant messaging application. The application designed below can be a system application that comes with the electronic device when it leaves the factory, or it can be a third-party application that the electronic device downloads and installs from the network side, or it can be an application that the electronic device receives from other electronic devices, and the embodiments of this application are not limited thereto.

[0081] The following takes the example of unlocking the mobile phone 100 by the under-screen fingerprint recognition technology as an example to illustrate the technical solution of the present application with reference to the accompanying drawings. Figure 2 and Figure 3 The structural schematic diagram and circuit diagram of the under-screen fingerprint recognition system of the mobile phone 100 are respectively shown.

[0082] like Figure 2 and Figure 3 As shown, the under-screen fingerprint recognition system of the mobile phone 100 includes: a touch screen 101, a capacitive sensor 102, an optical fingerprint sensor 103, and a micro control unit 104.

[0083] The touch screen 101, which can also be called a touch panel, can collect the user's touch operations on it (such as the user's operation on or near the touch screen using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a pre-set program. A display screen (not shown in the figure) can be set below the touch screen 101, which can be used to display information prompting the user to operate, such as "Perform fingerprint recognition authentication below". It can also display various applications in the mobile phone 100, such as Alipay, WeChat, Gallery, etc.

[0084] Capacitive sensors 102 are distributed on the touch screen 101 and are used to collect the capacitance fingerprint image of the user's finger when performing fingerprint recognition and the capacitance image of the base of the hand of the user's finger that uses fingerprint recognition. In addition, the capacitive sensors 102 can be distributed on the entire touch screen 101 or only on a part of the touch screen 101, such as the fingerprint recognition area.

[0085] The optical fingerprint sensor 103 is distributed on the touch screen 101 and is used to collect the optical fingerprint image of the user's finger when performing fingerprint recognition. In addition, the optical fingerprint sensor 103 can be distributed on the entire touch screen 101, or only on a part of the touch screen 101, such as the fingerprint recognition area.

[0086] The microcontroller unit 104 is used to control the sensor and perform operations on the sensor data. For example, the capacitive fingerprint image and the optical fingerprint image of the user's finger collected by the capacitive sensor 102 and the optical fingerprint sensor 103 are matched with the capacitive fingerprint image and the optical fingerprint image of the finger stored in the mobile phone 100. It can be understood that Figure 2 and Figure 3 The structure shown is only a specific structure for realizing the functions of the mobile phone 100 in the technical solution of the present application. Mobile phones 100 having other structures and capable of realizing similar functions are also applicable to the technical solution of the present application, and no limitation is made here.

[0087] The following is based on Figure 2 and Figure 3 The structure shown in FIG. 1 is a fingerprint recognition method described in detail according to an embodiment of the present application. Figure 4 As shown, the method includes:

[0088] 401: After the mobile phone 100 is started, after the user triggers fingerprint recognition, the mobile phone 100 collects the optical fingerprint image of the finger currently being fingerprinted by the user, the capacitance fingerprint image of the finger, and the capacitance image of the base of the hand to which the finger currently being fingerprinted belongs through the touch screen. Then the mobile phone 100 determines whether the finger currently being fingerprinted belongs to the left hand or the right hand based on the collected capacitance image of the base of the hand to which the finger currently being fingerprinted belongs. For the specific judgment process, please refer to Figure 5 Description.

[0089] If it is determined that the hand to which the finger currently used by the user belongs (for ease of explanation, it is assumed that it is the left hand below) is the left hand, then go to 402; if it is not determined which hand it belongs to, then go to 404.

[0090] 402: The mobile phone 100 matches the fingerprint feature information of at least one of the optical fingerprint image and the capacitance fingerprint image of the finger currently being fingerprinted by the user with the fingerprint feature information of each finger of the left hand that has been entered into the fingerprint information database. It can be understood that the fingerprint information database stores the fingerprint feature information of each finger of the two hands of the mobile phone 100 user, and the fingerprint feature information includes the fingerprint features extracted from at least one of the optical fingerprint image and the capacitance fingerprint image of each finger. For the specific matching process, please refer to Figure 6 Description.

[0091] If the match is successful, go to 405; otherwise, go to 403.

[0092] It can be understood that in some embodiments, when the fingerprint feature information of at least one of the collected optical fingerprint image and the capacitance fingerprint image of the user's finger is matched with the fingerprint feature information of the left hand finger in the fingerprint information library, the matching can be performed according to the frequency of the fingerprint feature information corresponding to each finger of the left hand in the fingerprint information library, and the one with the highest frequency is matched first. The frequency refers to the number of times the fingerprint feature information of each finger is successfully matched during the use of the mobile phone 100. The more times the fingerprint feature information of the corresponding finger is successfully matched, the higher its frequency.

[0093] 403: The mobile phone 100 matches the fingerprint feature information of at least one of the optical fingerprint image and the capacitance fingerprint image of the finger currently being fingerprinted by the user with the fingerprint feature information of the right hand finger in the fingerprint information database. For the specific matching process, please refer to Figure 6It can be understood that in some embodiments, when the fingerprint feature information of at least one of the optical fingerprint image and the capacitance fingerprint image of the user's finger is matched with the fingerprint feature information of each right hand finger in the fingerprint information library, the matching can be performed according to the frequency of the fingerprint feature information of each right hand finger in the fingerprint information library, and the one with the highest frequency is matched first. For the specific matching process, please refer to Figure 6 Description.

[0094] If the match is successful, go to 405; otherwise, go to 406.

[0095] 404: The mobile phone 100 matches the fingerprint feature information of at least one of the optical fingerprint image and the capacitance fingerprint image of the finger of the user currently undergoing fingerprint identification with the fingerprint feature information of the fingers of the two hands in the fingerprint information database.

[0096] It can be understood that in some embodiments, when the fingerprint feature information of at least one of the collected optical fingerprint image and the capacitance fingerprint image of the user's finger is matched with the fingerprint feature information of the fingers of the two hands in the fingerprint information library, the matching can be performed according to the frequency of the fingerprint feature information of the fingers of the two hands in the fingerprint information library, and the one with the higher frequency is matched first. For the specific matching process, please refer to Figure 6 Description.

[0097] If the match is successful, go to 405; otherwise, go to 406.

[0098] 405: The mobile phone 100 prompts the user that the fingerprint recognition is successful and increases the frequency of the fingerprint feature information of the finger that successfully matches the finger used by the user in the fingerprint information database by one.

[0099] 406: The mobile phone 100 prompts the user to enter a password for authentication or prompts the user that the fingerprint authentication fails.

[0100] The following introduces a technical solution for judging the left and right hands in 401 according to some embodiments of the present application.

[0101] Specifically, in some embodiments of the present application, left and right hand recognition can be performed by establishing a left and right hand judgment model, and model training can be performed on various computing devices with model training capabilities, such as desktop computers, servers, etc. The trained model is then transplanted to the mobile phone 100. For example, the model establishment process can be:

[0102] First, sample data is input into a convolutional neural network model, wherein the sample data includes a plurality of thumb and mouth capacitance images and / or a plurality of capacitance fingerprint images, and the hands to which these thumb and mouth capacitance images and capacitance fingerprint images belong are known.

[0103] After the sample data is input into the convolutional neural network model, the convolutional neural network model can extract the main features of the base of the thumb and / or fingers included in the sample data (e.g., the outline, shape, texture direction, etc. of the base of the thumb) through the processing of multiple convolutional layers and pooling layers, and output the confidence of the hand to which the sample data belongs determined based on these features, that is, whether it belongs to the left hand, the right hand, or it is impossible to determine the left or right hand. For example, the confidence can be represented by a numerical value, and a numerical value of (0, 0.2) indicates that it is determined to be the left hand, a numerical value of (0.8, 1) indicates that it is determined to be the right hand, and a numerical value of (0.2, 0.8) indicates that it is impossible to determine the left or right hand.

[0104] Then, according to the confidence of the convolutional neural network output, the various parameters of the convolutional neural network model are adjusted so that the output of the convolutional neural network model is consistent with the expected result of the sample data. For example, for a sample data A, the confidence of the convolutional neural network model output is 0.6, and the expected result of the sample data A is the right hand, then the parameters of the convolutional neural network model are adjusted so that the confidence of the convolutional neural network model output after the sample data A is input is between (0.8, 1). For another example, for a sample data B, the confidence of the convolutional neural network model output is 0.4, and the expected result of the sample data is the left hand, then the parameters of the convolutional neural network model are adjusted so that the confidence of the convolutional neural network model output after the sample data B is input is between (0, 0.2).

[0105] After a large amount of sample data is input into the above convolutional neural network model, if the confidence of the model output is consistent with the expected result of the sample data, it means that the model has been trained successfully.

[0106] Figure 5 According to some embodiments of the present application, the process of left and right hand recognition based on the above trained model is shown. Specifically, Figure 5 As shown:

[0107] 501: The mobile phone 100 inputs the capacitance information of the user collected by the mobile phone 100 into the convolutional neural network model. The capacitance information includes the capacitance image of the base of the hand to which the user's finger for fingerprint identification belongs or the capacitance image of the base of the hand to which the user's finger for fingerprint identification belongs and the capacitance fingerprint image of the finger for fingerprint identification of the user.

[0108] 502: The convolutional neural network model performs multi-layer convolution and pooling on the thumb capacitive image of the hand to which the user's finger currently undergoing fingerprint recognition belongs to, to extract the main features of the thumb capacitive image of the hand to which the user's finger currently undergoing fingerprint recognition belongs (e.g., the outline, shape, texture direction, etc. of the thumb capacitive image); or the convolutional neural network performs multi-layer convolution and pooling on the thumb capacitive image of the hand to which the user's finger currently undergoing fingerprint recognition belongs and the capacitance fingerprint image of the user's finger undergoing fingerprint recognition to extract the main features of the hand to which the user's finger currently undergoing fingerprint recognition belongs.

[0109] 503: The convolutional neural network model processes the features of the above-extracted thumb-to-mouth capacitance image of the hand to which the finger currently undergoing fingerprint recognition belongs through a fully connected layer to fuse all the above-obtained features.

[0110] 504: The convolutional neural network model outputs the confidence of the left and right hands of the user's current fingerprint identification finger from the fully connected layer.

[0111] Then, the hand to which the user's finger currently undergoing fingerprint recognition belongs is determined based on the output confidence levels of the left and right hands.

[0112] The following introduces the technical solution of fingerprint recognition based on some embodiments of the present application.

[0113] In some embodiments of the present application, the fingerprint matching in the above fingerprint recognition process can be performed by inputting at least one of the collected optical fingerprint image and the capacitance fingerprint image into a fingerprint matching neural network model. The fingerprint matching neural network model can be trained in any electronic device with model training capabilities. Specifically, Figure 6 According to an embodiment of the present application, a training process of a fingerprint matching neural network model is shown, including:

[0114] 601: The mobile phone 100 collects the optical fingerprint image of the user's specific finger (hereinafter, the right index finger is used as an example) and the capacitance value fingerprint image of the right index finger in advance, and pre-processes the collected optical fingerprint image and capacitance value fingerprint image of the user's specific finger (including denoising or discarding the optical fingerprint image or the capacitance value fingerprint image).

[0115] For example, when the user is performing fingerprint recognition authentication, since the mobile phone 100 will cache the optical fingerprint image and the capacitance value fingerprint image at a certain frequency when collecting them, the mobile phone 100 will collect multiple frames of optical fingerprint images and capacitance value fingerprint images of the fingerprint recognition area. There will be some invalid images in these multiple frames of optical fingerprint images or capacitance value fingerprint images of the fingerprint recognition area (for example, when the user starts to perform fingerprint recognition and presses the fingerprint recognition area, the contact area with the fingerprint recognition area is too small, which will cause the fingerprint image of the current frame collected by the fingerprint recognition area to be unclear or too small, and at this time, the fingerprint image of the current frame of the fingerprint recognition area is considered invalid). During preprocessing, these invalid frame fingerprint images will be discarded.

[0116] 602: The mobile phone 100 inputs the collected user optical fingerprint image into the optical fingerprint image convolution network of the fingerprint matching neural network model in advance to perform multi-layer convolution and pooling processing to obtain high-order feature information of the optical fingerprint image, such as the shape, size, and contour of the right index finger.

[0117] 603: Input the capacitance fingerprint image of the user's right hand collected in advance by the mobile phone 100 into the capacitance information convolution network of the fingerprint matching neural network model for multi-layer convolution and pooling processing to obtain high-order feature information of the capacitance fingerprint image of the user's right hand, such as the shape, size, and contour of the right index finger.

[0118] 604: Then, the high-order feature information of the optical fingerprint image of the user's right index finger obtained above and the high-order feature information of the capacitance value fingerprint image of the user's right hand are globally averaged pooled, and then the high-order feature information of the optical fingerprint image of the user's right index finger and the high-order feature information of the capacitance value fingerprint image of the user's right hand that have been globally averaged pooled are concatenated, that is, the high-order feature information of the optical fingerprint image of the user's right index finger obtained above and the high-order feature information of the capacitance value fingerprint image of the user's right hand are fused to obtain the fingerprint feature information of the corresponding user's right index finger.

[0119] 605: Input the fingerprint feature information of the right index finger after feature fusion into the fully connected network for training. When the result output by the fully connected network matches the fingerprint feature information of the optical fingerprint image of the user's right index finger and the right capacitance fingerprint image combined with the input, it is considered that the fingerprint matching neural network training is completed, otherwise the relevant parameters of the fully connected network (such as the weight value of the connection between each neuron) are modified until the result output by the fully connected network matches the fingerprint feature information of the right index finger of the user's right index finger combined with the optical fingerprint image of the user's right index finger and the right capacitance fingerprint image.

[0120] Specifically, a loss function (Formula 1) can be introduced to determine whether the output result of the fully connected network is consistent or similar to the input. When the loss function is close to 0 or converges, it is considered that the output result is consistent or similar to the input, that is, the fingerprint matching neural network training is completed.

[0121] In some embodiments of the present application, the accuracy of the fingerprint matching neural network model can be judged by the false acceptance rate (FAR), where the accuracy is used to characterize whether the result output by the fully connected network matches the input user's optical fingerprint image and capacitance fingerprint image. If the false acceptance rate is lower, the accuracy is higher, which means that the result output by the fully connected network matches the input more closely, that is, the result output by the fully connected network is consistent with or similar to the input. As shown in Formula 1, this is the loss function of the fingerprint matching neural network model:

[0122]

[0123]

[0124] When the fingerprint matching neural network model is trained, in order to reduce FAR, a constraint term (Formula 2) is added to the loss during the training process to limit FAR. Specifically, the upper limit threshold m of FAR is first set. If FAR is greater than the threshold m after a certain training, the constraint term in the loss function is If FAR is less than the threshold m after a training session, the constraint term in the loss function is 0.

[0125] In addition, it can be understood that the above-mentioned model training for fingerprint matching is based on the fingerprint feature information of the optical fingerprint image and the capacitance fingerprint image for fingerprint matching, and in other embodiments, model training and fingerprint matching can also be performed based only on the fingerprint feature information of the optical fingerprint image. The specific process is the same as Figure 6 The method shown in FIG. 1 is similar to that shown in FIG. 1 , except that only the high-order features of the optical fingerprint image are extracted, and the high-order features of the capacitance fingerprint image are not extracted, and then the high-order features of the optical fingerprint image are matched.

[0126] Reference Figure 7 In some embodiments of the present application, fingerprint recognition can be performed by using the features of the capacitance fingerprint image and the optical fingerprint image without distinguishing between the left and right hands, that is, the capacitance fingerprint image and the optical fingerprint image collected by the mobile phone 100 are input into Figure 6 Fingerprint matching neural network model shown in the figure is used to perform fingerprint recognition and authentication, specifically including:

[0127] 701: The mobile phone 100 collects an optical fingerprint image of a user's finger and a capacitive fingerprint image of the user's finger.

[0128] 702: The mobile phone 100 extracts the features of the optical fingerprint image and the capacitance fingerprint image of the user's finger, and matches the fingerprint feature information of the optical fingerprint image and the capacitance fingerprint image of the user's finger with the fingerprint feature information of all fingers in the fingerprint information library in the mobile phone 100. The specific matching method is as follows Figure 6 The matching method shown is consistent.

[0129] 703: If the match is successful, the frequency of the fingerprint feature information of the currently matched finger is increased by 1. If the match is unsuccessful, the user is prompted to enter a password for verification.

[0130] In addition, in the above embodiments, the capacitive fingerprint image and optical fingerprint image of the user are collected by the mobile phone 100, and then the left-right hand judgment model and the fingerprint matching neural network model are trained by the mobile phone 100, and the left-right hand judgment and fingerprint recognition authentication are performed by the mobile phone 100. It can be understood that the above-mentioned left-right hand judgment model and fingerprint matching neural network model can also be trained on the server side, and the left-right hand judgment and fingerprint recognition authentication are performed by the server. Among them, the left-right hand judgment and fingerprint recognition authentication functions can be trained with two separate models, or the left-right hand judgment and fingerprint recognition authentication functions can be combined into one model for training.

[0131] For ease of understanding, the following embodiments of the present application will be described in detail with Figure 2 and Figure 3 Taking the mobile phone 100 of the structure shown as an example, the fingerprint recognition method provided by the embodiment of the present application is specifically described in combination with the payment authentication scenario shown in Figures 8(a)-8(f). Specifically, it includes:

[0132] 1) The mobile phone 100 displays a home screen (800), which includes icons of multiple applications, for example, a WeChat icon 801.

[0133] 2) After the user opens WeChat, several of the user's frequently used contacts are displayed in the WeChat interface (802) in FIG8(b).

[0134] 3) The user selects his colleague Xiao Zhao to transfer money in the chat window in the WeChat interface (802), and enters the specific transfer amount of 100 yuan (see Figure 8(c)).

[0135] 4) When the user clicks "Confirm Payment", the mobile phone 100 interface will jump to the payment interface (803), and a prompt message "'WeChat Touch Tool D' Please verify the existing fingerprint for payment" (804) will pop up. At this time, the user can press the finger on the fingerprint recognition area (805) for fingerprint recognition (see Figure 8 (d)).

[0136] 5) The mobile phone 100 collects the user's optical fingerprint image, capacitance fingerprint image and the capacitance image of the base of the hand of the finger of the current user for fingerprint recognition through the touch screen, and uses the left and right hand recognition scheme and fingerprint matching technology mentioned above (reference Figure 4-6 When the fingerprint is successfully matched, the mobile phone 100 will jump to the payment success interface (806), and the user only needs to click "Finish" to complete the payment (see Figure 8 (e)).

[0137] 6) If fingerprint recognition is unsuccessful, the mobile phone 100 will vibrate to prompt the user to perform fingerprint recognition verification again. If the user fails to pay through fingerprint recognition authentication for multiple times (such as 3 times or other times), the mobile phone 100 will jump to the "Use Password" payment interface (807) to prompt the user to manually enter the password (see Figure 8 (f)). When the user enters the correct password, the mobile phone 100 interface will jump to the payment success interface (806).

[0138] For example, in some embodiments of the present application, the user can Fig. 9 The mobile phone 100 is operated in the manner shown. Then, when the user presses the fingerprint recognition area (805) with a finger to perform fingerprint recognition in the payment interface (803) (see FIG8(d)), the mobile phone 100 will collect the user's capacitance information through the touch screen, including the mobile phone 100 collecting the capacitance information of the user's finger currently performing fingerprint recognition through the fingerprint sensor and collecting the capacitance information of the base of the hand of the user operating the mobile phone 100 at this time through the capacitive sensor in the non-fingerprint recognition area of ​​the touch screen. And the mobile phone 100 obtains the capacitance fingerprint image of the finger currently performing fingerprint recognition, the pressing direction of the finger, etc. according to the capacitance information of the finger currently performing fingerprint recognition, and obtains the capacitance image of the base of the hand currently operating the mobile phone 100 according to the capacitance information of the base of the hand of the user operating the mobile phone 100 at this time.

[0139] Then, the mobile phone 100 will first obtain the confidence of the current hand based on the capacitive image of the current hand's base or the capacitive fingerprint image of the current finger, and then compare the confidence of the current hand with the confidence of the left and right hands already stored in the mobile phone 100. If the confidence of the current hand is greater than the confidence of the left hand already stored in the mobile phone 100 and greater than the confidence threshold, it is considered that the finger currently being fingerprinted by the user belongs to the left hand. If the confidence of the current hand is greater than the confidence of the right hand already stored in the mobile phone 100 and greater than the confidence threshold, it is considered that the finger currently being fingerprinted by the user belongs to the right hand. Continue as Fig. 9 As shown, it can be known that the finger currently being used for fingerprint recognition by the user belongs to the right hand.

[0140] Finally, the mobile phone 100 combines the collected optical fingerprint image of the user with the capacitance information of the finger currently being fingerprinted by the user and the optical fingerprint images of all the fingers of the right hand that have been stored in the mobile phone 100 according to the frequency of the optical fingerprint images of the right hand fingers, and matches them in sequence. Specifically, as shown in FIG10(a), the optical fingerprint images of all the fingers of the right hand that have been stored in the mobile phone 100, as well as the frequency of the optical fingerprint images (arranged in descending order of frequency). When matching, the mobile phone 100 will first match the optical fingerprint image of the finger currently being fingerprinted by the user with the optical fingerprint image of the right thumb with the highest frequency (frequency 2). If the match fails, it will continue to match the optical fingerprint image of the right index finger with a frequency of 2. If the match is successful, the frequency of the optical fingerprint image of the right index finger in the template library will be increased by one (see FIG10(b)).

[0141] When the optical fingerprint image of the user's finger currently undergoing fingerprint identification matches the optical fingerprint image of the right index finger already stored in the mobile phone 100, the user recognizes that the payment is successful through fingerprint identification, and the mobile phone 100 jumps to the payment success interface (806).

[0142] In some embodiments of the present application, when the optical fingerprint sensor 103 collects the optical fingerprint image of the user, the capacitance fingerprint image of the finger formed by the user's same finger in the fingerprint recognition area (805) of the mobile phone 100 touch screen will also be different, such as the pressing direction of the finger, etc. Therefore, in the above fingerprint matching process, the user's fingerprint can also be identified based on the capacitance fingerprint image of the finger currently performing fingerprint recognition authentication payment and the fingerprint feature information of the user's optical fingerprint image. For example, when the user enters the fingerprint information library, the pressing direction of the right index finger in the fingerprint recognition area (8805) of the mobile phone 100 touch screen is shown in Figure 11 (a), and when the user enters the fingerprint information library, the pressing direction of the right middle finger in the fingerprint recognition area (805) of the mobile phone 100 touch screen is shown in Figure 11 (b). Specifically, when it is determined that the finger of the user currently performing fingerprint identification belongs to the right hand, the mobile phone 100 matches the fingerprint feature information of the finger of the user currently performing fingerprint identification with the fingerprint feature information of all the fingers of the right hand in the fingerprint feature information library. For example, the pressing direction of the finger of the user currently performing fingerprint identification is shown in Figure 12(a), and the fingerprint feature information of all the fingers of the right hand in the fingerprint feature information library is shown in Figure 12(b) (arranged in descending order of frequency). Then, in the fingerprint matching process, the mobile phone 100 matches the fingerprint feature information of the finger of the user currently performing fingerprint identification with the fingerprint feature information of the right index finger and the fingerprint feature information of the right middle finger that have been entered in sequence according to frequency. When the fingerprint feature information of the finger of the user currently performing fingerprint identification successfully matches the fingerprint feature information of the right index finger with a high frequency (frequency is 2), the frequency of the fingerprint feature information of the right index finger in the fingerprint information library is increased by one (see Figure 12(c)).

[0143] When the fingerprint feature information of the user's finger currently undergoing fingerprint identification matches the fingerprint feature information of the user's right index finger that has been entered, the user recognizes that the payment is successful through fingerprint identification, and the mobile phone 100 jumps to the payment success interface (806).

[0144] Now combine Fig.13 The software structure block diagram of the terminal device of the embodiment of the present application is introduced. The software system of the terminal device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the terminal device. Fig.13 As shown, the layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer.

[0145] The application layer can include a series of application packages.

[0146] like Fig.13 As shown, the application package may include applications such as phone, camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0147] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0148] like Fig.13 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0149] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0150] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0151] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0152] The phone manager is used to provide communication functions for terminal devices, such as the management of call status (including answering, hanging up, etc.).

[0153] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0154] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal device vibrates, the indicator light flashes, etc.

[0155] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0156] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0157] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0158] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0159] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0160] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0161] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0162] A 2D graphics engine is a drawing engine for 2D drawings.

[0163] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0164] It should be noted that in the examples and description of this patent, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one" do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0165] Although the present application has been illustrated and described with reference to certain preferred embodiments thereof, it will be apparent to those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present application.

Claims

1. A fingerprint recognition method, characterized in that: include: Acquire capacitance information of a hand to which a finger used by a user for fingerprint recognition on the electronic device belongs, which is collected by a capacitance sensor of the electronic device, wherein the capacitance information includes at least one of a capacitance fingerprint image of the finger and a capacitive image of the base of the hand to which the finger belongs; Performing the fingerprint recognition based on the acquired capacitance information; The performing the fingerprint recognition based on the acquired capacitance information includes: Determining whether the hand to which the finger used by the user belongs is the left hand or the right hand of the user according to the thumb's mouth capacity image; The fingerprint of the finger used by the user is identified according to the determination result.

2. The method according to claim 1, characterized in that The performing the fingerprint recognition based on the acquired capacitance information further includes: Determine whether the hand to which the finger of the user belongs is the left hand or the right hand of the user according to the thumb-mouth capacitance image and the capacitance fingerprint image; The fingerprint of the finger used by the user is identified according to the determination result.

3. The method according to claim 2, characterized in that The identifying the fingerprint of the finger used by the user according to the determination result includes: When it is determined that the hand to which the finger used by the user belongs is the left hand or the right hand of the user, the fingerprint feature information of the finger used by the user is matched with the stored fingerprint feature information of each finger of the determined left hand or right hand, and when the match is successful, it is determined that the fingerprint recognition is passed; When it is not determined whether the hand to which the finger used by the user belongs is the left hand or the right hand of the user, the fingerprint feature information of the finger used by the user is matched with the stored fingerprint feature information of each finger of the left hand and the right hand, and the fingerprint recognition is determined to be passed if the match is successful.

4. The method according to claim 3, characterized in that The fingerprint feature information of the finger includes the fingerprint feature of the optical fingerprint image of the finger, wherein the optical fingerprint image is obtained by collecting the finger used by the user through the optical fingerprint sensor of the electronic device.

5. The method according to claim 4, characterized in that The fingerprint feature information of the finger also includes the fingerprint feature of the capacitance fingerprint image of the finger.

6. The method according to claim 3, characterized in that When matching the fingerprint feature information of the finger used by the user with the fingerprint feature information of each finger that has been stored, the order of the fingerprint feature information of each finger in matching is determined according to the frequency of the fingerprint feature information of each finger, wherein the greater the frequency of the fingerprint feature information of the finger, the higher the order in matching; The greater the frequency of the fingerprint feature information of a finger, the more times the fingerprint feature information of the finger has been matched in the historical fingerprint recognition process.

7. The method according to claim 1, characterized in that The performing the fingerprint recognition based on the acquired capacitance information includes: The fingerprint of the finger used by the user is identified based on the acquired capacitance fingerprint image and optical fingerprint image, wherein the optical fingerprint image is obtained by collecting the finger used by the user through the optical fingerprint sensor of the electronic device.

8. The method according to claim 1, characterized in that The identifying the fingerprint of the finger used by the user based on the acquired capacitance fingerprint image and optical fingerprint image includes: Extracting features from the capacitance fingerprint image and the optical fingerprint image respectively to obtain fingerprint feature information of the finger used by the user; The fingerprint feature information of the finger used by the user is matched with the stored fingerprint feature information of each finger, and if the match is successful, it is determined that the fingerprint recognition is passed.

9. An electronic device, characterized in that: include: A capacitive sensor, used to collect capacitance information of a hand to which a finger used by a user for fingerprint recognition on the electronic device belongs, wherein the capacitance information includes at least one of a capacitance fingerprint image of the finger and a capacitive image of the base of the hand to which the finger belongs; An optical fingerprint sensor, used to collect an optical fingerprint image of a finger used by a user when performing fingerprint recognition on the electronic device; a memory storing instructions; and At least one processor is configured to access the memory and to execute instructions on the memory to control the capacitive sensor and the optical fingerprint sensor to perform the fingerprint recognition method according to any one of claims 1 to 8.

10. A readable medium of an electronic device, characterized in that: The readable medium of the electronic device stores instructions, which, when executed on the electronic device, enable the electronic device to execute the fingerprint recognition method according to any one of claims 1 to 8.

Images