Fingerprint sensing device for determine a position of a fingerprint image and method thereof

Abstract

A fingerprint sensing device for determining the location of a fingerprint image includes a fingerprint sensor and a controller. The fingerprint sensor is used to sense the fingerprint of a finger to obtain a first fingerprint image. The controller is connected to the fingerprint sensor. The controller is used to generate a first coordinate based on the first fingerprint image. The controller has a built-in deep learning model, wherein the deep learning model is used to generate the first coordinate based on first fingerprint information, the first fingerprint information being the first fingerprint image or first image information of the first fingerprint image after processing.

Description

Technical Field

[0001] This invention relates to a fingerprint sensing device, and more particularly to a fingerprint sensing device and method for determining the position of a fingerprint image. Prior Technology

[0002] Existing fingerprint sensing devices can only provide registration progress indicators on the fingerprint pattern displayed on the screen during fingerprint registration. For example, Figure 1 shows the registration progress indicators that appear sequentially from the start to the completion of fingerprint registration, where the darker areas represent the fingerprint positions that have been pressed. Existing fingerprint sensing devices cannot determine the finger pressing position from the sensed fingerprint image. Therefore, users cannot know which areas of their fingerprint have not been registered or have been registered, which is quite inconvenient for users when registering or verifying their fingerprints. Summary of the Invention

[0003] One of the objectives of this invention is to provide a fingerprint sensing device and method for determining the location of a fingerprint image.

[0004] According to the present invention, a fingerprint sensing device for determining the location of a fingerprint image includes a fingerprint sensor and a controller. The fingerprint sensor is used to sense the fingerprint of a finger to obtain a first fingerprint image. The controller is connected to the fingerprint sensor. The controller is used to generate a first coordinate based on the first fingerprint image. The controller has a built-in deep learning model, wherein the deep learning model is used to generate the first coordinate based on first fingerprint information, wherein the first fingerprint information is the first fingerprint image or first image information of the first fingerprint image after processing.

[0005] According to the present invention, a method for determining the location of a fingerprint image includes the following steps: providing a first fingerprint information to a deep learning model, the first fingerprint information being a first fingerprint image or first image information of the first fingerprint image after processing; and generating a first coordinate based on the first fingerprint information, wherein the deep learning model is trained to predict the location of the first fingerprint image based on the first fingerprint image or the first image information.

[0006] The fingerprint sensing device of the present invention can determine the position of the fingerprint image, thus providing more accurate indication of the finger pressing position and reducing the recognition error rate. Simple Explanation of the Diagram

[0007] Figure 1 shows the registration progress prompts during fingerprint registration. 2 shows an embodiment of a fingerprint sensing device of the present invention. 3 shows a first embodiment of a method of the invention for determining the position of a fingerprint image. Figure 4 shows the user’s fingerprint. Figure 5 shows displaying a fingerprint pattern on the display to mark the area in the fingerprint that has been or has not been sensed. Figure 6 shows a fingerprint pattern displayed on the display as well as guidance indications prompting finger movement. Figure 7 is used to illustrate the training method of the deep learning model in Figure 2 . 8 shows a second embodiment of the method of determining the position of a fingerprint image of the present invention. Figure 9 shows the relative relationship between the first fingerprint image and the second fingerprint image. Figure 10 shows the fingerprint structure adjusted according to the relative relationship between multiple fingerprint images. Implementation

[0008] 2 shows an embodiment of a fingerprint sensing device of the present invention. The fingerprint sensing device 10 of FIG. The fingerprint sensor 11 is used to sense the fingerprint of the finger to obtain a fingerprint image. The controller 12 connects the fingerprint sensor 11 to obtain the fingerprint image. The controller 12 is used to generate a coordinate based on the fingerprint image. Controller 12 may be, but is not limited to, a microcontroller (MCU), the central processing unit (CPU) of a desktop computer, or the CPU of a notebook computer. A deep learning model 121 is built into the controller 12 . The deep learning model 121 is trained to generate the coordinates that correspond to the position of the fingerprint image based on a fingerprint of information. The fingerprint information may be, but is not limited to, the fingerprint image or the processed image information of the fingerprint image. In one embodiment, the controller 12 and the fingerprint sensor 11 may be integrated into an integrated circuit device.

[0009] Figure 3 shows a first embodiment of the method for determining the location of a fingerprint image according to the present invention. Figure 4 shows a user's fingerprint. Referring to Figures 2 to 4, the fingerprint sensor 11 senses the fingerprint of a finger to obtain a first fingerprint image 20, as shown in step S10. The controller 12 obtains the first fingerprint image 20 from the fingerprint sensor 11 and then performs step S11 to provide first fingerprint information to the deep learning model 121. The first fingerprint information is the first fingerprint image 20 or first image information generated after the first fingerprint image 20 is processed by the controller 11, wherein the first image information includes, but is not limited to, density images or model embedding layers. The deep learning model 121 generates a first coordinate 201 based on the first fingerprint information, as shown in step S12. In Figure 4, the first coordinate 201 is the center point of the first fingerprint image 20, but the present invention is not limited thereto.

[0010] Since the fingerprint sensing device 10 of the present invention can determine the position of the acquired fingerprint image, it can prompt the user's finger pressing position based on the fingerprint position determined according to the present invention during registration or authentication. For example, during registration, if the first fingerprint image 20, the second fingerprint image 21, and the third fingerprint image 22 in FIG4 have been acquired, the deep learning model 121 of the fingerprint sensing device 10 can generate a first coordinate 201, a second coordinate 211, and a third coordinate 221 based on the first fingerprint image 20, the second fingerprint image 21, and the third fingerprint image 22. Based on the first coordinate 201, the second coordinate 211, and the third coordinate 221, it can be determined that most of the currently registered fingerprint images are on the left side of the finger or the fingertip area. Therefore, a fingerprint pattern can be displayed on a display 30, and the areas that have been sensed or not sensed can be marked on the fingerprint pattern, as shown in FIG5. In FIG5, the brighter area in the upper left corner of the fingerprint pattern represents the area of ​​the fingerprint that has been sensed, and the remaining darker areas represent the areas of the fingerprint that have not been sensed. Users can adjust the pressure point of their fingers by displaying the fingerprint pattern on the display 30. For example, users can press the pad area, the base area, or the right side area of ​​their fingers to make the registered fingerprint more complete, which helps to improve the accuracy of fingerprint verification. When verifying the user's identity through fingerprint verification, if the registered fingerprint images are the first fingerprint image 20, the second fingerprint image 21, and the third fingerprint image 22 in Figure 4, when the user presses the fingerprint sensor 11 with the lower right part of their finger to obtain a fourth fingerprint image 23, as shown in Figure 6, the fingerprint verification result will be judged as a failure. A fingerprint pattern will be displayed on the display 30 of an electronic device (such as a laptop or mobile phone). The fingerprint pattern will indicate the registered fingerprint area (the brighter area in the upper left of Figure 6) and the position of the currently obtained fourth fingerprint image 23. Visual guidance, such as arrow 31, or auditory guidance (such as voice) will prompt the user to slide their finger to the lower right.

[0011] Figure 7 is used to illustrate the training method of the deep learning model in Figure 2 . As shown in Figure 7 , the training method of the deep learning model 121 includes providing multiple training fingerprint images 41 as well as the coordinates 411 of each training fingerprint image 41 . These coordinates 411 represent, respectively, the positions of these fingerprint images 41 in the fingerprint pattern 40 . In this embodiment, the coordinates 411 of the training fingerprint image 41 are the position of the center point of the training fingerprint image 41 relative to the center point 42 of the fingerprint pattern 40 (i.e., with the center point 42 as the coordinate origin), but the invention is not limited to this. Then, multiple sets of training fingerprint information are provided to the deep learning model 121 , where each set of training fingerprint information includes a training fingerprint image 41 as well as its coordinates 411 , or includes a processed training image information of a training fingerprint image 41 as well as coordinates 411 of the training fingerprint image 41 . The training image information includes, but is not limited to, density images or model embedding layers. The trained deep learning model 121 can predict the coordinates of the fingerprint image based on the acquired fingerprint image or the processed image information of the fingerprint image.

[0012] The angle of each press of the finger is not necessarily the same, and it is helpful to determine the position of the fingerprint image more accurately by resorting to a second embodiment of the method of the present invention for determining the position of the fingerprint image shown in FIG. The method of FIG. 8 includes steps S10 through S12 of FIG. Step 13 continues to sense the fingerprint of the finger with a fingerprint sensor 11 to obtain a second fingerprint image 21 . The controller 12 obtains the second fingerprint image 21 from the fingerprint sensor 11 , and then performs step S14 to provide the first-second fingerprint information to the deep learning model 121 . The second fingerprint information may be, but is not limited to, a second fingerprint image 21 or a second image information generated after the second fingerprint image 21 is processed by the controller 12 , wherein the second image information includes, but is not limited to, a density image or a model embedding layer. Next, the deep learning model 121 generates a second coordinate 211 based on that second fingerprint information, as shown in step S15. 4, the second coordinate 211 is the center point of the second fingerprint image 21 , but the invention is not limited to this.

[0013] According to the first coordinate 201 and the second coordinate 211 it can be determined whether there is an overlap between the first fingerprint image 20 and the second fingerprint image 21 . After judging that there is a partial overlap between the first fingerprint image 20 and the second fingerprint image 21 , step S16 is performed. Step S16 may determine the relative relationship of the first fingerprint image 20 and the second fingerprint image 21 (as shown in Figure 9 Determining the relative relationship between multiple fingerprint images is well known to those in the technical field familiar with fingerprint sensing and will not be detailed here.

[0014] After determining the relative relationship between the first fingerprint image 20 and the second fingerprint image 21, step S17 can adjust the first coordinate 201 of the first fingerprint image 20 and / or the second coordinate 211 of the second fingerprint image 21 according to the relative relationship to obtain a finger pressing position closer to the actual situation. Figure 10 shows the fingerprint structure after adjustment according to the aforementioned relative relationship between the fingerprint images. In Figure 10, fingerprint structure 50 is the fingerprint structure obtained according to steps S10-S15 of Figure 8, fingerprint structure 51 is the fingerprint structure obtained according to step S16 of Figure 8, and fingerprint structure 52 is the fingerprint structure obtained according to step S17 of Figure 8. As shown in Figure 10, the adjusted fingerprint structure 52 is more accurate than fingerprint structure 50. In one embodiment, the first coordinate 201 or the second coordinate 211 can be adjusted in step S17 using, but not limited to, the Random Sample Consensus (RANSAC) algorithm or the Least Squares Estimator (LSE) algorithm.

[0015] As can be understood from the above description, in the embodiments of Figures 3 and 8, the present invention may further include displaying a fingerprint pattern on a display screen, and marking on the fingerprint pattern displayed on the display screen according to a first coordinate and / or a second coordinate, for example, marking on the fingerprint pattern an area of ​​a fingerprint that has been sensed or not sensed. Alternatively, the present invention may further include displaying a fingerprint pattern on a display screen, and displaying a guidance instruction on the display screen according to the position of a registered fingerprint template and the first coordinate to guide the user to perform fingerprint verification.

[0016] The embodiments shown in Figure 8 do not intend to limit the order of the steps or the entity executing them. For example, steps S13 and S10 are both performed by the fingerprint sensor 11, and step S13 can be performed immediately after step S10. In the embodiments of Figures 3 and 8, steps S10 and S13 are performed by the fingerprint sensor 11, while steps S11, S12, S14, and S15 can be executed by the controller 12. In different embodiments, the deep learning model 121 can be built on the host computer (e.g., a laptop or mobile phone) connected to the controller 12, and steps S11, S12, S14, and S15 can be executed by the central processing unit of the host computer. Step S16 can be executed by the controller 12 or the central processing unit of the host computer. Step S17 can be executed by the controller 12 or the central processing unit of the host computer.

[0017] The above description is merely an embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above by way of embodiment, it is not intended to limit the present invention. Anyone skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

[0018] 10: Fingerprint sensor 11: Fingerprint sensor 12: Controller 121: Deep Learning Models 20: First fingerprint image 201: First Coordinate 21: Second fingerprint image 211: Second coordinate 22: Third fingerprint image 221: Third Coordinate 23: Fourth fingerprint image 30: Monitor 31: Guidance Instructions 40: Fingerprint pattern 41: Training fingerprint images 411: Training Coordinates 42: Center point 50: Fingerprint structure 51: Fingerprint Structure 52: Fingerprint Structure S10~S19: Steps

Claims

1. A fingerprint sensing device for determining the position of a fingerprint image, comprising: A fingerprint sensor is used to sense the fingerprint of a finger to obtain a first fingerprint image; The system also includes a controller connected to the fingerprint sensor. The controller generates a first coordinate based on the first fingerprint image. The controller has a built-in deep learning model, which generates the first coordinate based on first fingerprint information. The first fingerprint information includes the first fingerprint image or first image information processed from the first fingerprint image. The training method of the deep learning model includes providing multiple sets of training fingerprint information to the deep learning model. Each set of training fingerprint information includes a training fingerprint image and the coordinates of the training fingerprint image, or includes processed image information of the training fingerprint image and the coordinates of the training fingerprint image.

2. The fingerprint sensing device as claimed in claim 1, wherein the controller further includes providing a second fingerprint information to the deep learning model, the second fingerprint information being a second fingerprint image partially overlapping the first fingerprint image or second image information of the second fingerprint image after processing; the deep learning model generating a second coordinate based on the second fingerprint image; the controller using a fingerprint matching algorithm or computer vision method to determine the relative relationship between the first fingerprint image and the second fingerprint image, and adjusting the first coordinate or the second coordinate based on the relative relationship.

3. The fingerprint sensing device as claimed in claim 2, wherein the controller includes adjusting the first coordinate or the second coordinate using a random sampling consensus algorithm or a least squares estimation algorithm.

4. The fingerprint sensing device as claimed in claim 1, wherein the training image information includes a density image or a model embedding layer.

5. A method for determining the location of a fingerprint image, comprising the following steps: A. providing first fingerprint information to a deep learning model, the first fingerprint information including a first fingerprint image or first image information of the first fingerprint image after processing; and B. generating a first coordinate based on the first fingerprint information, wherein the deep learning model is trained to predict the location of the first fingerprint image based on the first fingerprint information; wherein... The training method of the deep learning model includes providing multiple sets of training fingerprint information to the deep learning model. Each set of training fingerprint information includes a training fingerprint image and the coordinates of the training fingerprint image, or includes training image information after processing the training fingerprint image and the coordinates of the training fingerprint image.

6. The method for determining the location of a fingerprint image as described in claim 5 further includes: C. Provide a second fingerprint information to the deep learning model, the second fingerprint information being a second fingerprint image that partially overlaps with the first fingerprint image or second image information after processing the second fingerprint image; D. Based on the second fingerprint information, the deep learning model generates a second coordinate; E. Use a fingerprint matching algorithm or computer vision method to determine the relative relationship between the first fingerprint image and the second fingerprint image; and F. Adjust the first coordinate or the second coordinate based on the relative relationship.

7. The method for determining the location of a fingerprint image as described in claim 6, wherein step F includes adjusting the first coordinate or the second coordinate using a random sampling consensus algorithm or a least squares estimation algorithm.

8. The method for determining the location of a fingerprint image as described in claim 5 or 7 further includes: Display a fingerprint pattern on a monitor; And mark the fingerprint pattern displayed on the display according to the first coordinate.

9. The method for determining the location of a fingerprint image as described in claim 8 further includes marking areas of a fingerprint that have been sensed or not sensed on the fingerprint pattern.

10. The method for determining the location of a fingerprint image as described in claim 5 or 7 further includes: Display a fingerprint pattern on a monitor; And based on the location of the registered fingerprint template and the first coordinate, a guidance instruction is displayed on the display.

11. The method for determining the location of a fingerprint image as described in claim 5, wherein the training image information includes a density image or a model embedding layer.

Images