Fingerprint sensing device and method for judging position of fingerprint image

Abstract

The invention discloses a fingerprint sensing device and method for judging the position of a fingerprint image. The fingerprint sensing device comprises a fingerprint sensor and a controller. The fingerprint sensor is used for sensing a fingerprint of a finger to obtain a first fingerprint image. The controller is connected with the fingerprint sensor. The controller is used for generating a first coordinate according to the first fingerprint image. A deep learning model is built in the controller, the deep learning model is used for generating the first coordinate according to first fingerprint information, and the first fingerprint information is the first fingerprint image or first image information obtained after the first fingerprint image is processed.

Description

Technical Field

[0001] The present invention relates to a fingerprint sensing device, and more particularly to a fingerprint sensing device and method for determining the position of a fingerprint image. Background Art

[0002] When an existing fingerprint sensing device performs fingerprint registration, it can only provide a registration progress prompt on the fingerprint pattern on the display. For example, Figure 1 display the registration progress prompts that appear in sequence during the process from the start of fingerprint registration to the completion of fingerprint registration, where the darker part represents the fingerprint position that has been pressed. The existing fingerprint sensing device cannot determine the finger pressing position from the sensed fingerprint image. Therefore, the user cannot know which areas of the finger fingerprint have not been registered or have been registered, which is quite inconvenient for the user to perform fingerprint registration or fingerprint verification. Summary of the Invention

[0003] One object of the present invention is to provide a fingerprint sensing device and method for determining the position of a fingerprint image.

[0004] The present invention provides a fingerprint sensing device for determining the position of a fingerprint image, which includes a fingerprint sensor and a controller. The fingerprint sensor is used to sense the fingerprint of a finger to obtain a first fingerprint image. The controller is connected to the fingerprint sensor. The controller is used to generate a first coordinate according to the first fingerprint image. A deep learning model is built in the controller, and the deep learning model is used to generate the first coordinate according to a first fingerprint information, where the first fingerprint information is the first fingerprint image or the first image information after processing the first fingerprint image.

[0005] The present invention also provides a method for determining the position of a fingerprint image, including the following steps: providing a first fingerprint information to a deep learning model, where the first fingerprint information is a first fingerprint image or the first image information after processing the first fingerprint image; and according to the first fingerprint information, the deep learning model generates a first coordinate, where the deep learning model is trained to predict the position of the first fingerprint image according to the first fingerprint image or the first image information.

[0006] The fingerprint sensing device of the present invention can determine the position of the fingerprint image, so it can more accurately prompt the pressing part of the finger, thereby reducing the recognition error rate. Brief Description of the Drawings

[0007] Figure 1 Show the registration progress prompt during fingerprint registration.

[0008] Figure 2 Show an embodiment of the fingerprint sensing device of the present invention.

[0009] Figure 3 Displays the first embodiment of the method for determining the position of the fingerprint image of the present invention.

[0010] Figure 4 Displays the user's fingerprint.

[0011] Figure 5 Displays a fingerprint pattern on the display to indicate the sensed or unsensed areas in the fingerprint.

[0012] Figure 6 Displays a fingerprint pattern and guiding instructions for prompting finger movement on the display.

[0013] Figure 7 Is used to illustrate Figure 2 The training method of the deep learning model in

[0014] Figure 8 Displays the second embodiment of the method for determining the position of the fingerprint image of the present invention.

[0015] Figure 9 Displays the relative relationship between the first fingerprint image and the second fingerprint image.

[0016] Figure 10 Displays the fingerprint structure adjusted according to the relative relationship between multiple fingerprint images.

[0017] Explanation of reference numerals: 10 - fingerprint sensing device; 11 - fingerprint sensor; 12 - controller; 121 - deep learning model; 20 - first fingerprint image; 201 - first coordinate; 21 - second fingerprint image; 211 - second coordinate; 22 - third fingerprint image; 221 - third coordinate; 23 - fourth fingerprint image; 30 - display; 31 - guiding instruction; 40 - fingerprint pattern; 41 - training fingerprint image; 411 - training coordinate; 42 - center point; 50 - fingerprint structure; 51 - fingerprint structure; 52 - fingerprint structure; S10~S19 - steps. Detailed implementation manners

[0018] Figure 2 Displays an embodiment of the fingerprint sensing device of the present invention. Figure 2The fingerprint sensing device 10 includes a fingerprint sensor 11 and a controller 12. The fingerprint sensor 11 is used to sense the fingerprint of a finger to obtain a fingerprint image. The controller 12 is connected to the fingerprint sensor 11 to obtain the fingerprint image. The controller 12 is used to generate a coordinate according to the fingerprint image. The controller 12 can be, but is not limited to, a microcontroller (MCU), a central processing unit (CPU) of a desktop computer, or a CPU of a laptop computer. The controller 12 has a built-in deep learning model 121. The deep learning model 121 is trained to generate the coordinate according to a fingerprint information, and the coordinate corresponds to the position of the fingerprint image. The fingerprint information can be, but is not limited to, the fingerprint image or the image information after processing the fingerprint image. In an embodiment, the controller 12 and the fingerprint sensor 11 can be integrated into an integrated circuit device.

[0019] Figure 3 Show the first embodiment of the method for determining the position of the fingerprint image of the present invention. Figure 4 Show the user's fingerprint. Refer to Figures 2 to 4 , the fingerprint sensor 11 senses the fingerprint of a finger to obtain a first fingerprint image 20, as shown in step S10. The controller 12 obtains the first fingerprint image 20 from the fingerprint sensor 11 and then proceeds to step S11 to provide a first fingerprint information to the deep learning model 121. The first fingerprint information is a first fingerprint image 20 or a first image information generated after the controller 11 processes the first fingerprint image 20, where the first image information includes, but is not limited to, a density image or a model embedding layer. The deep learning model 121 generates a first coordinate 201 according to the first fingerprint information, as shown in step S12. In Figure 4 , the first coordinate 201 is the center point of the first fingerprint image 20, but the present invention is not limited thereto.

[0020] Since the fingerprint sensing device 10 of the present invention can determine the position of the obtained fingerprint image, when registering or authenticating, the user can be prompted about the finger pressing position according to the fingerprint position determined by the present invention. For example, when registering, if the first fingerprint image 20, the second fingerprint image 21, and the third fingerprint image 22 in Figure 4 are obtained, the deep learning model 121 of the fingerprint sensing device 10 can generate a first coordinate 201, a second coordinate 211, and a third coordinate 221 according to the first fingerprint image 20, the second fingerprint image 21, and the third fingerprint image 22. According to the first coordinate 201, the second coordinate 211, and the third coordinate 221, it can be determined that most of the currently registered fingerprint images are in the left side or the fingertip area of the finger. Therefore, a fingerprint pattern can be displayed through a display 30, and the sensed or unsensed areas can be marked on the fingerprint pattern, as Figure 5 shown. In Figure 5Among them, the brighter area in the upper left of the fingerprint pattern represents the sensed area of the fingerprint, and the remaining darker areas represent the unsensed areas of the fingerprint. The user can adjust the pressing position of the finger through the indication of the fingerprint pattern on the display 30. For example, the user can change to press the fingertip area, the finger root area or the right side area of the finger to make the registered fingerprint more complete, which helps to improve the accuracy rate during fingerprint verification. When performing fingerprint verification to identify the user's identity, if the registered fingerprint images are Figure 4 the first fingerprint image 20, the second fingerprint image 21 and the third fingerprint image 22, when the user presses the fingerprint sensor 11 with the lower right part of the finger to obtain a fourth fingerprint image 23, as Figure 6 shown, the result of the fingerprint verification will be judged as failed and a fingerprint pattern will be displayed on the display 30 of an electronic device (such as a laptop or a mobile phone). The registered fingerprint area (such as Figure 6 the brighter area in the upper left) and the position of the currently obtained fourth fingerprint image 23 will be marked on the fingerprint pattern, and the user will be prompted to slide the finger to the lower right through visual guidance instructions, such as arrow 31 or auditory guidance instructions (such as voice).

[0021] Figure 7 is used to illustrate Figure 2 the training method of the deep learning model in Figure 7 As shown, the training method of the deep learning model 121 includes providing multiple training fingerprint images 41 and the coordinates 411 of each training fingerprint image 41. These coordinates 411 respectively represent the positions of these fingerprint images 41 in the fingerprint pattern 40. In this embodiment, the coordinates 411 of the training fingerprint image 41 are the positions of the center points of the training fingerprint images 41 relative to the center point 42 of the fingerprint pattern 40 (that is, with the center point 42 as the coordinate origin), but the present invention is not limited thereto. Then, multiple groups of training fingerprint information are provided to the deep learning model 121, where each group of training fingerprint information includes a training fingerprint image 41 and its coordinates 411, or includes the training image information after processing a training fingerprint image 41 and the coordinates 411 of the training fingerprint image 41. The training image information includes but is not limited to density images or model embedding layers. The trained deep learning model 121 can predict the coordinates of the obtained fingerprint image or the image information after processing the fingerprint image.

[0022] The angle of each finger press is not necessarily the same. Through Figure 8 the second embodiment of the method for judging the position of the fingerprint image shown in the present invention, it helps to more accurately judge the position of the fingerprint image. Figure 8 The method of Figure 3 includes steps S10 to S12 of Figure 3Description. In step 13, the fingerprint sensor 11 continues to sense the fingerprint of the finger to obtain a second fingerprint image 21. The controller 12 obtains the second fingerprint image 21 from the fingerprint sensor 11, and then proceeds to step S14 to provide a second fingerprint information to the deep learning model 121. The second fingerprint information may be, but is not limited to, the second fingerprint image 21 or the second image information generated after the second fingerprint image 21 is processed by the controller 12, where the second image information includes, but is not limited to, a density image or a model embedding layer. Next, the deep learning model 121 generates a second coordinate 211 according to the second fingerprint information, as shown in step S15. In Figure 4 , the second coordinate 211 is the center point of the second fingerprint image 21, but the present invention is not limited thereto.

[0023] According to the first coordinate 201 and the second coordinate 211, it can be determined whether the first fingerprint image 20 and the second fingerprint image 21 overlap. After determining that a part of the first fingerprint image 20 and the second fingerprint image 21 overlaps, step S16 is performed. Step S16 may use, but is not limited to, a fingerprint matching algorithm or a computer vision method (such as AKAZE, SURF, SIFT, or other feature point comparison methods) to determine the relative relationship between the first fingerprint image 20 and the second fingerprint image 21 (as Figure 9 shown). Determining the relative relationship between multiple fingerprint images is well known to those of ordinary skill in the art of fingerprint sensing, and will not be elaborated herein.

[0024] After determining the relative relationship between the first fingerprint image 20 and the second fingerprint image 21. Step S17 can adjust the first coordinate 201 of the first fingerprint image 20 and / or the second coordinate 211 of the second fingerprint image 21 according to the relative relationship to obtain a finger pressing position closer to the actual situation. Figure 10 Displays the fingerprint structure adjusted according to the relative relationship between the foregoing fingerprint images. In Figure 10 , the fingerprint structure 50 is the fingerprint structure obtained according to steps S10 - S15 of Figure 8 , the fingerprint structure 51 is the fingerprint structure obtained according to step S16 of Figure 8 , and the fingerprint structure 52 is the fingerprint structure obtained according to step S17 of Figure 8 . As Figure 10 shown, compared with the fingerprint structure 50, the adjusted fingerprint structure 52 is more accurate. In an embodiment, step S17 can be performed to adjust the first coordinate 201 or the second coordinate 211 using, but not limited to, the Random Sample Consensus (RANSAC) algorithm or the Least Squares Estimator (LSE) algorithm.

[0025] From the above description, it can be understood that the present invention is inFigure 3 In the embodiment related to Figure 8 , it may further include displaying a fingerprint pattern on a display, and marking on the fingerprint pattern displayed on the display according to the first coordinate and / or the second coordinate. For example, marking on the fingerprint pattern the areas of the fingerprint that have been sensed or not sensed. Alternatively, the present invention may further include displaying a fingerprint pattern on a display, and displaying a guiding indication on the display according to the position of the registered fingerprint template and the first coordinate to guide the user to perform fingerprint verification.

[0026] Figure 8 The shown embodiment is not intended to limit the order of each step and the execution entity. For example, both steps S13 and S10 are performed by the fingerprint sensor 11, and step S13 may be performed immediately after step S10. In Figure 3 In the embodiment related to Figure 8 , steps S10 and S13 are performed by the fingerprint sensor 11, while steps S11, S12, S14 and S15 may be performed by the controller 12. In different embodiments, the deep learning model 121 may be built on the host connected to the controller 12 (such as a laptop or a mobile phone), and steps S11, S12, S14 and S15 are performed by the central processing unit of the host. Step S16 may be performed by the controller 12 or the central processing unit of the host. Step S17 may be performed by the controller 12 or the central processing unit of the host.

[0027] The above description is only an embodiment of the present invention and does not impose any formal limitation on the present invention. Although the present invention has been provided with the above embodiments, it is not intended to limit the present invention. Any person with ordinary knowledge in the technical field to which the present invention pertains, without departing from the scope of the technical solution of the present invention, may make some changes or modifications to the above-disclosed technical content to form equivalent embodiments with equivalent changes. However, as long as it does not depart from the content of the technical solution of the present invention, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A fingerprint sensing device for determining the position of a fingerprint image, characterized in that: include: A fingerprint sensor, used for sensing a fingerprint of a finger to obtain a first fingerprint image; as well as A controller is connected to the fingerprint sensor, and the controller is used to generate a first coordinate according to the first fingerprint image. The controller has a built-in deep learning model, wherein the deep learning model is used to generate the first coordinate according to a first fingerprint information, and the first fingerprint information includes the first fingerprint image or the first image information after the first fingerprint image is processed.

2. The fingerprint sensing device according to claim 1, wherein: The controller also includes providing a second fingerprint information to the deep learning model, wherein the second fingerprint information is a second fingerprint image partially overlapping with the first fingerprint image or a second image information after the second fingerprint image is processed; the deep learning model generates a second coordinate according to the second fingerprint image; the controller uses a fingerprint matching algorithm or a computer vision method to determine the relative relationship between the first fingerprint image and the second fingerprint image, and adjusts the first coordinate or the second coordinate according to the relative relationship.

3. The fingerprint sensing device as claimed in claim 2, characterized in that: The controller includes using a random sampling consensus algorithm or a least square estimation algorithm to adjust the first coordinate or the second coordinate.

4. The fingerprint sensing device according to claim 1, wherein: The training method of the deep learning model includes providing multiple sets of training fingerprint information to the deep learning model, each set of training fingerprint information includes a training fingerprint image and the coordinates of the training fingerprint image, or includes processed image information of the training fingerprint image and the coordinates of the training fingerprint image.

5. The fingerprint sensing device as claimed in claim 4, characterized in that: The training image information includes a density image or a model embedding layer.

6. A method for determining the position of a fingerprint image, characterized in that: The following steps are involved: A. providing a first fingerprint information to a deep learning model, the first fingerprint information including a first fingerprint image or first image information after the first fingerprint image is processed; and B. Based on the first fingerprint information, the deep learning model generates a first coordinate, wherein the deep learning model is trained to predict the position of the first fingerprint image based on the first fingerprint information.

7. The method for determining the position of a fingerprint image as claimed in claim 6, characterized in that: Also includes: C. providing a second fingerprint information to the deep learning model, wherein the second fingerprint information is a second fingerprint image partially overlapping with the first fingerprint image or a second image information after the second fingerprint image is processed; D. The deep learning model generates a second coordinate according to the second fingerprint information; E. using a fingerprint matching algorithm or a computer vision method to determine the relative relationship between the first fingerprint image and the second fingerprint image; and F. Adjust the first coordinate or the second coordinate according to the relative relationship.

8. The method for determining the position of a fingerprint image as claimed in claim 7, wherein: The step F includes adjusting the first coordinate or the second coordinate using a random sampling consensus algorithm or a least square estimation algorithm.

9. The method for determining the position of a fingerprint image according to claim 6 or 8, characterized in that: Also includes: displaying a fingerprint pattern on a display; as well as The fingerprint pattern displayed on the display is marked according to the first coordinate.

10. The method for determining the position of a fingerprint image as claimed in claim 9, wherein: The method also includes marking on the fingerprint pattern the areas in the fingerprint that have been sensed or not sensed.

11. The method for determining the position of a fingerprint image according to claim 6 or 8, characterized in that: Also includes: displaying a fingerprint pattern on a display; as well as A guiding instruction is displayed on the display according to the position of the registered fingerprint template and the first coordinate.

12. The method for determining the position of a fingerprint image as claimed in claim 6, wherein: The training method of the deep learning model includes providing multiple sets of training fingerprint information to the deep learning model, each set of training fingerprint information includes a training fingerprint image and the coordinates of the training fingerprint image, or includes training image information after the training fingerprint image is processed and the coordinates of the training fingerprint image.

13. The method for determining the position of a fingerprint image as claimed in claim 12, wherein: The training image information includes a density image or a model embedding layer.

Images