Fingerprint identification method and fingerprint identification device

Abstract

The invention discloses a fingerprint identification method and a fingerprint identification device. The fingerprint identification method comprises the following steps: receiving an input fingerprint image; m image blocks are obtained by dividing the input fingerprint image, and M is an integer larger than 1; calculating a score of each of the M image blocks by inputting the M image blocks into a neural network for fingerprint recognition; in response to the fact that the scores of N image blocks in the M image blocks have been calculated and the scores of the remaining image blocks except the N image blocks in the M image blocks have not been calculated, when the scores of the N image blocks meet a preset condition, stopping calculating the scores of the remaining image blocks, and identifying a fingerprint in the input fingerprint image as true or false, where N is an integer less than or equal to M.

Description

Technical Field

[0001] The present disclosure relates to image processing, and more particularly, to a fingerprint recognition method and a fingerprint recognition device. Background Art

[0002] Biometrics are physical measurements and calculations related to human characteristics. For example, fingerprint biometrics can be used to verify or identify a person. Fingerprint recognition has been widely used and deployed in authentication scenarios due to its efficiency and accuracy. These authentication scenarios can include high-security applications (such as authentication in the context of financial transactions, smartphone unlocking, and access control).

[0003] Although fingerprint recognition has very high real-time requirements, it is generally deployed in electronic devices with limited resources (such as embedded devices). Summary of the invention

[0004] The present invention provides a fingerprint recognition method and a fingerprint recognition device.

[0005] In one aspect of the present disclosure, a fingerprint recognition method may include: receiving an input fingerprint image; determining a plurality of detail points appearing in the input fingerprint image, wherein the number of the plurality of detail points is M; and identifying the legitimacy of a fingerprint in the input fingerprint image by performing an iterative method, wherein the iterative method includes: obtaining an image block from the input fingerprint image, wherein the image block corresponds to a corresponding detail point among the plurality of detail points; calculating a score of the image block by inputting the image block into a neural network for fingerprint recognition; adding the score of the image block to a total score of a previously obtained image block to obtain an added score; and identifying the legitimacy of the fingerprint in the input fingerprint image when the added score satisfies a preset condition.

[0006] In one embodiment, the preset condition may include a first condition, wherein the first condition indicates that the total score is greater than or equal to the product of M and a preset threshold score, and wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image may include: when the total score satisfies the first condition, identifying the fingerprint in the input fingerprint image as true.

[0007] In one embodiment, the preset condition may include a second condition, wherein the second condition indicates that the sum of the total score and the first value is less than the product of M and a preset threshold, and the first value is equal to the product of the difference between M and N and the maximum score that the neural network can output, wherein N corresponds to a number obtained by adding one to the number of previously obtained image blocks, and wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image may include: when the total score satisfies the second condition, identifying the fingerprint in the input fingerprint image as false.

[0008] In one embodiment, a fingerprint recognition method may include: determining the center coordinates of an input fingerprint image; and determining a first number of image blocks corresponding to a first number of detail points closest to the center coordinates and a second number of image blocks corresponding to a second number of detail points farthest from the center coordinates among the multiple image blocks as M image blocks, wherein the sum of the first number and the second number is equal to M.

[0009] In one embodiment, the step of determining the plurality of minutiae points present in the input fingerprint image may include extracting the plurality of minutiae points from the input fingerprint image using a feature extraction algorithm, wherein each of the plurality of image blocks is centered around a different minutiae point among the plurality of minutiae points.

[0010] In one aspect of the present disclosure, a fingerprint recognition method is provided, the fingerprint recognition method comprising: receiving an input fingerprint image; dividing the input fingerprint image into M image blocks, wherein M is an integer greater than 1; calculating scores of N image blocks among the M image blocks by inputting consecutive image blocks among the M image blocks into a neural network for fingerprint recognition until the scores among the N image blocks meet a preset condition, wherein the score is a total score and N is an integer less than or equal to M; and suspending the calculation of the scores of the remaining image blocks among the M image blocks, and identifying the legitimacy of the fingerprint in the input fingerprint image.

[0011] In one embodiment, the preset conditions include a first condition, wherein the first condition indicates that the sum of the scores of the N image blocks is greater than or equal to the product of M and a preset threshold score, wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image includes: when the scores of the N image blocks meet the first condition, identifying the fingerprint in the input fingerprint image as true.

[0012] In one embodiment, the preset condition includes a second condition, wherein the second condition indicates that the sum of the scores of the N image blocks and the sum of the first value is less than the product of M and a preset threshold score, and the first value is equal to the product of the difference between M and N and the maximum score that the neural network can output, wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image includes: when the scores of the N image blocks meet the second condition, identifying the fingerprint in the input fingerprint image as false.

[0013] In one embodiment, the neural network includes a depthwise separable convolutional network, and the number of output channels of the point convolution of the depthwise separable convolutional network is consistent with the multiplication and accumulation operations supported by the fingerprint recognition device that executes the fingerprint recognition method.

[0014] In one embodiment, a depthwise separable convolutional network includes a separable convolutional structure having a predetermined number of output channels, wherein the separable convolutional structure includes a depthwise convolutional layer and a pointwise convolutional layer, wherein the convolution operation of the depthwise convolutional layer is accelerated using a convolutional hardware accelerator of a fingerprint recognition device, and the convolution operation of the pointwise convolutional layer is accelerated using a multiplication and accumulation hardware accelerator of the fingerprint recognition device.

[0015] In one embodiment, the step of obtaining the M image blocks by dividing the input fingerprint image includes: obtaining multiple image blocks by dividing the input fingerprint image; determining the central coordinates of the input fingerprint image; and determining a first number of image blocks closest to the central coordinates among the multiple image blocks and a second number of image blocks farthest from the central coordinates among the multiple image blocks as the M image blocks, wherein the sum of the first number and the second number is equal to M.

[0016] In one embodiment, the step of obtaining the plurality of image blocks by dividing the input fingerprint image comprises: extracting a plurality of detail points from the input fingerprint image using a feature extraction algorithm, wherein each of the plurality of image blocks is centered on a different detail point among the plurality of detail points.

[0017] In another aspect, a fingerprint recognition device is provided, comprising: a receiving module configured to receive an input fingerprint image; an obtaining module configured to obtain M image blocks by dividing the input fingerprint image, wherein M is an integer greater than 1; an identifying module configured to calculate scores of N image blocks among the M image blocks by inputting consecutive image blocks among the M image blocks into a neural network for fingerprint recognition until the scores of the N image blocks meet a preset condition, wherein the score is a total score and N is an integer less than or equal to M; suspending the calculation of the scores of the remaining image blocks among the M image blocks, and identifying the legitimacy of the fingerprint in the input fingerprint image, wherein N is an integer less than or equal to M.

[0018] In one embodiment, the preset conditions include a first condition, wherein the first condition indicates that the sum of the scores of the N image blocks is greater than or equal to the product of M and a preset threshold score, wherein the recognition module is configured to: when the scores of the N image blocks meet the first condition, identify the legitimacy of the fingerprint in the input fingerprint image as true.

[0019] In one embodiment, the preset condition includes a second condition, wherein the second condition indicates that the sum of the scores of the N image blocks and the sum of the first value is less than the product of M and a preset threshold score, and the first value is equal to the product of the difference between M and N and the maximum score that the neural network can output, wherein the recognition module is configured to: when the scores of the N image blocks meet the second condition, identify the legitimacy of the fingerprint in the input fingerprint image as false.

[0020] In one embodiment, the neural network includes a depthwise separable convolutional network, and the number of output channels of the point convolution of the depthwise separable convolutional network is consistent with the multiplication and accumulation operations supported by the fingerprint recognition device.

[0021] In one embodiment, a depthwise separable convolutional network includes a separable convolutional structure having a predetermined number of output channels, wherein the separable convolutional structure includes a depthwise convolutional layer and a pointwise convolutional layer, wherein the convolution operation of the depthwise convolutional layer is accelerated using a convolutional hardware accelerator of the fingerprint recognition device, and the convolution operation of the pointwise convolutional layer is accelerated using a multiplication and accumulation hardware accelerator of the fingerprint recognition device.

[0022] In one embodiment, the acquisition module is configured to: obtain multiple image blocks by dividing the input fingerprint image; determine the central coordinates of the input fingerprint image; and determine a first number of image blocks closest to the central coordinates among the multiple image blocks and a second number of image blocks farthest from the central coordinates among the multiple image blocks as the M image blocks, wherein the sum of the first number and the second number is equal to M.

[0023] In one embodiment, the obtaining module is configured to extract a plurality of minutiae points from the input fingerprint image using a feature extraction algorithm, wherein each image block in the plurality of image blocks is centered around a different minutiae point in the plurality of minutiae points.

[0024] In one embodiment, the method of identifying a module is implemented by a computer-readable storage medium storing a computer program when executed by a processor.

[0025] In yet another aspect, a fingerprint recognition device may include: a processor; and a computer-readable storage medium storing a computer program, wherein when the computer program is executed by the processor, a fingerprint recognition method of a recognition module is implemented.

[0026] According to the fingerprint recognition method of the example embodiment of the present disclosure, while calculating the score of the image block, it can determine whether to terminate the calculation of the scores of the remaining image blocks in advance and whether to finally identify the authenticity of the input fingerprint image based on whether the scores of the image blocks that have been calculated meet preset conditions. Therefore, it is feasible to finally identify the authenticity of the input fingerprint image without calculating the scores of all the image blocks, thereby reducing delays and ensuring real-time performance.

[0027] According to the example embodiments of the present disclosure, since the number of output channels of the point convolution of the deep separable convolutional network is consistent with the multiplication and accumulation operations supported by the fingerprint recognition device that executes the fingerprint recognition method, the operation of the deep separable convolutional network can be executed more quickly with the hardware support of the fingerprint recognition device, thereby reducing latency.

[0028] According to an example embodiment of the present disclosure, since the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolution network is accelerated using the convolution hardware accelerator of the fingerprint recognition device, and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolution network is accelerated using the multiplication and accumulation hardware accelerator of the fingerprint recognition device, the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolution network and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolution network can be executed more quickly with the hardware support of the fingerprint recognition device, thereby reducing latency.

[0029] According to an exemplary embodiment of the present disclosure, since the calculation of scores can be terminated in advance based on the first condition without calculating scores of all image blocks and the fingerprint in the input fingerprint image can be recognized as true, delay can be reduced and real-time performance can be ensured.

[0030] According to an exemplary embodiment of the present disclosure, since the calculation of scores can be terminated in advance based on the second condition without calculating scores of all image blocks and the fingerprint in the input fingerprint image can be identified as false, delay can be reduced and real-time performance can be ensured.

[0031] According to an exemplary embodiment of the present disclosure, compared to determining all of a plurality of image blocks as image blocks applicable to a neural network for fingerprint recognition, by determining a first number of image blocks among a plurality of image blocks that are closest to the center coordinates and a second number of image blocks among a plurality of image blocks that are farthest from the center coordinates as M image blocks applicable to the neural network for fingerprint recognition, on the one hand, the sufficiency of information of image blocks applicable to the neural network for fingerprint recognition can be ensured (i.e., the accuracy of the neural network for fingerprint recognition is not affected), and on the other hand, the time required for the neural network for fingerprint recognition to process image blocks can be reduced (i.e., the delay is reduced and real-time performance is ensured). BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The embodiments will be more clearly understood through the following detailed description in conjunction with the accompanying drawings, in which:

[0033] Figure 1 A flow chart of a fingerprint recognition method according to an exemplary embodiment of the present disclosure is shown.

[0034] Figure 2 Schematic diagram showing an input fingerprint image corresponding to a real fingerprint, an input fingerprint image corresponding to a fake fingerprint simulated by gelatin, an input fingerprint image corresponding to a fake fingerprint simulated by clay, and an input fingerprint image corresponding to a fake fingerprint simulated by Play-Doh.

[0035] Figure 3 A flowchart of a method for obtaining M image blocks according to an example embodiment of the present disclosure is shown.

[0036] Figure 4 A schematic diagram of a depthwise separable convolutional network according to an example embodiment of the present disclosure is shown.

[0037] Figure 5 An example of hardware acceleration is shown.

[0038] Figure 6 Schematic diagram showing point convolution hardware acceleration using MAC18.

[0039] Figure 7 A block diagram of a fingerprint recognition system according to an example embodiment of the present disclosure is shown.

[0040] Figure 8 A block diagram of an anti-spoofing module is shown according to an example embodiment of the present disclosure.

[0041] Fig. 9 A block diagram of an electronic device according to an example embodiment of the present disclosure is shown.

[0042] Fig.10 A block diagram of a fingerprint recognition device according to an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0043] The following detailed description is provided to help the reader gain a comprehensive understanding of the methods, devices and / or systems described herein. However, various changes, modifications and equivalents of the methods, devices and / or systems described herein will be clear from the description and can be learned through the practice of the disclosed embodiments. For example, the order of operations described herein is only an example and is not limited to those orders set forth herein, but can be changed as will be clear from the description, except for operations that must occur in a particular order. In addition, for greater clarity or conciseness, descriptions of features known in the art may be omitted.

[0044] The features described herein can be implemented in different forms and should not be interpreted as being limited to the examples described herein. On the contrary, the examples described herein have been provided to illustrate only some of the many possible ways to implement the methods, devices and / or systems described herein, and many possible ways will be clear from the description.

[0045] As used herein, the term "and / or" includes any one of the associated items and any combination of any two or more.

[0046] Although terms such as "first," "second," and "third" may be used herein to describe various members, components, regions, layers, or portions, these members, components, regions, layers, or portions should not be limited by these terms. Instead, these terms are only used to distinguish one member, component, region, layer, or portion from another member, component, region, layer, or portion. Thus, a first member, first component, first region, first layer, or first portion referred to in an example described herein may also be referred to as a second member, second component, second region, second layer, or second portion in another example.

[0047] In the specification, when an element (such as a layer, a region, or a substrate) is described as being “on”, “connected to”, or “coupled to” another element, the element may be directly disposed “on”, “connected to”, or “coupled to” another element, or one or more other elements disposed therebetween may exist. Conversely, when an element is described as being “directly on”, “directly connected to”, or “directly coupled to” another element, other elements disposed therebetween may not exist.

[0048] The terms used herein are only used to describe various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. The terms "comprise", "include" and "have" indicate the presence of the described features, quantities, operations, components, elements and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements and / or combinations thereof.

[0049] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by ordinary persons in the field to which the present disclosure belongs after understanding the present disclosure. Unless explicitly defined as such herein, terms (such as those defined in general dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant field and the present disclosure, and should not be interpreted in an idealized or overly formal manner.

[0050] Furthermore, in the description, when it is deemed that a detailed description of a well-known related construction or function would cause vague interpretation of the present disclosure, such detailed description may be omitted.

[0051] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. The same components in the drawings may use the same reference numerals, and their redundant descriptions may be omitted. However, the embodiments may be implemented in various forms and are not limited to the examples described herein.

[0052] Similar to many security technologies, fingerprint recognition systems can be vulnerable to attacks. For example, in the context of fingerprint recognition systems, a common attack method involves collecting user fingerprints, simulating and generating forged input fingerprint images using materials such as gelatin, wood glue, or clay, and using the forged input finger images to attack the fingerprint recognition system. Figure 1 A flowchart of a fingerprint recognition method according to an exemplary embodiment of the present disclosure is shown.

[0053] Reference Figure 1 , in operation S110, an input fingerprint image may be received.

[0054] like Figure 2 As shown in , an input fingerprint image corresponding to a real fingerprint, an input fingerprint image corresponding to a fingerprint simulated by gelatin, an input fingerprint image corresponding to a fingerprint simulated by clay, and an input fingerprint image corresponding to a fingerprint simulated by Play-Doh are shown. However, the input fingerprint image of the present disclosure is not limited thereto, and may include an image corresponding to any forged fingerprint.

[0055] In operation S120, M image blocks may be obtained. The M image blocks may be obtained by dividing the input fingerprint image, where M is an integer greater than 1.

[0056] The M image blocks may be obtained by dividing the input fingerprint image (e.g., by dividing the image into squares of equal size or by dividing the total size of the image by a desired number of segments). However, the embodiment is not limited thereto, and the input fingerprint may be divided in any manner. Each of the M image blocks may include at least a portion of the information of the input fingerprint image.

[0057] Fingerprint images can capture real information. For example, the information can include global features and local features. Global features can include information about the core and delta of the fingerprint, as well as information about loops and spirals. Local features can include information about ridge features called minutiae. For example, minutiae features can be related to areas where ridges terminate and bifurcate, bridges in ridges, or islands in ridges. These examples are non-limiting and other minutiae features can be used.

[0058] In one example, a plurality of minutiae points may be extracted from an input fingerprint image by using a feature extraction algorithm, and then a plurality of image blocks having a predetermined size (e.g., a size of K×L, where K and L are the same or different positive integers) may be obtained. Each of the plurality of image blocks may be centered on a corresponding minutiae point in the plurality of minutiae points. However, the above example is only non-limiting, and the image blocks may be obtained in any manner.

[0059] Will combine Figure 3 A method of obtaining M image blocks according to an exemplary embodiment of the present disclosure is described in more detail.

[0060] In step S130, the scores of N image blocks among the M image blocks may be determined. For example, the scores of N image blocks among the M image blocks are calculated by inputting consecutive image blocks among the M image blocks into a neural network for fingerprint recognition until the scores of the N image blocks meet a preset condition, wherein the score is a total score and N is an integer less than or equal to M.

[0061] In one example, the neural network for fingerprint recognition may include a convolutional neural network (CNN). However, the present disclosure is not limited thereto, and the neural network for fingerprint recognition may include any other neural network for fingerprint recognition.

[0062] In one embodiment, the neural network may include a depthwise separable convolutional network, and the number of output channels of the point convolution of the depthwise separable convolutional network may be consistent with the multiplication and accumulation operations supported by the fingerprint recognition device that executes the fingerprint recognition method.

[0063] In one embodiment, by forcing consistency between the number of output channels of the point convolution of the deep separable convolutional network and the multiplication and accumulation operations supported by the fingerprint recognition device that performs the fingerprint recognition method, the operation of the deep separable convolutional network can be performed more quickly with the hardware support of the fingerprint recognition device, and the latency can be reduced. Figure 4 A depthwise separable convolutional network according to an example embodiment of the present disclosure is described in more detail.

[0064] In addition, the depthwise separable convolution network may include a separable convolution structure having a predetermined number of output channels. The separable convolution structure may include a depthwise convolution layer and a pointwise convolution layer. The convolution operation of the depthwise convolution layer may be accelerated using a convolution hardware accelerator of the fingerprint recognition device, and the convolution operation of the pointwise convolution layer may be accelerated using a multiplication and accumulation hardware accelerator of the fingerprint recognition device. Figure 5 and Figure 6 Hardware acceleration is described in more detail.

[0065] In one embodiment, since the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolutional network can be accelerated using the convolution hardware accelerator of the fingerprint recognition device, and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolutional network can be accelerated using the multiplication and accumulation hardware accelerator of the fingerprint recognition device, the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolutional network and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolutional network can be executed more quickly with the hardware support of the fingerprint recognition device, and the latency can be reduced.

[0066] In addition, the score of each image block may indicate the degree to which the image block is identified as corresponding to a true fingerprint. For example, a higher score of an image block may indicate a greater possibility that the image block is identified as corresponding to a true fingerprint.

[0067] In operation S140, the fingerprint in the input fingerprint image may be identified as true or false. For example, in response to the scores of N image blocks among the M image blocks having been calculated and the scores of the remaining image blocks other than the N image blocks among the M image blocks having not been calculated, when the scores of the N image blocks meet a preset condition, the calculation of the scores of the remaining image blocks is stopped, and the legitimacy of the fingerprint in the input fingerprint image may be identified, where N is an integer less than or equal to M.

[0068] For example, by reducing the number of scores calculated in identifying the legitimacy of an input fingerprint image, the delay in calculation can be reduced. It should be understood that the identification of the legitimacy of an input fingerprint image can include identifying that the input fingerprint image is legitimate (i.e., an image of a real fingerprint) or identifying that the input fingerprint image is illegal (e.g., an image of a false or forged fingerprint). The fingerprint recognition method according to the example embodiment of the present disclosure can determine whether to terminate the calculation of the scores of the remaining image blocks before calculating the scores of all M image blocks and whether to identify the legitimacy of the input fingerprint image based on whether the scores of the image blocks that have been calculated meet one or more preset conditions. Therefore, it is feasible to ultimately identify the authenticity of the input fingerprint image without calculating the scores of all the image blocks obtained, which can reduce delays and improve real-time performance.

[0069] In one embodiment, the preset condition may include a first condition, wherein the first condition indicates that the sum of the scores of the N image blocks is greater than or equal to the product of M and a preset threshold score. When the scores of the N image blocks meet the first condition, the fingerprint in the input fingerprint image may be identified as true.

[0070] More specifically, when the scores of N image blocks meet the first condition, no matter what the (unknown) scores of the remaining image blocks other than the N image blocks are among the M image blocks, the average score of the M image blocks will be greater than the preset threshold score. Therefore, the fingerprint in the input fingerprint image can be directly returned as true.

[0071] In one embodiment, the reduction in latency and the improvement in real-time performance may be related to the number of scores that are not calculated for the remaining image blocks according to the first condition that the fingerprint in the input fingerprint image can be identified as true.

[0072] In another embodiment, the preset condition may include a second condition. That is, the preset condition may include one or more conditions. The second condition may indicate that the sum of the scores of the N image blocks and the sum of the first value is less than the product of M and the preset threshold score, and the first value is equal to the product between the difference between M and N and the maximum score that the neural network can output. When the scores of the N image blocks meet the second condition, the fingerprint in the input fingerprint image can be identified as false.

[0073] More specifically, when the scores of N image blocks meet the second condition, no matter what the scores of the remaining image blocks other than the N image blocks are among the M image blocks (for example, even if the score of each remaining image block is the maximum score that the neural network can output), the average score of the M image blocks may not exceed the preset threshold, and the fingerprint in the input fingerprint image may be directly returned as false.

[0074] In one embodiment, the reduction in latency and the improvement in real-time performance may be related to the number of scores that are not calculated for the remaining image blocks according to the second condition that the fingerprint in the input fingerprint image may be identified as false.

[0075] The embodiments described herein may be combined. That is, the preset condition may include the first condition and / or the second condition.

[0076] According to the exemplary embodiment of the present disclosure, assuming that scores have been calculated for a certain number of image blocks, the evaluation of the condition can be used to confirm whether the calculation can be terminated in advance before calculating the scores of M image blocks, which can reduce the delay of the calculation. Assuming that there are a total of M image blocks that can be calculated, the i-th (1<=i<=M) image block is currently calculated, and the score of the i-th image block is score i , the maximum score output by the neural network is score MAX , the threshold is T, and the judgment condition can be written as follows:

[0077] ∑score i ≥M*T (1)

[0078] ∑score i +(Mi)*scoreMAX <M*T (2)

[0079] When evaluating condition (1) and calculating the score of each consecutive image block, once condition (1) is true, the average score of the M image blocks will be greater than the threshold value regardless of the score of any subsequent image block, so the fingerprint can be determined to be true without considering any subsequent image block. In other words, when evaluating condition (1) and calculating the score of each consecutive image block, once condition (1) is true, condition (1) will not become false, so the fingerprint can be determined to be true without considering any subsequent image block. Similarly, if condition (2) is true, the average score of the M image blocks will be less than the threshold value, so the fingerprint can be determined to be false without considering any subsequent image block. In other words, when evaluating condition (2) and calculating the score of each consecutive image block, once condition (2) becomes true, condition (2) will not become false, so the fingerprint can be determined to be false without considering any subsequent image block. This can reduce the running time of the fingerprint recognition device without affecting the accuracy. Figure 3 A flowchart of a method for obtaining M image blocks according to an example embodiment of the present disclosure is shown.

[0080] Reference Figure 3 In step S310, a plurality of image blocks may be obtained by dividing the input fingerprint image.

[0081] For example, the number of the plurality of image blocks may be greater than or equal to M. The plurality of image blocks may be obtained by dividing the input fingerprint image. Each of the plurality of image blocks may include a portion of the information of the input fingerprint image.

[0082] In one example, a plurality of minutiae points may be extracted from an input fingerprint image. The plurality of minutiae points may be extracted from the input fingerprint image by using a feature extraction algorithm. An image block may be obtained from the input fingerprint image, wherein the image block is centered on a corresponding minutiae point among the plurality of minutiae points. For example, the image block may have a predetermined size (e.g., a size of K×L, wherein K and L may be the same or different positive integers) (i.e., a rectangular block). However, the above examples are merely examples, and the plurality of image blocks may be obtained by dividing the input fingerprint image in any manner.

[0083] In step S320, the center coordinates of the input fingerprint image may be determined.

[0084] The center coordinates of the input fingerprint image may be coordinates corresponding to the center of the input fingerprint image. The coordinates may be coordinates in any coordinate system. For example, the center coordinates of the input fingerprint image may correspond to pixels at the center of the input fingerprint image.

[0085] In step S330, a plurality of image blocks M may be determined. The plurality of image blocks M are determined as a first number of image blocks closest to the center coordinates among the plurality of image blocks and a second number of image blocks farthest from the center coordinates among the plurality of image blocks, where the sum of the first number and the second number is equal to M. For example, a third number of image blocks between the first number of image blocks and the second number of image blocks may be excluded.

[0086] According to an exemplary embodiment of the present disclosure, by determining multiple image blocks M as image blocks applicable to a neural network for fingerprint recognition, the sufficiency of information of image blocks applicable to the neural network for fingerprint recognition can be ensured, and the time required for the neural network for fingerprint recognition to process image blocks can be reduced, compared to determining all of the multiple image blocks as image blocks applicable to the neural network for fingerprint recognition. For example, the information contained in the multiple image blocks M may not significantly affect the accuracy of the fingerprint recognition neural network. In addition, by selecting less than all of the multiple image blocks as image blocks applicable to the fingerprint recognition neural network, latency can be reduced and real-time performance can be improved.

[0087] Although Figure 1 and Figure 3 An example method is shown, but the embodiment is not limited thereto. For example, the operations performed in the steps can be modified or performed in a different order.

[0088] For example, in some embodiments, the fingerprint recognition method may include: receiving an input fingerprint image at step S110, determining a plurality of minutiae points appearing in the input fingerprint image at step S120, wherein the number of the plurality of minutiae points is M, and identifying the legitimacy of the fingerprint in the input fingerprint image at steps S130 and S140. The step of determining the plurality of minutiae points appearing in the input fingerprint image may be similar to step S120, except that the M image blocks may be determined as required by step S130 of calculating the score of the next image block. At step S140, the legitimacy of the fingerprint in the input fingerprint image may be identified by performing an iterative method, the iterative method including: obtaining an image block (e.g., the next image block) from the input fingerprint image at step S310, wherein the image block corresponds to a corresponding minutiae point among the plurality of minutiae points, calculating the score of the image block by inputting the image block into a neural network for fingerprint recognition at step S140, adding the score of the image block to the total score of the previously obtained image block to obtain the added score, and identifying the legitimacy of the fingerprint in the input fingerprint image when the added score satisfies a preset condition.

[0089] In one embodiment, the preset conditions may include a first condition, wherein the first condition indicates that the total score is greater than or equal to the product of M and a preset threshold score, and wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image may include: when the total score satisfies the first condition, identifying the fingerprint in the input fingerprint image as true.

[0090] In one embodiment, the preset condition may include a second condition, wherein the second condition indicates that the sum of the total score and the first value is less than the product of M and a preset threshold, and the first value is equal to the product of the difference between M and N and the maximum score that the neural network can output, and wherein the step of identifying the legitimacy of the fingerprint in the input fingerprint image may include: when the total score satisfies the second condition, identifying the fingerprint in the input fingerprint image as false.

[0091] In one embodiment, the fingerprint recognition method may include: determining the center coordinates of the input fingerprint image in step S320; and determining a first number of image blocks corresponding to a first number of detail points closest to the center coordinates and a second number of image blocks corresponding to a second number of detail points farthest from the center coordinates among a plurality of image blocks as M image blocks in step S330, wherein the sum of the first number and the second number is equal to M.

[0092] In one embodiment, step S120 may further include: determining the plurality of detail points appearing in the input fingerprint image, and the step of determining the plurality of detail points appearing in the input fingerprint image may include: extracting the plurality of detail points from the input fingerprint image using a feature extraction algorithm, wherein each of the plurality of image blocks is centered on a different detail point among the plurality of detail points. For example, M image blocks may be obtained as needed to perform step S130, and delay may be reduced and real-time performance may be improved.

[0093] Figure 4 A schematic diagram showing a depthwise separable convolutional network according to an example embodiment of the present disclosure.

[0094] Reference Figure 4 , the depthwise separable convolutional network 400 may include a convolutional layer, a separable convolutional layer, a maximum pooling layer, a fully connected layer, and an output layer.

[0095] Although, Figure 4 An example number and a specific order of convolutional layers, separable convolutional layers, maximum pooling layers, fully connected layers, and output layers are shown, but the number and order of convolutional layers, separable convolutional layers, maximum pooling layers, fully connected layers, and output layers of the present disclosure are not limited thereto, and the depthwise separable convolutional network 400 can be variously configured.

[0096] also, Figure 4An example size of data processed by the convolutional layer, separable convolutional layer, maximum pooling layer, fully connected layer, and output layer is shown (e.g., a 24×24×8 matrix), but the size of the data processed by the convolutional layer, separable convolutional layer, maximum pooling layer, fully connected layer, and output layer of the present disclosure is not limited thereto, and the data processed at each layer can be of any other size.

[0097] In one example, the depthwise separable convolutional network 400 may use a separable convolutional structure with a certain number of output channels (e.g., 18 output channels or 28 output channels).

[0098] Figure 5 An example of hardware acceleration is shown.

[0099] exist Figure 5 In the example of MAC18 hardware acceleration, the fingerprint recognition device can support multiplication and accumulation (MAC18). MAC18 refers to the parallel calculation of the point multiplication and addition of 18 groups of numbers (18 input / weight pairs), which can be used to calculate the output of the fully connected layer of the convolutional neural network. For example, the number of output channels can be consistent with the number of groups in the multiplication and accumulation. In the example of MAC18 hardware acceleration, the number of output channels can be 18.

[0100] In one example, the fingerprint recognition device may support hardware acceleration of any other type of MAC operation (e.g., MAC28, MAC216, etc.). In another example, the fingerprint recognition device may support hardware acceleration of 3×3 convolution operations and hardware acceleration of the fully connected (FC) layer, and the FC layer hardware acceleration method includes MAC18, MAC28, and MAC216, etc.

[0101] Figure 6 Schematic diagram showing point convolution hardware acceleration using MAC18.

[0102] Reference Figure 6 The convolution operation of the point-by-point convolution layer can be accelerated using the multiplication and accumulation hardware accelerator (HWA) of the fingerprint recognition device. For example, through the multiplication and accumulation hardware accelerator, 24×24×18 data can be processed into 1×1×18 data, and the processing time can be reduced.

[0103] Figure 7 A block diagram of a fingerprint recognition system according to an example embodiment of the present disclosure is shown.

[0104] Reference Figure 7 , the fingerprint recognition system 700 may include an image preprocessor 705 , a feature extractor 710 , an anti-spoofing module 715 and a matching module 720 .

[0105] The image preprocessor 705 may receive an input fingerprint image, obtain a grayscale stretched image by preprocessing the fingerprint image, and send the grayscale stretched image to the feature extractor 710 .

[0106] The feature extractor 710 may obtain information (eg, coordinates) of a plurality of detail points by performing feature extraction on the preprocessed image, and send the information of the plurality of detail points to the anti-spoofing module 715 .

[0107] The anti-spoofing module 715 can use an image block centered on M detail points among the multiple detail points to identify whether the input fingerprint image is true or false. If the input fingerprint image is true, the anti-spoofing module 715 sends the input fingerprint image to the matching module 720 for matching. If it is false, the input fingerprint image is rejected. For example, the anti-spoofing module 715 can be implemented by a neural network for identifying whether the input fingerprint image is true or false. Figure 8 The anti-spoofing module 715 is described in more detail.

[0108] Matching module 720 can match the image determined as authentic by anti-spoofing module 715 with the fingerprint data of the real user registered in the database. This matching can be performed as a check on the authentication of anti-spoofing module 715.

[0109] Figure 8 A block diagram of an anti-spoofing module is shown according to an example embodiment of the present disclosure.

[0110] Reference Figure 8 , the anti-spoofing module 800 may include a detail point filter and a convolutional neural network (CNN) 810.

[0111] The detail point filter 805 may filter out M detail points from a plurality of detail points (eg, the first detail point to the Kth detail point), so that M image blocks corresponding to the M detail points may be obtained.

[0112] The convolutional neural network 810 may be used to receive M image blocks and output M scores based on the M image blocks. The M scores may be used to obtain a total score. For example, the M scores may be added together to obtain a total score. The total score may be used to determine whether the input fingerprint image is true or false.

[0113] In another embodiment, the convolutional neural network 810 may be used to perform a fingerprint recognition method.An example fingerprint recognition method has been described in the present application; however, the embodiments are not limited thereto.

[0114] Fig. 9 A block diagram of an electronic device according to an example embodiment of the present disclosure is shown.

[0115] Reference Fig. 9According to an embodiment of the present disclosure, the electronic device 900 may include a processor 910 and a memory 920. Here, the memory 920 may store a computer program, wherein the computer program may implement reference to Figures 1 to 8 For simplicity, refer to Figures 1 to 8 The fingerprint recognition method is performed by the processor 910, and its repeated description may be omitted.

[0116] Fig.10 A block diagram of a fingerprint recognition device according to an exemplary embodiment of the present disclosure is shown.

[0117] Reference Fig.10 According to an embodiment of the present disclosure, the fingerprint recognition device 1000 may include a receiving module 1010 , an obtaining module 1020 , and a recognition module 1030 .

[0118] The receiving module 1010 may receive an input fingerprint image. The obtaining module may obtain M image blocks by dividing the input fingerprint image, where M is an integer greater than 1. The identifying module 1030 may calculate the score of each of the M image blocks by inputting the M image blocks into a neural network for fingerprint recognition. The identifying module 1030 may calculate the scores of N image blocks among the M image blocks by inputting continuous image blocks among the M image blocks into a neural network for fingerprint recognition until the scores of the continuous image blocks meet a preset condition. These continuous image blocks may be N image blocks. The identifying module 1030 may suspend calculating the scores of the remaining image blocks and identify the fingerprint in the input fingerprint image as true or false, where N is an integer less than or equal to M.

[0119] The steps of receiving an input fingerprint image, obtaining M image blocks, and identifying the fingerprint in the input fingerprint image have been described elsewhere herein, and therefore, for the sake of brevity, the description of the individual processes respectively performed by the receiving module 1010, the obtaining module 1020, and the identifying module 1030 may be omitted.

[0120] In addition, the method according to the exemplary embodiment of the present disclosure can be implemented as a computer program in a computer-readable recording medium. Those skilled in the art can implement the computer program according to the description of the above method. When the computer program is executed in a computer, the fingerprint recognition method of the present disclosure is implemented.

[0121] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may be provided, on which a computer program is stored, and when the computer program is executed by a processor, the processor implements any of the methods disclosed in the present application. For example, when the computer program is executed by the processor, the processor performs the following steps: receiving an input fingerprint image; obtaining M image blocks by dividing the input fingerprint image, where M is an integer greater than 1; calculating the score of each image block in the M image blocks by inputting the M image blocks into a neural network for fingerprint recognition; in response to the scores of N image blocks in the M image blocks having been calculated and the scores of the remaining image blocks in the M image blocks except the N image blocks have not been calculated, when the scores of the N image blocks meet a preset condition, suspending the calculation of the scores of the remaining image blocks, and identifying the fingerprint in the input fingerprint image as true or false, where N is an integer less than or equal to M.

[0122] According to the fingerprint recognition method of the example embodiment of the present disclosure, while calculating the score of the image block, it can determine whether to terminate the calculation of the scores of the remaining image blocks in advance and whether to finally identify the authenticity of the input fingerprint image based on whether the scores of the image blocks that have been calculated meet preset conditions. Therefore, it is feasible to finally identify the authenticity of the input fingerprint image without calculating the scores of all the image blocks, thereby reducing delays and ensuring real-time performance.

[0123] According to the fingerprint recognition method of the exemplary embodiment of the present disclosure, since the number of output channels of the point convolution of the deep separable convolutional network is consistent with the multiplication and accumulation operations supported by the fingerprint recognition device that executes the fingerprint recognition method, the operation of the deep separable convolutional network can be executed more quickly with the hardware support of the fingerprint recognition device, thereby reducing latency.

[0124] According to the fingerprint recognition method of the exemplary embodiment of the present disclosure, since the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolution network is accelerated using the convolution hardware accelerator of the fingerprint recognition device, and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolution network is accelerated using the multiplication and accumulation hardware accelerator of the fingerprint recognition device, the convolution operation of the depth-by-depth convolution layer in the depth-wise separable convolution network and the convolution operation of the point-by-point convolution layer in the depth-wise separable convolution network can be executed more quickly with the hardware support of the fingerprint recognition device, thereby reducing latency.

[0125] According to the fingerprint recognition method of the exemplary embodiment of the present disclosure, since the score calculation can be terminated in advance based on the first condition without calculating the scores of all image blocks and the fingerprint in the input fingerprint image can be recognized as true, the delay can be reduced and the real-time performance can be ensured.

[0126] According to an exemplary embodiment of the present disclosure, since the calculation of scores can be terminated in advance based on the second condition without calculating scores of all image blocks and the fingerprint in the input fingerprint image can be identified as false, delay can be reduced and real-time performance can be ensured.

[0127] According to the fingerprint recognition method of the exemplary embodiment of the present disclosure, compared to determining all of the multiple image blocks as image blocks applicable to the neural network for fingerprint recognition, by determining a first number of image blocks among the multiple image blocks that are closest to the center coordinates and a second number of image blocks among the multiple image blocks that are farthest from the center coordinates as M image blocks applicable to the neural network for fingerprint recognition, on the one hand, the sufficiency of information of the image blocks applicable to the neural network for fingerprint recognition can be ensured (i.e., the accuracy of the neural network for fingerprint recognition is not affected), and on the other hand, the time required for the neural network for fingerprint recognition to process the image blocks can be reduced (i.e., the delay is reduced and real-time performance is ensured).

[0128] In addition, it should be understood that the various units in the device according to the exemplary embodiments of the present disclosure can be implemented as hardware components and / or software components. Those skilled in the art can, for example, use a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC) to implement the various units according to the processing performed by the defined various units.

[0129] Although the present disclosure includes specific examples, it will be clear to those of ordinary skill in the art that various changes in form and detail may be made in these examples without departing from the spirit and scope of the claims and their equivalents. The examples described herein should be considered to be descriptive only and not for limiting purposes. The description of features or aspects in each example should be considered to be applicable to similar features or aspects in other examples. If the described techniques are performed in a different order, and / or if the components in the described systems, architectures, devices, or circuits are combined in different ways, and / or replaced or supplemented by other components or their equivalents, suitable results can be achieved. Therefore, the scope of the disclosure is not limited by specific embodiments, but by the claims and their equivalents, and all changes within the scope of the claims and their equivalents should be interpreted as included in the disclosure.

Claims

1. A fingerprint recognition method, the fingerprint recognition method include: Receiving an input fingerprint image; determining a plurality of minutiae points present in the input fingerprint image, wherein the number of the plurality of minutiae points is M; and The legitimacy of the fingerprint in the input fingerprint image is identified by performing an iterative method, the iterative method comprising: obtaining an image block from an input fingerprint image, wherein the image block corresponds to a corresponding minutiae point among the plurality of minutiae points; The score of the image patch is calculated by inputting the image patch into the neural network for fingerprint recognition; adding the score of the image patch to the total score of the previously obtained image patches to obtain an added score; and When the added scores meet the preset conditions, the legitimacy of the fingerprint in the input fingerprint image is identified.

2. The fingerprint recognition method according to claim 1, in, The preset conditions include a first condition, The first condition indicates that the total score is greater than or equal to the product of M and a preset threshold score, and The step of identifying the legitimacy of the fingerprint in the input fingerprint image includes: When the total score satisfies the first condition, the fingerprint in the input fingerprint image is recognized as true.

3. The fingerprint recognition method according to claim 1, in, The preset conditions include a second condition, wherein the second condition indicates that the sum of the total score and the first value is less than a product of M and a preset threshold, and the first value is equal to a product of a difference between M and N and a maximum score that the neural network can output, wherein N corresponds to a number obtained by adding one to the number of previously obtained image blocks, and The step of identifying the legitimacy of the fingerprint in the input fingerprint image includes: When the total score satisfies the second condition, the fingerprint in the input fingerprint image is identified as false.

4. The fingerprint recognition method according to claim 1, further comprising: include: Determine the center coordinates of the input fingerprint image; as well as A first number of image blocks corresponding to a first number of detail points closest to the center coordinates and a second number of image blocks corresponding to a second number of detail points farthest from the center coordinates among the multiple image blocks are determined as M image blocks, wherein the sum of the first number and the second number is equal to M.

5. The fingerprint recognition method according to claim 4, in, The step of determining the plurality of minutiae points present in the input fingerprint image comprises: extracting the plurality of minutiae points from the input fingerprint image using a feature extraction algorithm, Each of the multiple image blocks is centered on a different detail point among the multiple detail points.

6. A fingerprint recognition method, the fingerprint recognition method include: Receiving an input fingerprint image; Divide the input fingerprint image into M image blocks, where M is an integer greater than 1; Calculating scores of N image blocks among the M image blocks by inputting consecutive image blocks among the M image blocks into a neural network for fingerprint recognition until the scores of the N image blocks meet a preset condition, wherein the score is a total score and N is an integer less than or equal to M; and The calculation of the scores of the remaining image blocks among the M image blocks is stopped, and the legitimacy of the fingerprint in the input fingerprint image is identified.

7. The fingerprint recognition method according to claim 6, in, The preset conditions include a first condition, The first condition indicates that the sum of the scores of the N image blocks is greater than or equal to the product of M and a preset threshold score. The step of identifying the legitimacy of the fingerprint in the input fingerprint image includes: When the scores of the N image blocks satisfy the first condition, the fingerprint in the input fingerprint image is recognized as true.

8. The fingerprint recognition method according to claim 6, in, The preset conditions include a second condition, The second condition indicates that the sum of the scores of the N image blocks and the first value is less than the product of M and a preset threshold score, and the first value is equal to the product of the difference between M and N and the maximum score that the neural network can output, The step of identifying the legitimacy of the fingerprint recognition in the input fingerprint image includes: When the scores of the N image blocks satisfy the second condition, the fingerprint in the input fingerprint image is identified as false.

9. The fingerprint recognition method according to claim 6, in, The neural network includes a depthwise separable convolutional network, and the number of output channels of the point convolution of the depthwise separable convolutional network is consistent with the multiplication and accumulation operations supported by the fingerprint recognition device that executes the fingerprint recognition method.

10. The fingerprint recognition method according to claim 9, in, The deep separable convolutional network includes a separable convolutional structure with a predetermined number of output channels. Among them, the separable convolution structure includes depth-wise convolution layer and point-wise convolution layer. Among them, the convolution operation of the depth-by-depth convolution layer is accelerated using the convolution hardware accelerator of the fingerprint recognition device, and the convolution operation of the point-by-point convolution layer is accelerated using the multiplication and accumulation hardware accelerator of the fingerprint recognition device.

11. The fingerprint recognition method according to claim 6, in, The step of dividing the input fingerprint image into the M image blocks comprises: Obtaining a plurality of image blocks by dividing the input fingerprint image; Determine the center coordinates of the input fingerprint image; A first number of image blocks closest to the central coordinates among the multiple image blocks and a second number of image blocks farthest from the central coordinates among the multiple image blocks are determined as the M image blocks, wherein the sum of the first number and the second number is equal to M.

12. The fingerprint recognition method according to claim 11, in, The step of dividing the input fingerprint image into the plurality of image blocks comprises: Use feature extraction algorithm to extract multiple minutiae points from the input fingerprint image, Each of the multiple image blocks is centered on a different detail point among the multiple detail points.

13. A fingerprint recognition device, the fingerprint recognition device include: The receiving module is configured to: receive an input fingerprint image; An obtaining module is configured to: obtain M image blocks by dividing the input fingerprint image, where M is an integer greater than 1; The recognition module is configured to: calculate the scores of N image blocks among the M image blocks by inputting consecutive image blocks among the M image blocks into a neural network for fingerprint recognition, until the scores of the N image blocks meet a preset condition, wherein the score is a total score and N is an integer less than or equal to M; suspend the calculation of the scores of the remaining image blocks among the M image blocks, and identify the legitimacy of the fingerprint in the input fingerprint image.

Images