Biometric information discrimination method, and biometric information discrimination system

Abstract

To improve discrimination of a biological portion corresponding to biological information for use in biometric authentication to thereby improve authentication accuracy of biometric authentication.SOLUTION: The present invention is directed to a biological information discrimination method executed by at least one computer. The biological information discrimination method has the steps of: acquiring a captured image obtained by capturing a plurality of fingers of a subject; detecting each of the fingers based on the captured image; based on a size of a detection region of each of the plurality of detected fingers, determining, among the plurality of fingers, a reference finger as a discrimination reference of the plurality of fingers to discriminate the reference finger; and, based on a position of the reference finger, discriminating which fingers the other fingers are to output the discrimination result of the plurality of fingers.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to a biometric information identification method and a biometric information identification system. [Background technology]

[0002] Patent Literature 1 discloses a method for detecting zones that may contain at least one fingerprint in an image of at least a portion of an individual's hand. The method involves acquiring an image of at least a portion of the hand illuminated by an optical sensor without contact, and determining a plurality of zones in the acquired image where points exhibiting a light intensity level equal to or greater than a light intensity threshold. To select zones from the determined zones that may represent the fingerprint of at least one finger, the method selects a set of thresholds from among finger shape identification criteria and / or fingerprint standard texture identification criteria and / or finger standard color identification criteria, and stores the plurality of zones of the image using each threshold included in the selected set of thresholds. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 10,248,832 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the configuration of Patent Document 1 does not identify which finger is shown in the finger image. In fingerprint authentication using a fingerprint acquired in this way, if the fingerprint acquired during fingerprint authentication differs from the fingerprint at the time of fingerprint registration, the fingerprint may not be recognized even if it is the person's fingerprint, and the authentication accuracy of the fingerprint authentication may decrease.

[0005] The present disclosure has been devised in consideration of the above-described conventional circumstances, and aims to provide a biometric information identification method and a biometric information identification system that identify a body part corresponding to biometric information used for biometric authentication and improve the authentication accuracy of the biometric authentication. [Means for solving the problem]

[0006] The present disclosure provides a biometric information identification method performed by at least one computer, which acquires an image of a subject's multiple fingers, detects each of the multiple fingers based on the captured image, determines a reference finger that serves as an identification standard for the multiple fingers based on the sizes of detection areas of the detected multiple fingers, identifies the reference finger, identifies which of the other fingers are based on the position of the reference finger, and outputs an identification result for the multiple fingers.

[0007] The present disclosure also provides a biometric information identification system including a camera and at least one computer capable of communicating with the camera, wherein the camera captures images of a subject's multiple fingers and transmits the images to the computer, and the computer detects each of the multiple fingers based on the captured images, determines a reference finger that serves as an identification standard for the multiple fingers based on the sizes of the detection areas of the detected multiple fingers, identifies the reference finger, identifies which of the other fingers are based on the position of the reference finger, and outputs the identification results for the multiple fingers to a monitor. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to identify a body part corresponding to biometric information used for biometric authentication, thereby improving the authentication accuracy of biometric authentication. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the overall configuration of a biometric information identification system according to an embodiment. [Figure 2]1 is a flowchart showing an example of an operation procedure of a biometric information identification device according to an embodiment. [Figure 3] An example of fingertip identification using deep learning [Figure 4] 1 is a flowchart showing an example of an operation procedure of a biometric information identification device according to an embodiment. [Figure 5] An example of bounding box selection [Figure 6] An example of finger identification using bounding box selection. [Figure 7] FIG. 10 is a diagram showing an example of fingertip identification in a modified example of the embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, with reference to the drawings as appropriate, detailed descriptions of embodiments that specifically disclose the configuration and operation of a biometric information identification method and a biometric information identification system according to the present disclosure will be provided. However, unnecessary detailed descriptions may be omitted. For example, detailed descriptions of well-known matters or redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0011] (Embodiment) An example of the overall configuration of a biometric information identification system 100 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the overall configuration of a biometric information identification system 100 according to an embodiment.

[0012] The biometric information identification system 100 acquires a captured image (hereinafter referred to as a "hand image") of fingers, which are a body part used for biometric authentication (hereinafter referred to as a "biological body part"), and detects the multiple fingers of the user that appear in the hand image. The biometric information identification system 100 identifies which of the multiple detected fingers each belongs to.

[0013] In the following description, an example will be described in which multiple fingers captured in a captured image of a hand are identified to perform fingerprint authentication, but the method for identifying a biological part described in the embodiment may be applied to other biometric authentications. For example, the method for identifying a biological part described in the embodiment may be used to identify fingers used in vein authentication.

[0014] The biometric information identification system 100 includes a biometric information identification device P1, a camera 13, and a monitor .

[0015] The biometric information identification device P1 includes a communication unit 10, a processor 11, a memory 12, and a biometric information database DB. The biometric information identification device P1 may also include a camera 13 or a monitor 14. In this case, the biometric information identification system 100 may be realized by one device (biometric information identification device P1).

[0016] The communication unit 10 is configured using a communication interface circuit for transmitting and receiving data. The communication unit 10 is connected to external devices, such as a camera 13 and a monitor 14, so as to be able to communicate wirelessly or via wire, and transmits and receives data.

[0017] The processor 11 is configured using, for example, a central processing unit (hereinafter referred to as "CPU"), a graphics processing unit (hereinafter referred to as "GPU"), or a field programmable gate array (hereinafter referred to as "FPGA"), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 refers to the programs and data stored in the memory 12 and executes the programs to realize the function of identifying (specifying) each of the multiple fingers appearing in the hand image.

[0018] After identifying the finger, the processor 11 may generate a finger image by cutting out an area containing the fingerprint, or may extract features indicating the user's individuality from the finger image. The processor 11 may perform a biometric information registration process by associating the identification information of the identified finger with the finger image or features and storing (registering) them in the biometric information database DB. The processor 11 may also perform fingerprint authentication by matching the identification information and finger image or features of the identified finger with biometric information previously registered in the biometric information database DB. The biometric information registration process or fingerprint authentication may be performed by an external device connected to the biometric information identification device P1 so as to be able to transmit and receive data therefrom.

[0019] The biometric information referred to here is data from which features indicative of the user's individuality can be extracted, or the features themselves indicative of the user's individuality, such as a finger image showing a fingerprint, which is a biological part, a fingerprint pattern, or fingerprint features indicative of the user's individuality.

[0020] The memory 12 includes, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when executing each process of the processor 11, and a read only memory (hereinafter referred to as "ROM") that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operation of the processor 11.

[0021] Camera 13 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is, for example, a solid-state imaging element such as a Charged-Coupled Device (hereinafter referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter referred to as "CMOS"), and converts an optical image formed on the imaging surface into an electrical signal, which is output to processor 11 via communication unit 10.

[0022] The image of the hand and fingers may be captured with the fingertips in contact with any surface such as a glass surface, or may be captured with the fingertips not in contact with any surface.

[0023] The monitor 14 is configured using, for example, a Liquid Crystal Display (hereinafter referred to as "LCD") or an organic electroluminescence (hereinafter referred to as "EL") monitor. The monitor 14 outputs and displays the hand image captured by the camera 13, or the fingertip detection result screen or notification generated by the processor 11.

[0024] The fingertip detection result screen referred to here is a screen generated including information on the identification result that identifies which finger the detected fingertip belongs to, and will be described later.

[0025] The biometric information database DB is a so-called storage, and is configured using a storage medium such as a flash memory, a hard disk drive (hereinafter referred to as an "HDD"), or a solid state drive (hereinafter referred to as an "SSD"), etc. The biometric information database DB stores (registers) identification information identified by the processor 11 and biometric information in association with each other for each user.

[0026] Next, an example of the operation procedure of the biometric information identification device P1 will be described with reference to Fig. 2 and Fig. 3. Fig. 2 is a flowchart showing an example of the operation procedure of the biometric information identification device P1 in an embodiment. Fig. 3 is a diagram showing an example of fingertip identification using deep learning.

[0027] In the following description, an example will be described in which four fingers, namely, the index finger, middle finger, ring finger, and little finger, are identified from a finger image, but the fingers to be imaged are not limited to this. The fingers to be imaged include at least the little finger, and it is sufficient that the image is captured so as to include all fingers starting from the little finger to those used for biometric authentication. For example, if the fingertip used for biometric authentication is the middle finger, it is sufficient that the three fingers from the little finger to the middle finger are imaged.

[0028] In the following explanation, an example will be described in which the user's hand is inserted into the field of view of camera 13 along the Y-axis direction shown in FIG. 3 and multiple fingers are arranged in a line along the X-axis direction, but it goes without saying that the arrangement of the fingers during imaging is not limited to this.

[0029] Processor 11 acquires hand image IMG1 (see FIG. 3) captured by camera 13, which captures the entire hand of a user or the tips of multiple fingers including the little finger (St11). Note that processor 11 may determine whether the image is a planar image captured so that each of the multiple fingertips (finger pads), which are biological parts, and the light receiving surface of an image sensor (not shown) of camera 13 are approximately planar. Processor 11 may acquire hand image IMG1 only when it is determined that the hand image IMG1 captured by camera 13 is a planar image.

[0030] Processor 11 executes image analysis processing using a learning model previously trained by deep learning, detects objects likely to be fingertips from hand image IMG1 (St12), and generates bounding boxes (e.g., bounding boxes FM01, FM02, FM03, FM04, FM05, FM11, FM12, FM13, FM14, FM15, FM16, FM21, FM22, FM23, FM24, and FM25; see FIG. 5) surrounding the areas where the likely fingertip objects are detected. Note that known techniques may be used to detect and identify fingertips using training data from deep learning.

[0031] Processor 11 groups the generated bounding boxes by class. Note that a class here refers to a type of finger. Processor 11 calculates a confidence score indicating the likelihood that a fingertip is captured in the area enclosed by each bounding box. Based on the calculated confidence scores, processor 11 selects the bounding box with the highest confidence score for each class (St13).

[0032] In the example shown in FIG. 3, bounding box FM10 is the bounding box with the highest confidence score among the bounding boxes grouped into the class "index finger." Bounding box FM20 is the bounding box with the highest confidence score among the bounding boxes grouped into the class "middle finger." Bounding box FM30 is the bounding box with the highest confidence score among the bounding boxes that detected the class "ring finger." Bounding box FM40 is the bounding box with the highest confidence score among the bounding boxes that detected the class "pinky finger."

[0033] The processor 11 determines whether or not each of the four fingers (index finger, middle finger, ring finger, and little finger) has been detected based on the class into which each of the picked-up bounding boxes is grouped (St14).

[0034] When it is determined in the process of step St14 that each of the four fingers has been detected (St14, YES), processor 11 determines whether the order of the detected four fingers is correct based on the class of each of the four fingers (St15). Specifically, processor 11 determines whether the order of the detected four fingers is "index finger (index), middle finger (middle), ring finger (ring), little finger (pinky)" or "pinky finger (pinky), ring finger (ring), middle finger (middle), index finger (index)" based on the X coordinate of a predetermined corner (e.g., the upper left corner) of the four corners of the bounding box of the detected four fingers, or the X coordinate of the center coordinate of the bounding box.

[0035] 3, multiple fingertips, which are biological parts, are imaged so as to be lined up in the X direction. In this case, processor 11 determines whether the order of the detected four fingers is correct based on the X coordinate of the center coordinate Pt10 of the index finger, the X coordinate of the center coordinate Pt20 of the middle finger, the X coordinate of the center coordinate Pt20 of the middle finger, the X coordinate of the center coordinate Pt30 of the ring finger, the X coordinate of the center coordinate Pt30 of the ring finger, and the X coordinate of the center coordinate Pt40 of the little finger.

[0036] On the other hand, if it is determined in the process of step St14 that none of the four fingers has been detected (St14, NO), the processor 11 proceeds to the process of step St 18. The process of step St18 will be described later.

[0037] When processor 11 determines in step St15 that the order of the four fingers is incorrect (St15, NO), processor 11 proceeds to step St 18. The process of step St18 will be described later.

[0038] On the other hand, if the processor 11 determines in the processing of step St15 that the order of the four fingers is correct (St15, YES), it further determines whether or not the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is equal to or less than a predetermined distance D1 (for example, the predetermined distance D1 = 20 mm) based on the center coordinates of the bounding boxes corresponding to each detected finger (St16).

[0039] The value of the predetermined distance D1 may be a variable. For example, the processor 11 may estimate the size of the user's hand based on the size of the angle of view of the camera 13, the size of the hand relative to the size of the angle of view of the camera 13, the size of the bounding box, or the distance between the center coordinates of the bounding boxes corresponding to both fingers (e.g., the little finger and the index finger), and determine the value of the predetermined distance D1 based on the estimated size of the hand. This allows the processor 11 to more accurately determine whether the detected fingers are the same finger or whether the detected fingers overlap, even if the user is a child or a person whose fingers are smaller than those of an adult.

[0040] In the example shown in FIG. 3, after determining that the arrangement order of the detected four fingers is correct, the processor 11 calculates the center coordinates Pt10, Pt20, Pt30, and Pt40 of each of the bounding boxes FM10 to FM40. Based on the calculated center coordinates Pt10 to Pt40 of each of the bounding boxes FM10 to FM40, the processor 11 determines whether the distance between two adjacent fingers is equal to or less than a predetermined distance D1. As a result, the processor 11 determines whether adjacent fingers of the detected four fingers overlap or whether the positions of the adjacent fingers are too close. Note that, in the process of step St15, when the arrangement order of the four fingers is determined based on the coordinate information of the bounding boxes, the calculation process of the center coordinates may be executed in step St15.

[0041] 3, the processor 11 determines whether a distance L11 between the center coordinate Pt10 of the index finger and the center coordinate Pt20 of the middle finger, a distance L12 between the center coordinate Pt20 of the middle finger and the center coordinate Pt30 of the ring finger, and a distance L13 between the center coordinate Pt30 of the ring finger and the center coordinate Pt40 of the little finger are each equal to or less than a predetermined distance D1. This allows the processor 11 to more effectively prevent multiple fingers that are imaged overlapping or close to each other from being erroneously detected (misidentified) as a single finger.

[0042] It is possible that the user's fingers are not aligned in a predetermined direction (for example, the X direction in FIG. 3) due to an arbitrary image posture of the user's fingers. In such a case, the processor 11 may further perform image analysis on the length of each bounding box, the position of the palm, the direction in which the fingers extend, and the like, to estimate the direction in which the fingers are aligned on the captured image IMG1.

[0043] If the processor 11 determines in the processing of step St16 that the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is not equal to or less than the predetermined distance D1 (St16, NO), it determines that the finger detection result estimated by deep learning is correct, generates a fingertip detection result screen (see FIG. 3) in which the fingertip detection result is superimposed on the hand image IMG1, and outputs it to the monitor 14 (St17). Note that the fingertip detection result shown in FIG. 3 shows an example including a bounding box (frame line) indicating the area in which each finger is detected and a finger name indicating which finger each detected finger belongs to, but may include only the bounding box (frame line).

[0044] On the other hand, if the processor 11 determines in the processing of step St16 that the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is equal to or less than the predetermined distance D1 (St16, YES), it determines that the finger detection result estimated by deep learning may be incorrect, and proceeds to the processing of step St18. The processing of step St18 will be described later.

[0045] Next, an example of the operation procedure of the biometric information identification device P1 will be described with reference to each of Figs. 4 to 6. Fig. 4 is a flowchart showing an example of the operation procedure of the biometric information identification device P1 in an embodiment. Fig. 5 is a diagram explaining an example of selecting a bounding box. Fig. 6 is a diagram showing an example of identifying fingers by selecting a bounding box. Note that, in Fig. 5, in order to make the explanation of the example of selecting a bounding box easier to understand, only the index finger and middle finger of the finger image IMG1 shown in Fig. 3 are shown, and the other fingers are not shown.

[0046] Processor 11 refers to the reliability scores of all the bounding boxes that have been generated, and selects bounding boxes whose reliability scores are equal to or greater than a threshold as candidates for the bounding box that detects the object (fingertip) that is the detection target (hereinafter referred to as "bounding box candidates") (St18). Note that the threshold here is a threshold set for the reliability score of the bounding box itself.

[0047] 5, processor 11 determines whether the reliability score of each of bounding boxes FM01 to FM05, FM11 to FM16, and FM21 to FM25 is equal to or greater than a threshold. Processor 11 selects each of bounding boxes FM11 to FM16 and FM21 to FM25 whose reliability score is determined to be equal to or greater than the threshold as a bounding box candidate. This allows biometric information identification device P1 to exclude from bounding box candidates bounding boxes that have low reliability scores and are unlikely to detect a fingertip, thereby preventing a decrease in fingertip detection accuracy.

[0048] Processor 11 calculates the similarity of the center coordinates of the bounding box candidates using the k-means algorithm, and groups the picked bounding box candidates into four groups based on the calculated similarity (St19). Note that the number of groupings may be set to any number depending on the number of fingers to be detected.

[0049] 5, processor 11 groups bounding boxes FM11 to FM16 as an index finger group and bounding boxes FM21 to FM25 as a middle finger group based on the similarity of the center coordinates of each bounding box. For ease of explanation, in FIG. 5, as an example of the grouping results, bounding boxes FM11 to FM16 are surrounded by group GR1 (index finger group) and bounding boxes FM21 to FM25 are surrounded by group GR2 (middle finger group) to visualize the bounding boxes belonging to each group.

[0050] The processor 11 selects the bounding box having the maximum reliability score from the grouped bounding box candidates for each class (finger) (St20).

[0051] 5, processor 11 selects bounding box FM12, which has the highest reliability score, from among the bounding box candidates belonging to group GR1. Processor 11 also selects bounding box FM23, which has the highest reliability score, from among the bounding box candidates belonging to group GR2.

[0052] The processor 11 determines the bounding box having the smallest area (size) of the four selected bounding boxes (for example, the bounding box FM44 shown in FIG. 6) as the bounding box of the little finger (St21).

[0053] In the example shown in FIG. 6, processor 11 determines that bounding box FM44, which has the smallest area (size) of the bounding boxes FM12, FM23, FM35, and FM44 selected for each class, is the bounding box for the little finger.

[0054] The processor 11 determines the classes (fingers) corresponding to the other three selected bounding boxes based on the bounding box of the little finger (St22).

[0055] In the example shown in FIG. 6, processor 11 uses the bounding box FM44 of the little finger (pinky) as a reference and determines, in order of proximity to bounding box FM44, the class of the other three bounding boxes FM12 to be the ring finger (ring), the class of bounding box FM23 to be the middle finger (middle), and the class of bounding box FM35 to be the index finger (index).

[0056] The processor 11 determines whether the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is equal to or less than a predetermined distance D2 (St23). Note that the predetermined distance D2 may be the same as the predetermined distance D1. The value of the predetermined distance D2 may be a variable, similar to the predetermined distance D1.

[0057] 6, the processor 11 calculates the center coordinates Pt12, Pt23, Pt35, and Pt44 of the bounding boxes FM12, FM23, FM35, and FM44, respectively. Based on the calculated center coordinates Pt12, Pt23, Pt34, and Pt44 of the bounding boxes FM12, FM23, FM35, and FM44, respectively, the processor 11 determines whether the distance between two adjacent fingers is equal to or greater than a predetermined distance D2, that is, whether adjacent fingers overlap or are positioned too close to each other.

[0058] 6, processor 11 determines whether a distance L21 between the center coordinate Pt12 of the index finger and the center coordinate Pt23 of the middle finger, a distance L22 between the center coordinate Pt23 of the middle finger and the center coordinate Pt35 of the ring finger, and a distance L23 between the center coordinate Pt35 of the ring finger and the center coordinate Pt44 of the little finger are each equal to or less than a predetermined distance D2. This allows processor 11 to determine whether there are multiple fingers that are imaged overlapping each other, or whether the same finger is detected in multiple different bounding boxes.

[0059] It is possible that the user's fingers are not aligned in a predetermined direction (for example, the X direction in FIG. 6) due to an arbitrary image posture of the user's fingers. In such a case, the processor 11 may further perform image analysis on the length of each bounding box, the position of the palm, the direction in which the fingers extend, and the like, to estimate the direction in which the fingers are aligned on the captured image IMG1.

[0060] If the processor 11 determines in the processing of step St23 that the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is equal to or shorter than the predetermined distance D2 (St23, YES), the processor 11 determines that the distance between the fingers is small and that the adjacent fingers overlap. Therefore, the processor 11 determines that fingertip detection is not possible, generates a notification to that effect, and outputs the notification to the monitor 14 (St25).

[0061] On the other hand, if the processor 11 determines in the processing of step St23 that the distance between the center coordinates of the bounding boxes corresponding to two adjacent classes (fingers) is not equal to or less than the predetermined distance D2 (St23, NO), it determines that the distance between the two adjacent fingers is sufficiently far apart and that the adjacent fingers do not overlap.

[0062] Processor 11 further determines whether the X coordinate value of the center coordinate of the bounding box of the little finger is the minimum or maximum (St24). In other words, processor 11 determines whether the position of the little finger is located at the end of the arrangement of the four fingers.

[0063] Although the present embodiment illustrates an example in which it is determined whether the X-coordinate value of the center coordinate of the bounding box is minimum or maximum, it goes without saying that the value used in the determination process of step St24 is not limited to this. The value used in the determination process of step St24 may be set to any value based on the direction in which the multiple fingertips are lined up on the fingering image IMG1, or may be the Y-coordinate value of the center coordinate when the user's hand is inserted in the X-axis direction. Furthermore, the determination process of step St24 may determine whether the distance between any of the four corners of the fingering image IMG1 that are aligned in the direction in which the fingertips are lined up, or any of the four sides of the rectangular fingering image IMG1 that intersect with the direction in which the fingertips are lined up, and the center coordinate of the little finger's bounding box is minimum or maximum, based on the direction in which the multiple fingertips are lined up on the fingering image IMG1.

[0064] If the processor 11 determines in the processing of step St24 that the X coordinate value of the center coordinate of the little finger bounding box is minimum or maximum (St24, YES), it determines that the detection and identification of the fingertips is complete, generates a fingertip detection result screen IMG2 (see FIG. 6) in which the fingertip detection results are superimposed on the hand image IMG1, and outputs it to the monitor 14 (St26). Note that the fingertip detection results shown in FIG. 6 show an example in which they include bounding boxes (frame lines) indicating the areas in which each finger is detected and finger names indicating which finger each detected finger belongs to, but they may be bounding boxes (frame lines) only.

[0065] On the other hand, if the processor 11 determines in the processing of step St24 that the X coordinate value of the center coordinate of the little finger bounding box is not the minimum or maximum (St24, NO), it determines that the fingertip cannot be detected and identified, generates a notification that the fingertip cannot be detected, and outputs it to the monitor 14 (St25).

[0066] As described above, the biometric information identification device P1 in the embodiment uses a conventional fingertip detection and identification method, and generates a bounding box that surrounds the area of ​​an object that appears to be a fingertip in the hand image IMG1 through image analysis processing using a learning model generated by deep learning.As a result, the biometric information identification device P1 can identify (specify) which finger each detected finger is, using the generated bounding box.

[0067] Furthermore, even if the predetermined number of fingers are not detected by conventional image analysis processing or the arrangement of the detected fingers is incorrect (if the answer is "NO" in steps St14 and St15 shown in FIG. 2, or if the answer is "YES" in step St16), the biometric information identification device P1 can identify (specify) which finger each detected finger is by performing the processing of steps St18 to St24 and identifying the bounding box that detected each finger from among the multiple bounding boxes that were generated.

[0068] As described above, the biometric information identification device P1 in the embodiment can identify with high accuracy which finger each of the multiple fingertips in the hand image belongs to. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the authentication accuracy of the biometric authentication.

[0069] (Modification of the embodiment) In the embodiment, an example has been shown in which the little finger is used as a reference finger to identify each of the other fingers (ring finger, middle finger, and index finger). In the following modified example of the embodiment, an example will be described in which the thumb is used as a reference finger to identify each of the other fingers (ring finger, middle finger, and index finger).

[0070] The biometric information identification device according to the modified example of the embodiment has the same configuration as the biometric information identification device P1 according to the embodiment, and therefore the description thereof will be omitted.

[0071] The fingertip identification process executed by the biometric information identification device P1 according to the modified embodiment will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of fingertip identification according to the modified embodiment.

[0072] In the following description, an example of identifying each of five fingers from a finger image will be described, but the fingers to be imaged are not limited to this. The fingers to be imaged should include at least the thumb (reference finger), and should be imaged so as to include all fingers starting from the thumb to be used for biometric authentication. For example, if the fingertip to be used for biometric authentication is the middle finger, it is sufficient to image the three fingers from the thumb to the middle finger.

[0073] The processor 11 in the biometric information identification device P1 executes the processes of steps St11 to St13 shown in FIG. 2 to detect five fingers appearing in the hand image IMG3 (St14), and determines whether the order of the detected five fingers is correct (St15).

[0074] The biometric information identification device P1 executes the processing of step St16 shown in FIG. 2, and if it determines that the fingertip detection result using deep learning is correct (St16, NO), it outputs the fingertip detection result (St17), and if it determines that the fingertip detection result using deep learning may be incorrect (St16, YES), it executes the processing of steps St18 to St20 shown in FIG. 4.

[0075] Processor 11 determines the bounding box with the largest area as the bounding box of the thumb (reference finger) (St21A). After determining the bounding box of the thumb, which is the reference finger, processor 11 executes the processes of steps St22 to St23 and determines whether the center coordinates of the bounding box of the thumb (reference finger) are minimum or maximum (St24).

[0076] If the processor 11 determines that the center coordinates of the bounding box of the thumb (reference finger) are the minimum or maximum (St24, YES), it outputs the fingertip detection result determined in step St22 (St26). If the processor 11 determines that the center coordinates of the bounding box of the thumb (reference finger) are not the minimum or maximum (St24, NO), it determines that the fingertip cannot be detected and identified, generates a notification that the fingertip cannot be detected, and outputs it to the monitor 14 (St25).

[0077] As described above, the biometric information identification device P1 in the embodiment can identify with high accuracy which finger each of the multiple fingertips in the hand image belongs to, even when the reference finger is set to the thumb. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the authentication accuracy of biometric authentication.

[0078] (Addendum) The above description of each embodiment discloses the following techniques.

[0079] (Technology 1) A biometric information identification method performed by at least one computer (biometric information identification device P1), A captured image (finger image) of a subject's multiple fingers is acquired, Detecting each of the plurality of fingers based on the captured image (finger image); determining a reference finger (little finger or thumb) that serves as an identification reference for the plurality of fingers based on the sizes of the detected detection regions of the plurality of fingers (for example, bounding boxes FM12, FM23, FM35, FM44), and identifying the reference finger; identifying which of the other fingers is based on the position of the reference finger, and outputting the identification results of the plurality of fingers; Biometric identification methods. With this configuration, the biometric information identification device P1 can identify with high accuracy which finger each of the multiple fingertips in the hand image belongs to by using the little finger or the thumb as a reference. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the accuracy of biometric authentication.

[0080] (Technology 2) identifying the other fingers based on a distance between the reference finger and the other fingers; A biometric information identification method described in (Technology 1). With this configuration, the biometric information identification device P1 can identify with high accuracy which finger each of the multiple fingertips in the hand image belongs to, based on the distance between the little finger and the thumb. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the accuracy of biometric authentication.

[0081] (Technology 3) identifying the finger with the smallest detection area among the detection areas of the plurality of fingers as the little finger; A biometric information identification method according to (Technology 1) or (Technology 2). With this configuration, the biometric information identification device P1 can identify the little finger among multiple fingers captured in a hand image with higher accuracy, thereby improving the identification accuracy of each of multiple fingertips based on the little finger.

[0082] (Technology 4) identifying the finger with the largest detection area among the detection areas of the plurality of fingers as the thumb; A biometric information identification method according to (Technology 1) or (Technology 2). With this configuration, the biometric information identification device P1 can identify the thumb among multiple fingers captured in a hand image with higher accuracy, thereby improving the accuracy of identifying each of the multiple fingertips based on the thumb.

[0083] (Technology 5) Calculating the distances (e.g., distances L21 to L23) between the two detection regions corresponding to the two adjacent fingers based on the identification results of the plurality of fingers; If it is determined that all of the calculated distances (for example, distances L21 to L23) are equal to or greater than a predetermined distance D2, an identification result of the plurality of fingers is output. The biometric information identification method according to any one of (Technology 1) to (Technology 4). With this configuration, the biometric information identification device P1 determines that one finger is detected for one bounding box based on the positions of multiple detected fingers. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the authentication accuracy of biometric authentication.

[0084] (Technology 6) If it is determined that at least one of the distances (for example, distances L21 to L23) is not equal to or greater than the predetermined distance D2, a notification that the plurality of fingers cannot be identified is generated and output. A biometric information identification method described in (Technology 5). With this configuration, the biometric information identification device P1 can suppress erroneous detection or erroneous identification of fingers due to whether or not two adjacent fingers overlap, suppress erroneous detection or erroneous identification of fingers due to adjacent fingers being too close to each other, and determine that the same finger may be detected in multiple different bounding boxes. As a result, the biometric information identification device P1 can more effectively suppress a decrease in the accuracy of finger identification and fingerprint authentication by determining that finger identification is impossible.

[0085] (Technology 7) outputting an identification result of the plurality of fingers when it is determined that the reference finger is located at an end of the arrangement of the plurality of fingers based on the positions of the detection areas of the plurality of fingers (i.e., the center coordinates of the bounding boxes); The biometric information identification method according to any one of (Technology 1) to (Technology 6). With this configuration, the biometric information identification device P1 can identify the little finger among multiple fingers captured in a hand image with higher accuracy, thereby improving the identification accuracy of each of multiple fingertips based on the little finger.

[0086] (Technology 8) If it is determined that the reference finger is not located at an end of the array, a notification that the plurality of fingers cannot be identified is generated and output. A biometric information identification method described in (Technical 7). With this configuration, if the biometric information identification device P1 determines that the reference finger is not the little finger, it determines that the finger cannot be identified, thereby more effectively preventing a decrease in the accuracy of finger identification and the authentication accuracy of fingerprint authentication.

[0087] (Technology 9) Camera 13 and A biometric information identification system 100 including at least one computer (biometric information identification device P1) capable of communicating with the camera 13, The camera 13 is An image of a plurality of fingers of a subject is captured and transmitted to the computer (biometric information identification device P1), The computer (biometric information identification device P1) Detecting each of the plurality of fingers based on the captured image (finger image); determining a reference finger (little finger) that serves as an identification standard for the plurality of fingers based on the sizes of the detected detection regions of the plurality of fingers (for example, bounding boxes FM12, FM23, FM35, FM44), and identifying the reference finger; identifying which of the other fingers is based on the position of the reference finger, and outputting the identification results of the plurality of fingers to a monitor. Biometric identification system 100. With this configuration, the biometric information identification system 100 can identify with high accuracy which finger each of the multiple fingertips in the hand image belongs to by using the little finger as a reference. This enables the biometric information identification device P1 to identify (specify) the biometric information used for biometric authentication, thereby improving the authentication accuracy of biometric authentication.

[0088] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]

[0089] The present disclosure is useful as a biometric information identification method and a biometric information identification system that identify a body part corresponding to biometric information used in biometric authentication and improve the authentication accuracy of biometric authentication. [Explanation of symbols]

[0090] 10. Communications Department 11 processors 12 Memory 13 Camera 14 monitors 100 Biometric Identification System D1,D2 Predetermined distance DB Biological Information Database FM01,FM02,FM03,FM04,FM05,FM1,FM2,FM3,FM4,FM5,FM10,FM11,FM12,FM13,FM14,FM15,FM16,FM20,FM21,FM22,FM23,FM24,FM25,FM30,FM35,FM40,FM44 Bounding Box IMG1 Hand image IMG2 Fingertip detection result screen L11,L12,L13,L21,L22,L23 distance P1 Biometric Identification Device Pt10,Pt12,Pt20,Pt23,Pt30,Pt34,Pt35,Pt40,Pt44 Center coordinates

Claims

1. 1. A biometric identification method implemented by at least one computer, comprising: Acquire a captured image of a plurality of fingers of a subject; Detecting each of the plurality of fingers based on the captured image; determining a reference finger that serves as an identification standard for the plurality of fingers based on the size of the detection regions of the plurality of fingers detected, and identifying the reference finger; identifying which of the other fingers is based on the position of the reference finger, and outputting the identification results of the plurality of fingers; Biometric identification methods.

2. identifying the other fingers based on a distance between the reference finger and the other fingers; The biometric information identification method according to claim 1 .

3. identifying the finger with the smallest detection area among the detection areas of the plurality of fingers as the little finger; The biometric information identification method according to claim 1 .

4. identifying the finger with the largest detection area among the detection areas of the plurality of fingers as the thumb; The biometric information identification method according to claim 1 .

5. calculating distances between two of the detection areas corresponding to two adjacent fingers based on the identification results of the plurality of fingers; outputting a result of identifying the plurality of fingers when it is determined that all of the calculated distances are equal to or greater than a predetermined distance; The biometric information identification method according to claim 1 .

6. If it is determined that at least one of the distances is not equal to or greater than the predetermined distance, a notification that the plurality of fingers cannot be identified is generated and output. The method for identifying biometric information according to claim 5.

7. outputting an identification result of the plurality of fingers when it is determined that the reference finger is located at an end of the arrangement of the plurality of fingers based on the positions of the detection areas of the plurality of fingers; The biometric information identification method according to claim 1 .

8. generating and outputting a notification that the plurality of fingers cannot be identified when it is determined that the reference finger is not located at an end of the array; The biometric information identification method according to claim 7.

9. A camera and and at least one computer capable of communicating with the camera, The camera is capturing an image of a plurality of fingers of a subject and transmitting the image to the computer; The computer Detecting each of the plurality of fingers based on the captured image; determining a reference finger that serves as an identification standard for the plurality of fingers based on the size of the detection regions of the plurality of fingers detected, and identifying the reference finger; identifying which of the other fingers is based on the position of the reference finger, and outputting the identification results of the plurality of fingers to a monitor. Biometric identification system.

Images