Information processing device and information processing method

Abstract

To detect a posture relating to two rotation axes in roll rotation and pitch rotation.SOLUTION: An information processing apparatus is configured to: hold an image of a hand; calculate a first axis ratio indicating a roll rotation amount of a finger of the hand based on a length of a first finger and a width of the hand in the image; calculate a second axis ratio indicating a pitch rotation amount of the finger based on a length and a width of a second finger in the image; determine whether the image is an image to be registered based on the first axis ratio and the second axis ratio; and when the image is determined to be the image to be registered, output information on the hand and posture information including the first axis ratio and the second axis ratio.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device and an information processing method. [Background technology]

[0002] Among various biometric authentication technologies, finger vein authentication is known to be highly accurate. Finger vein authentication achieves high authentication accuracy by using the blood vessel pattern inside the finger, and is more difficult to forge and tamper with than fingerprint authentication, providing a high level of security.

[0003] In recent years, mobile devices such as notebook PCs (Personal Computers), smartphones, and tablets have come into widespread use around the world, and many of them are equipped with general-purpose color cameras. Finger vein authentication, which uses this camera to photograph fingers, is expected to become more widely used in the future because it eliminates the need for dedicated finger vein authentication devices. In addition, by displaying the image of the biometric subject being photographed on the device screen, it becomes easier to photograph the biometric subject, enabling highly accurate authentication.

[0004] As biometric authentication on mobile devices becomes more common, it is necessary to maintain the convenience and accuracy of authentication systems at a high level. However, when a finger needs to be held over a camera and held still in the air, there are an infinite number of possible finger positions, and the finger vein information acquired changes depending on the distance between the fingers and the inclination of the fingers.

[0005] Background of the present invention is Patent Document 1. This publication states that "The biometric authentication program according to the present invention displays, as a guide image on the screen, an arched trajectory image that prompts the user to keep their fingertip along the arched trajectory without covering the screen" (see abstract). [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2020-3873 A Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 detects the posture of the finger, and if the angle or inclination of the finger is not within a predetermined range, displays a message on the screen urging the user to adjust it. The technology described in Patent Document 1 detects the inclination of the finger when rotating on one axis. Specifically, the technology described in Patent Document 1 detects the inclination of the finger in roll rotation, which is the rotation of the palm surface with the straight line connecting the midpoint of the connection part of the middle finger and the palm and the center of the outline of the wrist as the rotation axis.

[0008] However, not only the roll rotation alone can cause the hand and fingers to tilt, but the tilt can also be caused by the pitch rotation, which is the rotation of the palm surface around an axis that is perpendicular to the roll rotation axis and horizontal to the palm surface. Therefore, one aspect of the present invention detects the posture regarding the two rotation axes, the roll rotation and the pitch rotation. [Means for solving the problem]

[0009] In order to solve the above problem, one aspect of the present invention employs the following configuration: An information processing device includes a processor and a memory, the memory holds an image of a hand, and the processor calculates a first axis ratio indicating a roll rotation amount of the fingers of the hand based on a length of a first finger and a width of the hand in the image, calculates a second axis ratio indicating a pitch rotation amount of the fingers based on a length and width of a second finger in the image, determines whether the image is a registration target image based on the first axis ratio and the second axis ratio, and outputs information about the hand and posture information including the first axis ratio and the second axis ratio when it is determined that the image is the registration target image. Effect of the Invention

[0010] One embodiment of the present invention is capable of detecting attitudes regarding two rotation axes, roll rotation and pitch rotation.

[0011] Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing a configuration example of a biometric authentication system according to a first embodiment. [Diagram 2] 11 is a flowchart illustrating an example of a registration process according to the first embodiment. [Diagram 3] 11 is a flowchart illustrating an example of an authentication process according to the first embodiment. [Figure 4] 11 is a flowchart illustrating an example of a posture detection process in the first embodiment. [Diagram 5] FIG. 4 is an explanatory diagram illustrating an example of a posture detection process in the first embodiment. [Figure 6] FIG. 11 is an explanatory diagram showing an example of a guide displayed on a display device during the registration process in the first embodiment to encourage the user to present their hand, change the spacing between their fingers, and rotate their hand up, down, left, and right around their wrist. [Figure 7] 13 is an example of a guide displayed on a display device to guide a user to correct the posture of his or her hand during authentication processing in the first embodiment. [Figure 8] 13 is a flowchart illustrating an example of a registration process according to the second embodiment. [Figure 9] 13 is a flowchart illustrating an example of an authentication process according to the second embodiment. [Figure 10] 13 is a system configuration example for performing identity authentication when accessing a system for online services such as Internet banking using a mobile terminal according to a third embodiment. [Figure 11] 13 is a configuration example of an authentication system according to a fourth embodiment, which performs payment using only biometric information in retail stores such as supermarkets and restaurants. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following description and drawings are examples for explaining the present invention, and some omissions and simplifications are made for clarity of explanation. The present invention can be implemented in various other forms. Unless otherwise specified, each component may be singular or plural.

[0014] In order to facilitate understanding of the invention, the position, size, shape, range, etc. of each component shown in the drawings may not represent the actual position, size, shape, range, etc. Therefore, the present invention is not necessarily limited to the position, size, shape, range, etc. disclosed in the drawings. EXAMPLES

[0015] 1 is a block diagram showing an example of the configuration of a biometric authentication system 10. The biometric authentication system 10 authenticates biometrics from an image of a hand, for example, by using an image of blood vessels (e.g., finger veins) in the fingers. The biometric authentication system 10 includes, for example, an imaging device 2, and an information processing device 3 and a server 5 connected via a network 4 such as the Internet. The imaging device 2 may be provided in the information processing device 3.

[0016] The imaging device 2 captures a video of the hand and inputs it to the information processing device 3. The information processing device 3 executes a series of processes including video processing, posture detection processing, posture determination processing, feature extraction processing, and template generation processing based on the video of the hand. The information processing device 3 and the server 5 execute registration processing and authentication processing via the network 4. If registration or authentication is successful, or if posture correction is required, that fact is displayed on the display device 36 of the information processing device 3. The information processing device 3 is configured by a computer including, for example, a CPU 31, a memory 32, an auxiliary storage device 33, a communication device , an input device 35, and a display device .

[0017] The CPU 31 includes a processor and executes programs stored in the memory 32. The memory 32 includes a ROM (Read Only Memory), which is a non-volatile storage element, and a RAM (Random Access Memory), which is a volatile storage element. The ROM stores immutable programs (e.g., a Basic Input / Output System (BIOS)). The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the CPU 31 and data used when the programs are executed.

[0018] The auxiliary storage device 33 is a large-capacity non-volatile storage device such as a magnetic storage device (HDD (Hard Disk Drive)) or a flash memory (SSD (Solid State Drive)), and stores the programs executed by the CPU 31 and data used when the programs are executed. That is, the programs are read from the auxiliary storage device 33, loaded into the memory 32, and executed by the CPU 31.

[0019] In this embodiment, the information used by the biometric authentication system 10 does not depend on the data structure and may be expressed in any data structure. For example, the information may be stored in a data structure appropriately selected from a table, a list, a database, or a queue.

[0020] The input device 35 is a device such as a keyboard or a mouse that receives input from an operator. The display device 36 is a device such as a display device or a printer that outputs the execution result of a program in a format that can be viewed by an operator. The information processing device 3 may also have an input interface for receiving input of moving images from the imaging device 2, or may receive moving images from the imaging device 2 via the network 4. The communication device 34 is a network interface device that controls communication with other devices according to a predetermined protocol.

[0021] The program executed by the CPU 31 may be provided to the information processing device 3 from another information processing device equipped with a non-transitory storage device via a removable medium (such as a CD-ROM or a flash memory) which is a non-transitory storage medium or via the network 4, and may be stored in a non-volatile auxiliary storage device 33 which is a non-transitory storage medium. For this reason, the information processing device 3 may have an interface for reading data from the removable medium. The same applies to the server 5.

[0022] The information processing device 3 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate in separate threads on the same computer, or may operate on a virtual computer constructed on multiple physical computer resources. The same applies to the server 5.

[0023] The CPU 31 includes, for example, a moving image input unit 320, a moving image processing unit 321, a posture detection unit 322, a posture determination unit 323, a feature extraction unit 324, a template generation unit 325, and a biometric authentication unit 326.

[0024] The video input unit 320 receives input of a video from the imaging device 2. The video processing unit 321 converts the input video into a plurality of still images. The posture detection unit 322 detects the posture of a hand included in each of the plurality of still images. The posture determination unit 323 performs posture determination based on the posture detected by the posture detection unit 322.

[0025] The feature extraction unit 324 extracts features from a still image. Finger veins, fingerprints, and joints are examples of features extracted by the feature extraction unit 324. The template generation unit 325 generates a template used for registration or authentication from the extracted features. The biometric authentication unit 326 communicates with the server 5 via the network 4 and performs processing related to registration or authentication.

[0026] In the case of registration processing, the biometric authentication unit 326 transmits a registration request and registration data including a template and posture information to the server 5. In the case of authentication processing, the biometric authentication unit 326 transmits a template request to the server 5, and the server 5, having received the request, transmits the registration data to the information processing device 3. The biometric authentication unit 326 compares the template generated during the authentication processing with the registration data received from the server 5 to perform biometric authentication.

[0027] For example, CPU 31 functions as video input unit 320 by operating according to a video input program loaded in memory 32, and functions as video processing unit 321 by operating according to a video processing program loaded in memory 32. The same applies to the relationships between programs and functional units for other functional units included in CPU 31. The same applies to the relationships between programs and functional units for functional units described later included in CPU 51 of server 5.

[0028] Some or all of the functions of the functional units included in the CPU 31 and the CPU 51 may be realized by hardware such as an Application Specific Integrated Circuit (ASIC) or a Field Programmable Gate Array (FPGA).

[0029] The server 5 is configured by a computer including, for example, a CPU 51, a memory 52, an auxiliary storage device 53, a communication device 54, an input device 55, and a display device 56. Descriptions of the hardware of the CPU 51, the memory 52, the auxiliary storage device 53, the communication device 54, the input device 55, and the display device 56 are omitted because they are similar to the descriptions of the hardware of the CPU 31, the memory 32, the auxiliary storage device 33, the communication device 34, the input device 35, and the display device 36.

[0030] The CPU 51 includes a data registration unit 521 and a data acquisition unit 522. When the data registration unit 521 receives a registration request and registration data from the information processing device 3, it registers the registration data in the database 530. When the server 5 receives an authentication request from the information processing device 3, the data acquisition unit 522 acquires the registration data registered in the database 530 and transmits it to the information processing device 3. The auxiliary storage device 53 holds the database 530. The database 530 holds a template including biometric information of each user.

[0031] 1, the CPU 31 of the information processing device 3 includes the biometric authentication unit 326, but the server 5 may perform the biometric authentication, in which case the CPU 51 of the server 5 includes the biometric authentication unit 326. When the server 5 performs the biometric authentication, the biometric authentication unit 326 transmits authentication data including a template and posture information together with an authentication request to the server 5, and the server 5 performs the biometric authentication.

[0032] An example of the registration process and an example of the authentication process will be described with reference to Figures 2 and 3. In the registration process, the information processing device 3 selects one or more images suitable for use as registration data based on the detection of the hand posture, and registers them in the server 5. In the authentication process, the information processing device 3 determines whether the posture information of the held up hand is within a certain range of posture information in the registration data using the detection and determination of the hand posture, and if it is determined to be within the certain range, performs biometric authentication, and if it is determined to be outside the certain range, displays a guide to encourage the user to correct the way the hand is held up.

[0033] 2 is a flowchart showing an example of the registration process. The video input unit 320 displays on the display device 36 a guide encouraging the user to present their hand, change the distance between their fingers, and rotate their hand up, down, left, and right around their wrist (S201). The user holds their hand over the imaging device 2 according to the guide.

[0034] When the information processing device 3 is a mobile terminal such as a smartphone or tablet including an imaging device 2 (for example, a camera mounted on the mobile terminal), the user holds his / her hand in the air so that the imaging device 2 captures the hand within its imaging range.

[0035] For example, the video input unit 320 may display on the display device 36 a guidance video that prompts the user to change the distance between the fingers or a guide video that prompts the user to rotate around the wrist, in addition to a guidance message such as "Please hold your hand" as a guide to encourage the user to present his or her hand. This allows the user to visually understand how to hold his or her hand, improving the user's convenience. An example of the guide will be described with reference to FIG. 5. The imaging device 2 captures a moving image of the user's hand for a certain period of time, and the moving image input unit 320 accepts input of the moving image (S202).

[0036] The video processing unit 321 converts the video input in step S202 into a plurality of still images (S203). In step S203, the video processing unit 321 may convert each of all frames in the video into still images, or may convert only some frames of the video into still images. The video processing unit 321 may also determine the quality of each still image obtained in step S203, and in this case, may delete still images with image quality below a certain level, that is, the deleted still images may not be processed in the subsequent processes.

[0037] In step S202, the imaging device 2 may capture a plurality of still images instead of a moving image, and the moving image input unit 320 may accept the input of the plurality of still images. In this case, the process of step S203 may be omitted.

[0038] The posture detection unit 322 selects one still image that has not been subjected to the process of step S205 from the still images generated in step S203 (S204), and detects the posture of the hand shown in the selected still image (S205).

[0039] Specifically, for example, the posture detection unit 322 detects the outline of the hand, the finger crotch defined by the joint between adjacent fingers, the base defined by the midpoint of the joint between each finger and the palm, and the position of the fingertip. Furthermore, based on the detected information, the posture detection unit 322 calculates the angle difference between adjacent fingers indicating the degree of finger spreading, a first axis ratio that quantifies the amount of roll rotation, and a second axis ratio that quantifies the amount of pitch rotation as posture information of the hand. A specific example of a method for calculating the first axis ratio and the second axis ratio will be described later with reference to FIG. 4.

[0040] The posture detection unit 322 determines whether the process of step S205 has been performed on all still images generated in step S203 (S206). If the posture detection unit 322 determines that there is a still image for which the process of step S205 has not been performed (S206: NO), the process returns to step S204.

[0041] When it is determined that the process of step S205 has been performed on all still images (S206: YES), the posture detection unit 322 selects an image to be used for registration (S207). Specifically, for example, the posture detection unit 322 refers to the posture information of each image detected in step S204 and selects one or more images in which the angle difference between adjacent fingers is equal to or greater than a predetermined threshold value and the first axis ratio and the second axis ratio (for example, the sum, weighted sum, or average of the first axis ratio and the second axis ratio) are maximum. Note that the posture detection unit 322 may select an image in which the angle difference between adjacent fingers is equal to or greater than a predetermined threshold value and the first axis ratio and the second axis ratio, or the sum, weighted sum, or average of the first axis ratio and the second axis ratio are equal to or greater than a predetermined threshold value.

[0042] The posture detection unit 322 can determine whether the intervals between the fingers are sufficient by determining whether each angle difference is equal to or greater than a predetermined threshold. The first axis ratio and the second axis ratio become larger as the fingers are held horizontally relative to the imaging device 2, and become smaller as the fingers are tilted relative to the imaging device 2. Therefore, by selecting an image with the largest first axis ratio and second axis ratio, the posture detection unit 322 can select a still image including a hand with fingers held most horizontally relative to the imaging device 2, that is, an image suitable for registration. Note that the process of calculating the angle difference and the process of determining whether each angle difference is equal to or greater than a predetermined threshold, performed by the posture detection unit 322, may be omitted.

[0043] The feature extraction unit 324 extracts features from the image selected in step S207 (S208). Specifically, for example, the feature extraction unit 324 cuts out the image area of ​​each finger based on the position information of the interdigital space, the base of the finger, and the tip of the finger detected in step S205, and extracts features included in the finger image after adjusting the orientation and magnification of each finger to be constant.

[0044] The template generating unit 325 generates a template for biometric authentication based on the feature amount extracted in step S208 (S209). Specifically, for example, the template generating unit 325 converts the biometric information into a template from which the original biometric information cannot be restored from the generated template. Conversion from biometric information based on Public Biometrics Infrastructure (PBI) technology to a PBI template is an example of conversion into a template from which the original biometric information cannot be restored. The template generating unit 325 may convert or generate a template from which the original biometric information cannot be restored based on other technology, not limited to the PBI template.

[0045] The template generation unit 325 transmits enrollment data including the template generated in step S209 and posture information including the angle differences between the fingers, the first axis ratio, and the second axis ratio detected in step S204, together with a registration request to the server 5 via the network 4 (S210).

[0046] The data registration unit 521 of the server 5 receives the registration request and the registration data transmitted in step S210, and registers the registration data in the database 530 (S211), whereupon the registration process ends.

[0047] In addition, in step S207, the orientation detection unit 322 may select multiple images (for example, select a predetermined number of images in descending order of the sum of the first axis ratio and the second axis ratio), and the processes of steps S208 to S211 are executed for each selected image (that is, the registration data includes the angle difference, template, and orientation information for each of the multiple still images).

[0048] 3 is a flowchart showing an example of the authentication process. The video input unit 320 displays a guide on the display device 36 to encourage the user to present his / her hand (S301), and the user follows the guide to place his / her hand over the screen. The processes of steps S302, S303, and S304 are similar to the processes of steps S202, S203, and S205, respectively, and therefore will not be described here. However, for the sake of simplicity, it is assumed that only one still image is generated in the process of step S303.

[0049] The biometric authentication unit 326 transmits a registration data request to the server 5 via the network 4, and the data acquisition unit 522 of the server 5 acquires the registration data from the database 530 and transmits it to the information processing device 3 (S305). When registration data corresponding to a plurality of still images is registered in the database 530, the data acquisition unit 522 transmits all of the registration data corresponding to each of the plurality of still images to the information processing device 3. The process of step S305 may be executed at any time between steps S301 and S306. For example, the process of step S305 may be executed immediately after the process of step S301, and then the processes of steps S302 to S304 may be executed.

[0050] The posture determination unit 323 compares the posture information of the hand detected from the still image with the posture information of the enrollment data, and determines whether the image is suitable for authentication (S306). Specifically, for example, the posture determination unit 323 performs a first determination as to whether the angle difference between adjacent fingers is equal to or greater than a predetermined threshold, a second determination as to whether the absolute difference between the detected first axis ratio and the first axis ratio of the enrollment data is within a predetermined threshold, and a third determination as to whether the absolute difference between the detected second axis ratio and the second axis ratio of the enrollment data is within a predetermined threshold.

[0051] In the first determination, if the angle differences between adjacent fingers are equal to or greater than a predetermined threshold, it can be determined that the fingers are spaced apart sufficiently. In the second and third determinations, if the absolute differences between the detected first and second axis ratios and the first and second axis ratios of the enrollment data are within a threshold, it can be determined that the still image obtained from the hand held up to the imaging device 2 in the authentication process is not significantly rotated relative to the enrollment data and is suitable for authentication.

[0052] If the posture determination unit 323 determines that the determination condition is not satisfied in at least one of the first, second, and third determinations (S306: FAIL), the video input unit 320 displays a guide on the display device 36 to prompt the user to correct the hand posture (S307), and returns to step S302. Note that when returning to step S302, the registration data has already been acquired from the server 5, so the process of step S305 may be skipped.

[0053] It is desirable that the content of the guide displayed by video input unit 320 in step S307 differs depending on the posture determination result in step S306. For example, if posture determination unit 323 determines in the first determination in step S306 that the finger spacing is insufficient (i.e., if it determines that the angle difference between adjacent fingers is less than a predetermined threshold), video input unit 320 may display a guidance message such as "Please spread your fingers apart" on display device 36 as a guide to encourage the user to spread the finger spacing. A specific example of the guide to be displayed will be described later with reference to FIG. 6.

[0054] When the posture determination unit 323 determines that the determination conditions are satisfied in all of the first, second, and third determinations (S306: PASS), the process proceeds to step S308. The processes of steps S308 and S309 are similar to the processes of steps S207 and S208, respectively, and therefore will not be described.

[0055] The biometric authentication unit 326 executes biometric authentication using the enrollment template of the enrollment data acquired from the server 5 and the authentication template generated in step S309 (S310), and ends the authentication process. Specifically, for example, the biometric authentication unit 326 compares the enrollment template with the authentication template to calculate the similarity, and determines whether the two are similar according to the calculated similarity.

[0056] If the biometric authentication unit 326 determines that the authentication is successful when it determines that the enrollment template and the authentication template are similar, it determines that the authentication is unsuccessful when it determines that the enrollment template and the authentication template are not similar. When the information processing device 3 receives enrollment data corresponding to each of the multiple still images from the server 5 in step S305, the biometric authentication unit 326 may perform biometric authentication on all of the enrollment data, set the highest similarity as the final similarity, and perform authentication by determining whether the enrollment template and the authentication template are similar, or may perform authentication by determining whether each of the enrollment templates is similar to the authentication template.

[0057] Although an example has been described above in which the moving image processing unit 321 generates one still image in the moving image processing in step S303, the moving image processing unit 321 may generate multiple still images in step S303. Two specific examples of processing when the moving image processing unit 321 generates multiple still images will be described.

[0058] In the first example, the information processing device 3 performs step S304 (posture detection process) and step S306 (posture determination process) on one still image, and if the posture determination is successful, proceeds to feature extraction in step S308, and if the posture determination is unsuccessful, selects a different still image and returns to the process of step S304 again. At this time, if the posture determination unit 323 determines that the posture determination for all generated still images has failed, the process transitions to step S307 (guide display).

[0059] In the second example, the information processing device 3 performs the posture detection process in step S304 on all generated still images, selects one image with the maximum first axis ratio and second axis ratio (for example, the maximum sum of the first axis ratio and the second axis ratio), and executes the posture determination process in step S306. At this time, if the posture determination unit 323 determines that the posture determination of the selected image is successful, it transitions to the feature extraction process in step S308, and if it determines that the posture determination is unsuccessful, it transitions to the process in step S307.

[0060] 4 is a flowchart showing an example of the posture detection process in steps S205 and S304. The posture detection unit 322 detects the hand outline, as well as the positions of each finger gap, the base of each finger, and the fingertip from the image (S401). The posture detection unit 322 can apply a method of extracting the hand outline based on separation of the background and the hand region by machine learning using teacher data such as CNN (Convolutional Neural Network), and calculating the positions of each finger gap, the base of each finger, and the fingertip from the extracted hand outline.

[0061] A method for extracting the contour of a hand using machine learning will be described. In the method for extracting the contour of a hand using machine learning, a large number of images (for example, a predetermined number of images) of the hands of various subjects taken in various environments are prepared in advance, labels are created for each image that define the hand and finger parts and the background parts, and pairs of each image and its label are accumulated as teacher data. In this contour extraction method, machine learning parameters are learned so that when an image of the teacher data is input, the label is output as accurately as possible. This allows the posture detection unit 322 to acquire only the hand area when an unknown image is input. The posture detection unit 322 obtains the contour of the hand by tracing the periphery of the acquired hand area.

[0062] Next, a method for calculating the positions of each finger interdigital space, the base and the fingertip of each finger from the hand contour will be described. FIG. 5 is an explanatory diagram showing an example of the posture detection process in step S304. FIG. 5(a) shows an example of the detected hand contour and each coordinate. When tracing the hand contour 110, the posture detection unit 322 defines a position where the curvature of the contour line is high (for example, a position where the curvature is maximum) as the fingertip 111, and defines a position where the curvature is high but the positive and negative signs are opposite to those of the fingertip (for example, a position where the curvature is minimum) as the finger interdigital space 112. The posture detection unit 322 calculates the position of the base 113 of each finger based on the positions of each finger interdigital space and the hand contour.

[0063] The posture detection unit 322 defines the root of the middle finger and the root of the ring finger as the midpoint of a line passing through two adjacent finger crotches, and defines the root of the index finger and the root of the little finger as the tangent point of the vertical line and the midpoint of the adjacent finger crotch when a perpendicular line to the average contour tracing direction of the adjacent finger crotches is drawn from the crotch of the index finger to pass through the contour line located opposite the finger whose root is to be calculated. In (a) of Fig. 5, a vertical line 114 is drawn from the adjacent crotch of the index finger and the little finger to pass through the contour line located opposite each finger (that is, the contour line from the tip of the index finger to the thumb in the case of the index finger, and the contour line from the tip of the little finger to the wrist in the case of the little finger), and a tangent point 115 between the vertical line 114 and the contour 110 of the hand is shown.

[0064] The above-mentioned method for detecting the hand contour, the finger crotch, the base of each finger, and the fingertip positions is an example, and other methods may be used for detection. As an example of another method, when preparing a large amount of training data as described above, training data may be created including finger information such as the positions of the fingertips, the finger crotch, and the base of the fingers, and machine learning may be performed including such information, and the hand contour and finger information may be estimated and output in response to the input of an unknown hand image.

[0065] Next, the posture detection unit 322 calculates the angle of each finger based on the base position and the tip position of each finger detected in step S401 (S402). Specifically, for example, as shown in Fig. 5(b), the posture detection unit 322 determines the line passing through the base 113 and the tip 111 of each finger as the central axis 116 of the finger. The posture detection unit 322 determines the angle of the finger by taking the vertical direction as 0 degrees, and determining that the central axis 116 of the finger rises to the right as positive and that the central axis 116 of the finger falls to the right as negative.

[0066] Next, the posture detection unit 322 calculates the width and length of the middle finger (S403). Specifically, for example, as shown in (c) of Fig. 5, the posture detection unit 322 calculates the distance between two finger wedges 112 adjacent to the middle finger detected in step S401 as the width 117 of the middle finger, and calculates the distance between the base 113 of the middle finger and the tip 111 of the middle finger as the length 118 of the middle finger.

[0067] Next, the posture detection unit 322 calculates the width of the hand (S404). Specifically, for example, the posture detection unit 322 calculates the distance between the contact points 115 of the index finger and the little finger calculated in step S401 as the width 119 of the hand.

[0068] The posture detection unit 322 calculates the angle difference between adjacent fingers based on the angle of each finger calculated in step S402 (S405). Specifically, for example, the posture detection unit 322 calculates the angle difference between the index finger and middle finger, the angle difference between the middle finger and ring finger, and the angle difference between the ring finger and little finger.

[0069] The posture detection unit 322 calculates a first axis ratio that quantifies the roll rotation (S406). The ratio of the width of the hand calculated in step S404 to the vertical width of the middle finger calculated in step S403 is an example of the first axis ratio. The more the hand rolls relative to the imaging device 2, the smaller the width of the hand becomes, but the vertical width of the middle finger is constant. Therefore, the ratio of the vertical width of the middle finger to the horizontal width of the hand becomes larger as the hand is more horizontal relative to the imaging device 2, and becomes smaller as the hand rolls relative to the imaging device 2. In addition, since the first axis ratio is defined using the ratio of the vertical width of the middle finger to the horizontal width of the hand, it is not affected by the size of the hand in the image. Therefore, the first axis ratio can realize the quantification of the roll rotation of the hand relative to the imaging device 2. Note that the width of any finger may be used instead of the width of the hand to calculate the first axis ratio (however, it is desirable to always use the same width of the finger when detecting the posture).

[0070] The posture detection unit 322 calculates a second axis ratio that quantifies the pitch rotation (S407). The ratio of the width and length of the middle finger calculated in step S404 is an example of a second axis ratio. The more the hand rotates pitchwise relative to the imaging device 2, the smaller the length of the middle finger becomes, but the width of the middle finger is constant. Therefore, the ratio of the width and length of the middle finger becomes larger as the hand is more horizontal relative to the imaging device 2, and becomes smaller as the hand rotates pitchwise relative to the imaging device 2. In addition, since the second axis ratio is defined using the ratio of the width and length of the middle finger, it is not affected by the size of the hand in the image. Therefore, the pitch rotation of the hand relative to the imaging device 2 can be quantified by the second axis ratio.

[0071] In the above example, the first axis ratio and the second axis ratio are calculated using the vertical width and horizontal width of the middle finger (i.e., the rotation axis is set on the middle finger), but the first axis ratio and the second axis ratio may be calculated using the vertical width and horizontal width of another finger (the first finger of the thumb, index finger, middle finger, ring finger, or little finger) instead of the middle finger. Also, different vertical widths and horizontal widths of fingers may be used in the calculation of the first axis ratio and the calculation of the second axis ratio. The posture detection unit 322 can calculate the horizontal width and vertical width of other fingers (thumb, index finger, ring finger, and little finger) in the same manner as the calculation of the horizontal width and vertical width of the middle finger.

[0072] In the above example, the rotation axis is set on the middle finger to calculate the first axis ratio and the second axis ratio, but the rotation axis may be set at any point on the finger, palm, or wrist. Specifically, the first axis ratio may be calculated using a line connecting a point on the palm outline or finger outline and a point on the wrist outline as the rotation axis.

[0073] The posture detection unit 322 outputs the angle difference between adjacent fingers, the first axis ratio, and the second axis ratio calculated in steps S405 to S407 as posture information of the input hand image (S408), and ends the posture detection process.

[0074] Note that the order of steps S402 to S407 may be changed as long as the information necessary for each step is obtained. For example, step S405 may be executed immediately after step S402, or step S405 may be executed after step S407.

[0075] Fig. 6 is an explanatory diagram showing an example of a guide displayed on the display device 36 to encourage the user to present the hand, change the spacing between the fingers, and rotate the hand up, down, left, and right around the wrist during registration processing. For example, a guidance message 331, a finger spacing guide 332 encouraging the user to change the spacing between the fingers, and a rotation guide 333 encouraging the user to rotate the hand up, down, left, and right are displayed on the display device 36. The user holds his or her hand over and moves it according to the displayed guide, whereby images of hands in various postures are input. In the example of Fig. 6, the user holds up his or her left hand, but the user may hold up his or her right hand.

[0076] The guidance message 331, the finger spacing guide 332, and the rotation guide 333 are each an example, and the contents of the guidance message and the guide are not important as long as the user can visually understand how to present the hand. Also, the finger spacing guide 332 and the rotation guide 333 may be displayed as a still image or a moving image.

[0077] 7 is an example of a guide displayed on the display device 36 to guide the user to correct the posture of the hand during authentication processing. Display device 36 displays the held-up hand, as well as a first axis guide 334 and a second axis guide 335 that visualize the absolute difference between the first axis ratio and the second axis ratio calculated in step S306 of the authentication processing. If it is determined in step S306 that the finger spacing is insufficient (i.e., if it is determined that the angle difference between adjacent fingers is equal to or greater than a predetermined threshold), a guidance message 336 may also be displayed on the display device 36. By correcting the posture of the hand based on the displayed guide, the user can input an image of the hand in a posture that is likely to be successful in biometric authentication.

[0078] Each of the first axis guide 334 and the second axis guide 335 is formed of two rectangles, one with a fixed length and the other with a variable length. In each axis guide, the smaller the absolute difference in the ratio of the axis, the closer the variable length rectangle is to the fixed length rectangle, and when the absolute difference is 0, the rectangle is the same as the fixed length rectangle. The color of the variable length rectangle may change depending on the size. Specifically, the color may be red when the variable rectangle is small (the absolute difference in the ratio is large), and green when the variable rectangle is large (the absolute difference in the ratio is small). By displaying the closeness of the held up hand to the posture of the registered data in a shape or color, the user can visually understand how to correct the hand.

[0079] Figure 7(a) shows an example of a case where the held up hand has no roll or pitch rotation relative to the registered data posture information. When there is no roll or pitch rotation relative to the registered data, the absolute difference on that axis is 0, so the variable rectangle is the same size as the fixed rectangle in the guides for both axes and is displayed in an overlapping position, so only the variable rectangle is visible.

[0080] Fig. 7(b) shows an example of a case where the held up hand has not rolled or pitched relative to the posture information of the registered data, but the spacing between the fingers is insufficient. As in Fig. 7(a), since there is no roll or pitch rotation relative to the registered data, the variable rectangle is the same size as the fixed rectangle. However, since the spacing between the fingers is insufficient, a guidance message 336 is displayed to encourage the user to increase the spacing between the fingers.

[0081] 7(c) is an example in which a roll rotation occurs in the held-up hand relative to the posture information of the registered data, but a pitch rotation does not occur. Because no pitch rotation occurs in the held-up hand relative to the registered data, the variable rectangle of the second axis guide 335 is the same as the fixed rectangle. However, because a roll rotation occurs in the held-up hand relative to the registered data, the variable rectangle of the first axis guide 334 is smaller than the fixed rectangle.

[0082] 7(d) is an example in which the held up hand does not roll relative to the posture information of the registered data, but does pitch. Because the held up hand does not roll relative to the registered data, the variable rectangle of the first axis guide 334 is the same as the fixed rectangle. However, because the held up hand pitches relative to the registered data, the variable rectangle of the second axis guide 335 is smaller than the fixed rectangle.

[0083] The guide displayed to guide the user to correct the hand posture is not limited to the content shown in Fig. 7. For example, if rotation occurs on both axes with respect to the posture information of the registered data, a guide according to the absolute difference in posture on both axes is displayed (for example, if the absolute difference is equal to or greater than a predetermined value, a message urging a large roll rotation or pitch rotation is displayed, and if the absolute difference is equal to or less than a predetermined value, a message urging a small roll rotation or pitch rotation is displayed). As another example, if a roll rotation occurs with respect to the posture information of the registered data and the finger spacing is insufficient, a guidance message 336 may be displayed in addition to the guide on both axes.

[0084] As described above, the biometric authentication system 10 according to this embodiment selects the most suitable image to be used as registration data from multiple still images generated from a captured video based on posture detection during registration, and registers the selected image. During authentication, posture detection and posture determination are used to determine whether the posture information of the held up hand is within a certain range of the posture information in the registration data, and then performs biometric authentication or displays a guide encouraging the user to correct their hand.

[0085] Moreover, the biometric authentication system 10 according to the present embodiment uses the width of the user's hand, the length of the middle finger, and the width of the middle finger in the posture detection process and posture determination process, thereby allowing for variations in hand shape and finger length that differ from user to user, and enabling registration and authentication in the optimal hand posture for each user. Furthermore, the biometric authentication system 10 according to the present embodiment performs posture determination in two axes, making it easy to reproduce the posture at the time of registration during authentication, thereby reducing the burden on the user. EXAMPLES

[0086] In this embodiment, some of the processes included in the registration process and authentication process are different from those in the first embodiment. The biometric authentication system 10 of this embodiment selects and registers images in multiple postures based on hand posture detection during registration, obtains enrollment data that is closest to the hand posture information detected using the hand posture detection during authentication, and performs biometric authentication using the enrollment data. The following mainly describes the differences from the first embodiment.

[0087] 8 is a flowchart showing an example of the registration process. When it is determined that all still images have been processed (S206: YES), the attitude detection unit 322 selects an image that satisfies a predetermined condition from among the unselected images (S807). Specifically, for example, the attitude detection unit 322 selects an image in which the first axis ratio and the second axis ratio are maximum (for example, the sum of the first axis ratio and the second axis ratio is maximum), an image in which the first axis ratio is maximum and the ratio of the second axis ratio to the maximum ratio of the second axis (i.e., the maximum second axis ratio calculated in step S205) is a predetermined value (or the ratio may be within a predetermined range), an image in which the second axis ratio is maximum and the ratio of the first axis ratio to the maximum ratio of the first axis (i.e., the maximum first axis ratio calculated in step S205) is a predetermined value (or the ratio may be within a predetermined range), and an image in which the ratio of the first axis ratio to the maximum ratio of the first axis is within a predetermined range and the ratio of the second axis ratio to the maximum ratio of the second axis is within a predetermined range.

[0088] This specific example is merely an example, and for example, a plurality of values ​​for each of the first axis ratio and the second axis ratio may be set in advance, and posture combinations (i.e., combinations of the first axis ratio and the second axis ratio) may be prepared in advance based on the set values. In this case, the posture detection unit 322 selects an image with a posture (a combination of the first axis ratio and the second axis ratio that is closest to the combination of the first axis ratio and the second axis ratio) that is closest to each of the posture combinations (combinations of the first axis ratio and the second axis ratio) set in advance.

[0089] Also, for example, the posture detection unit 322 may select multiple images based on the increase or decrease of each ratio due to the rotation of the user's hand. The first axis ratio and the second axis ratio become larger as the finger is held horizontally relative to the imaging device 2, and the respective ratios become smaller as the finger is tilted relative to the imaging device 2. Therefore, when the hand is rotated from a state inclined left to a state inclined right relative to the imaging device 2, the first axis ratio increases and becomes maximum, and then decreases. At that time, the posture detection unit 322 can select images with different tilts by selecting an image at a time when the ratio is increasing (for example, an image when the first axis ratio has increased by a predetermined value or more within a predetermined time (or has increased the most)) and an image at a time when the ratio is decreasing (for example, an image when the first axis ratio has decreased by a predetermined value or more within a predetermined time (or has decreased the most)). This is the same when the hand is rotated from right to left in the first axis ratio, and the ratio also increases or decreases when the hand is rotated up and down in the second axis ratio.

[0090] Also, for example, the posture detection unit 322 may select multiple images based on the width of each finger of the user, or the width of the fingertip side and the width of the finger base side. For example, the width of the finger may be defined as the width of the connection between the finger and the palm, or the distance between adjacent finger crotches or the contact points 115 shown in FIG. 5(a) may be defined as the width of the finger. Also, the width of the fingertip side and the width of the finger base side may be defined as the width at a position where a certain percentage of the finger length is moved from the fingertip to the finger base, and the distance between two points on the hand outline when the certain percentage of the distance is moved in each direction from the fingertip can be calculated as the respective widths. For example, the widths at 1 / 4 and 3 / 4 of the finger length from the fingertip may be defined as the width of the fingertip side and the width of the finger base side, respectively.

[0091] When the hand is rolling relative to the imaging device 2, one of the index finger or the little finger moves closer to the imaging device 2, and the other finger moves away from the imaging device 2. Therefore, the posture detection unit 322 can select multiple images with different inclinations on the first axis based on the ratio between the width of the index finger and the width of the little finger (for example, the posture detection unit 322 selects an image in which the ratio of the width of the little finger to the width of the index finger is equal to or greater than a predetermined value, and an image in which the ratio is less than the predetermined value).

[0092] Similarly, when the hand is pitch-rotating with respect to the imaging device 2, one of the fingertip side or the base side moves closer to the imaging device 2, and the other moves away from the imaging device 2. Therefore, the posture detection unit 322 can select multiple images with different inclinations on the second axis based on the ratio of the width of the fingertip side to the width of the base side of a specific finger, for example, the middle finger (for example, the posture detection unit 322 selects an image in which the ratio of the width of the base side of the middle finger to the width of the fingertip side is equal to or greater than a predetermined value, and an image in which the ratio is less than a predetermined value).

[0093] The posture detection unit 322 executes the processes of steps S208 to S211 on the image selected in step S807. The posture detection unit 322 determines whether a sufficient amount of images in various postures have been registered (S812).

[0094] Specifically, for example, the posture detection unit 322 may determine whether a sufficient amount of images have been registered based on the number of registered posture combinations (for example, determining whether images corresponding to each combination of a predetermined first axis ratio and a predetermined second axis ratio have been registered), or may determine whether a sufficient amount of images have been registered based on the number of registered data (images).

[0095] For example, in the case where nine posture combinations are predefined, the posture detection unit 322 determines that a sufficient amount of images have been registered if it determines that one or more pieces of data (images) for each posture combination have been registered. Also, for example, the posture detection unit 322 determines that a sufficient amount of images have been registered if it determines that ten or more pieces of data (images) have been registered regardless of posture combination.

[0096] If the orientation detection unit 322 determines that a sufficient amount of images have been registered (S812: YES), it ends the registration process, and if it determines that a sufficient amount of images have not been registered (S812: NO), it returns to step S807. 9 is a flowchart showing an example of the authentication process. For the sake of simplicity, it is assumed that only one still image is generated in step S303.

[0097] Following step S304, the data acquisition unit 522 of the server 5 acquires from the server 5 the registered data that is closest to the posture information detected in step S304 (S905). Specifically, for example, in step S304, the posture detection unit 322 transmits the detected posture information together with the registered data request to the server 5. In step S905, the data acquisition unit 522 of the server 5 acquires from the database 530 data registered with posture information that is closest to the received posture information, and transmits the data to the information processing device 3. At this time, when multiple data are registered with the same posture information, the data acquisition unit 522 transmits all the corresponding registered data to the information processing device 3.

[0098] Following step S905, the process proceeds to step S306, and the subsequent processing is the same as in Fig. 3, so a description thereof will be omitted. However, if the information processing device 3 receives a plurality of enrollment data from the server 5 in step S905, the biometric authentication unit 326 performs biometric authentication on all of the enrollment data in step S310, determines the highest similarity as the final similarity, and determines whether the enrollment template and the authentication template are similar.

[0099] As described above, the biometric authentication system 10 according to this embodiment generates multiple still images from a video captured during enrollment, and selects and enrolls one or more images in each of multiple postures based on the hand posture detection. Furthermore, during authentication, the biometric authentication system 10 according to this embodiment obtains enrollment data that is closest to the posture information of the held-up hand using posture detection, determines whether the posture information of the enrollment data is within a certain range using posture determination, and displays a guide that prompts the user to perform biometric authentication or correct the hand image.

[0100] The biometric authentication system 10 of this embodiment registers data of a plurality of postures and performs posture determination based on the most similar registered data, thereby making it easier to reproduce the posture at the time of registration, and thus further reducing the burden on the user. EXAMPLES

[0101] In this embodiment, an authentication system using biometric information captured by a camera mounted on a mobile terminal 300 such as a smartphone or tablet will be described. The configuration of the mobile terminal 300 is similar to that of the information processing device 3 described in the first embodiment, the camera 200 of the mobile terminal 300 is similar to that of the imaging device 2 in the first embodiment, and the configuration of the online service server 500 is similar to that of the server 5 in the first embodiment.

[0102] 10 shows an example of a system configuration for performing identity authentication when accessing a system for online services such as Internet banking using a mobile terminal 300. The registration process and authentication process executed in this embodiment are similar to those described in the first or second embodiment.

[0103] When a user completes the account issuance procedure to use online services via the mobile terminal 300, the online service server 500 sends an account issuance completion notification to the mobile terminal 300 by email (the account issuance completion notification may also be sent to the user in paper form).

[0104] When the user receives the account issuance completion notice, the user presents a certificate issued by a public institution, such as a driver's license, to the institution that owns the online service server 500 for confirmation, so that the institution that owns the online service server 500 can confirm that the user to whom the account is to be issued is the real person. The account issuance completion notice includes, for example, the download location (such as the URL of an internet site) of an application for using the online service and information required for initial access. The user can log in to the issued account by installing the application in the mobile terminal 300 and inputting information required for initial access into the mobile terminal 300.

[0105] At this time, the application installed on the mobile terminal 300 connects to the online service server 500 via the network 4, generates registration data from the image of the hand by the method shown in the first or second embodiment, and the registration data is stored in the online service server 500. At this time, the user may register both the left and right hands, or only one of the hands.

[0106] Thereafter, when the user wishes to use an online service, he or she launches the application installed on the mobile terminal 300. At this time, the application is connected to the online service server 500 via the network 4.

[0107] When starting the application, the user holds his / her hand over the camera 200 to perform authentication processing by the method shown in the first or second embodiment, and if the authentication is successful, the application notifies the online service server 500 of the success of the authentication. An example of a notification protocol to the online service server 500 when the authentication is successful is an exchange using authentication information defined by the FIDO (Fast IDentity Online) Alliance.

[0108] When the online service server 500 confirms that the authentication was successful, the app becomes able to use the services provided by the online service server 500. When the app notifies the online service server 500 that the necessary use has been completed, the personal authentication expires, and biometric authentication is required again if the app is to use the app again. This makes it possible to prevent the mobile terminal 300 from being misused due to loss or theft.

[0109] The online service system according to the present embodiment allows biometric authentication to be performed using only the general-purpose mobile terminal 300, without using a smartphone or the like equipped with a dedicated biometric authentication sensor. In other words, the online service system according to the present embodiment allows for versatile, convenient, and highly accurate authentication to be performed.

[0110] In this embodiment, registration and authentication are performed using the mobile terminal 300 owned by the user, but since the registration data is stored in the online service server 500, authentication may be performed on the online service server 500. In this case, since the user can perform identity authentication from any mobile terminal 300, authentication can be performed in a similar manner using, for example, a tablet installed in a store.

[0111] The online service system of this embodiment can be used for identity authentication for accessing and using various services such as Internet banking, online shopping, and streaming services. EXAMPLES

[0112] In this embodiment, a payment system using biometric information captured by a camera 200 mounted on a mobile terminal 300 such as a smartphone or tablet will be described. The configuration of the mobile terminal 300 is similar to that of the information processing device 3 described in the first embodiment, the configuration of a tablet terminal 600 installed in a retail store or restaurant is similar to that of the information processing device 3 described in the first embodiment, and the camera 200 of the mobile terminal 300 or the tablet terminal 600 is similar to that of the imaging device 2.

[0113] 11 shows an example of the configuration of an authentication system that performs payment using only biometric information at retail stores such as supermarkets and restaurants. When a user registers as a member of a retail store via a mobile terminal 300, a notice for member registration is sent by email from an online service server 500 to the mobile terminal 300 (the notice for member registration may also be sent to the user in paper form). The notice for member registration includes the location where the app can be downloaded (such as the URL of the internet site) and information required for initial access.

[0114] A user installs the application in the mobile terminal 300 owned by the user and logs in by inputting information required for initial access into the mobile terminal 300. When starting the application, the user holds his / her hand over the camera 200 and registers finger information in the online service server 500 by the method shown in the first or second embodiment.

[0115] Tablet terminals 600 are installed at the cash registers of stores such as retail stores and restaurants, and when a user goes to a retail store or restaurant that is affiliated with the app, the tablet terminal 600 is used to authenticate the user at the time of payment using the method shown in Example 1 or Example 2, and the amount used is paid using a pre-set method.

[0116] The system according to the present embodiment allows users to shop empty-handed even if they forget their wallets. Stores and apps can also recommend products and restaurants based on past purchase and usage histories to encourage purchases and usage. In addition, the mobile terminal 300 owned by the user can register biometric information regardless of time or place, which leads to an increase in the number of users.

[0117] In this embodiment, an example of registering biometric information using a mobile terminal 300 owned by a user has been described, but biometric information may also be registered using a tablet terminal 600 installed in a store. This allows even people who do not own a mobile terminal 300 such as a smartphone to use a payment system that uses biometric authentication.

[0118] The present invention is not limited to the above-mentioned embodiment, and various modifications are included. For example, the above-mentioned embodiment has been described in detail to clearly explain the present invention, and is not necessarily limited to those having all the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0119] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0120] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other.

[0121] In the biometric authentication system 10 according to the present embodiment, the processing by each or a part of the components may be performed by a separate device, and information held by each or a part of the components may be held by a separate device. For example, the processing by the biometric authentication unit 326 may be executed by the server 5, or may be executed by a separate device that is prepared. [Explanation of symbols]

[0122] 3 Information processing device, 5 Server, 31 CPU, 32 Memory, 33 Auxiliary storage device, 34 Communication device, 35 Input device, 36 Output device, 51 CPU, 52 Memory, 53 Auxiliary storage device, 54 Communication device, 55 Input device, 56 Output device, 320 Video input unit, 321 Video processing unit, 322 Posture detection unit, 323 Posture determination unit, 324 Feature extraction unit, 325 Template generation unit, 326 Biometric authentication unit, 521 Data registration unit, 522 Data acquisition unit, 530 Database

Claims

1. An information processing device, A processor and a memory are included. the memory holds an image of a hand; The processor, Calculating a first axis ratio indicating a roll rotation amount of the fingers of the hand based on a length of a first finger and a width of the hand in the image; calculating a second axis ratio indicating a pitch rotation amount of the second finger based on a length and a width of the second finger in the image; determining whether the image is a registration target image based on the first axis ratio and the second axis ratio; an information processing device that outputs, when it is determined that the image is the registration target image, information about the hand and posture information including the first axis ratio and the second axis ratio.

2. 2. The information processing device according to claim 1, An information processing device, wherein the first finger and the second finger are middle fingers.

3. 2. The information processing device according to claim 1, A display device, The processor, The information processing device displays, on the display device, a guide to encourage shooting of an image in which the angle of the image of the hand has been corrected based on at least one of the first axis ratio and the second axis ratio.

4. 2. The information processing device according to claim 1, The processor, Calculating an angle difference between adjacent fingers in the image; determining whether the image is a registration target image based on the angle difference, the first axis ratio, and the second axis ratio; The information processing device, wherein the posture information includes the angle difference.

5. 5. The information processing device according to claim 4, A display device, The processor, The information processing device displays on the display device a guide encouraging the user to take an image with the distance between the fingers adjusted based on the angle difference.

6. 2. The information processing device according to claim 1, The processor, Obtain a hand image for authentication, calculating a first axis ratio for authentication indicating a roll rotation amount of a finger of the authentication hand based on a length of the first finger in the authentication hand image and a width of the authentication hand; calculating an authentication second axis ratio indicating a pitch rotation amount of the finger of the authentication hand based on a length and a width of the second finger in the authentication hand image; comparing the first axis ratio for authentication and the second axis ratio for authentication with the posture information to determine whether the hand image for authentication is an image suitable for authentication; When it is determined that the hand image for authentication is an image suitable for authentication, the information processing device performs authentication based on hand information indicated by the hand image for authentication and the output hand information.

7. 2. The information processing device according to claim 1, the memory holds a plurality of images of hands; The processor, Calculating the first axis ratio for each of the plurality of images based on the length of the first finger and the width of the hand; Calculating the second axis ratio for each of the plurality of images based on the length and width of the second finger; determining hand images in a plurality of poses that satisfy a predetermined condition based on a combination of the first axis ratio and the second axis ratio for each of the plurality of images as registration target images; An information processing device that outputs information about the hands indicated by each of the determined plurality of registration target images, and posture information including the first axis ratio and the second axis ratio for each of the determined plurality of registration target images.

8. The information processing device according to claim 7, The processor, Obtain a hand image for authentication, calculating a first axis ratio for authentication indicating a roll rotation amount of a finger of the authentication hand based on a length of the first finger in the authentication hand image and a width of the authentication hand; calculating an authentication second axis ratio indicating a pitch rotation amount of the finger of the authentication hand based on a length and a width of the second finger in the authentication hand image; comparing the first axis ratio for authentication and the second axis ratio for authentication with posture information of each of the determined plurality of registered images to determine posture information that is closest to a hand posture of the hand image for authentication; An information processing device that performs authentication based on hand information indicated by the authentication hand image and hand information corresponding to the determined posture information.

9. An information processing method by an information processing device, The information processing device includes a processor and a memory, the memory holds an image of a hand; The information processing method includes: The processor calculates a first axis ratio indicating a roll rotation amount of the fingers of the hand based on a length of a first finger in the image and a width of the hand; the processor calculates a second axis ratio indicative of an amount of pitch rotation of the second finger based on a length and a width of the second finger in the image; The processor determines whether the image is a registration target image based on the first axis ratio and the second axis ratio; An information processing method, wherein when the processor determines that the image is the registration target image, it outputs information about the hand and posture information including the first axis ratio and the second axis ratio.

Images