Biological data conversion model generating device, biological data conversion model generating method, and biological data conversion model generating system

Abstract

A trained model is generated to convert biometric data acquired using a method different from that used at the time of enrollment into biometric data suitable for biometric authentication. [Solution] A device for generating a biometric data conversion model acquires one or more first biometric data acquired using a first acquisition method and second biometric data acquired using a second acquisition method, and uses a learning model to convert the second biometric data into biometric data equivalent to the first biometric data.If it is determined that the degree of match between the features of the first biometric data and the features of the converted second biometric data does not satisfy a predetermined condition, it performs learning on the learning model in a way that reduces the degree of match, and if it is determined that the predetermined condition is satisfied, it outputs the learning model as a trained model for converting the second biometric data.

Description

[Technical Field]

[0001] The present disclosure relates to a biological data conversion model generation device, a biological data conversion model generation method, and a biological data conversion model generation system. [Background technology]

[0002] Patent document 1 discloses a biometric authentication device including a teacher data generation unit that generates a plurality of pairs of first biometric images and second biometric images from a plurality of first biometric images and a plurality of second biometric images; a learning data generation unit that extracts features from the first biometric images and the second biometric images using a plurality of different provisional parameters; a calculation unit that calculates the degree of agreement between the features extracted from the paired first biometric images and second biometric images for the plurality of pairs generated by the teacher data generation unit; and an optimal solution determination unit that determines the provisional parameters based on the calculation results of the calculation unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-101390 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure has been devised in consideration of the above-described conventional circumstances, and aims to provide a biometric data conversion model generation device, a biometric data conversion model generation method, and a biometric data conversion model generation system that generate a trained model that converts biometric data acquired using a method different from that used at the time of registration into biometric data that can be biometrically authenticated. [Means for solving the problem]

[0005] The present disclosure relates to a method for acquiring first biometric data of a plurality of biometric parts of a person to be authenticated, the first biometric data being acquired by a first acquisition method. Captured imageand second biometric data of the body part acquired by a second acquisition method different from the first acquisition method. Captured image and an acquisition unit that acquires the second biometric data using a learning model. Captured image the first biometric data acquired by the first acquisition method, Captured image equivalent to image a data conversion unit for converting the first biometric data into data; Captured image and the converted second biometric data. Captured image a calculation unit that calculates a degree of agreement with the feature of the biological part included in the second biological data, and when it is determined that the calculated degree of agreement does not satisfy a predetermined condition, performs learning on the learning model with the degree of agreement being a loss, and when it is determined that the degree of agreement satisfies the predetermined condition, Captured image obtained by the first method The image and a control unit that outputs a trained model that converts the data into biometric data.

[0006] The present disclosure also provides a method for generating a biometric data conversion model, which is performed by one or more processors, and includes: Captured image and second biometric data of the body part acquired by a second acquisition method different from the first acquisition method. Captured image and using a learning model, The captured image the first biometric data acquired by the first acquisition method, The captured image equivalent to image converting the first biometric data into data; The captured image and the converted second biometric data. The captured image and if it is determined that the calculated degree of agreement does not satisfy a predetermined condition, it performs learning on the learning model with the degree of agreement being a loss, and if it is determined that the calculated degree of agreement satisfies the predetermined condition, it performs learning on the learning model with the second biological data. The captured image obtained by the first method The imageA method for generating a biological data conversion model is provided, which outputs a trained model for converting biological data into biological data.

[0007] The present disclosure also provides a method for acquiring first biometric data of a plurality of biometric parts of a person to be authenticated by a first acquisition method. Captured image and second biometric data of the body part acquired by a second acquisition method different from the first acquisition method. Captured image and a conversion model generation device capable of communicating between the first acquisition device and the second acquisition device, wherein the first acquisition device acquires one or more of the first biometric data. The captured image and transmits the second biometric data to the conversion model generation device, The captured image and transmits the second biometric data to the conversion model generation device, and the conversion model generation device converts the second biometric data into the first biometric data acquired by the first acquisition method using a learning model. The captured image equivalent to image converting the first biometric data into data; The captured image and the converted second biometric data. The captured image and if it is determined that the calculated degree of agreement does not satisfy a predetermined condition, it performs learning on the learning model with the degree of agreement being a loss, and if it is determined that the calculated degree of agreement satisfies the predetermined condition, it performs learning on the learning model with the second biological data. The captured image obtained by the first method The image We provide a system for generating a biological data conversion model that outputs a trained model that is converted into data. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to generate a trained model that converts biometric data acquired using a method different from that used at the time of enrollment into biometric data that can be used for biometric authentication. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a block diagram showing a first use case example of a biological data conversion model generation system according to an embodiment; [Figure 2] Flowchart showing a procedure for generating learning data for an information processing device [Figure 3] Flowchart showing a procedure for generating a first trained model in an information processing device [Figure 4] A diagram explaining an example of generating a first trained model [Figure 5] Flowchart showing the procedure for generating a second trained model of an information processing device [Figure 6] A diagram explaining an example of generating a second trained model [Figure 7] Flowchart showing the procedure for generating a third trained model of an information processing device [Figure 8] Diagram explaining an example of generating the third trained model [Figure 9] FIG. 10 is a block diagram showing a second use case example of the biological data conversion model generation system according to the embodiment; [Figure 10] Flowchart showing the procedure for inferring biometric data for authentication using the first to third trained models DETAILED DESCRIPTION OF THE INVENTION

[0010] (Background to this disclosure) There are two methods for acquiring biometric data (e.g., fingerprints, veins, palm prints, etc.) used in biometric authentication: a method for acquiring biometric data from a user's biometric part (e.g., fingers or palm) in a non-contact state, and a method for acquiring biometric data from a user's biometric part in a contact state. However, biometric data acquired in a contact state may have a different shape than biometric data acquired in a non-contact state because pressure is applied to the biometric part when the biometric part is placed (contacted) on the placement surface. For this reason, in biometric authentication using biometric data acquired in different ways, the biometric data for authentication must be used to estimate the biometric data for enrollment, which was acquired in a different way. However, since the features that indicate the individuality of the user contained in the biometric data differ depending on the method for acquiring the biometric data, there is a possibility that the authentication accuracy may decrease.

[0011] Recently, there has been a technology that estimates biometric data acquired in a contact state from biometric data acquired in a non-contact state by performing a simulation in which a 3D model of a biometric part (e.g., a finger) of a real person or a fictional person is pressed against a mounting surface. However, biometric parts have different three-dimensional shapes for each user and for each part (e.g., index finger, middle finger, ring finger, etc.). Therefore, biometric authentication using biometric data estimated using a 3D model of a real person or a fictional person can only improve authentication accuracy when the biometric data has the same three-dimensional shape as the 3D model, making it difficult to improve the authentication accuracy of biometric authentication that is widely performed by a variety of users.

[0012] Cited Document 1 discloses a biometric authentication device that extracts features from biometric images acquired through various operational styles, such as imprint fingerprints collected in advance for criminal investigations or biometric images of latent fingerprints collected at crime scenes. This biometric authentication device performs learning based on the degree of separability between the degree of agreement between feature amounts acquired from the same type of finger and the degree of agreement between feature amounts acquired from different types of fingers, and can be said to aim to more clearly distinguish whether fingerprints collected through different methods are from the same finger or different fingers, and to optimize parameters for extracting fingerprint features regardless of the collection method.

[0013] Therefore, in order to improve the authentication accuracy in biometric authentication using biometric data acquired in different ways, such as contact and non-contact states, a technology is required that can estimate the biometric data for registration with higher accuracy using the biometric data for authentication.

[0014] Hereinafter, with reference to the drawings as appropriate, detailed descriptions will be given of embodiments that specifically disclose the configurations and operations of a biological data conversion model generation device, a biological data conversion model generation method, and a biological data conversion model generation system according to the present disclosure. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of well-known matters or redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter recited in the claims.

[0015] The term "biometric data" in the present disclosure refers to data including features of a biological part used for biometric authentication, and includes, for example, image data of an image of a biological part, data of one or more feature points indicative of individuality in a biological part, or data of feature amounts indicative of individuality acquired from a biological part. The term "feature" in the present disclosure refers to an element used to identify a fingerprint in the authentication process of biometric authentication, and includes, for example, elements such as the pattern or shape of a biological part, feature amounts of a biological part, or feature points of a biological part.

[0016] <First use case example> A first use case example of the biological data conversion model generation system 100 according to the embodiment 1 will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the first use case example of the biological data conversion model generation system 100 according to the embodiment.

[0017] In the first use case example, the biometric data conversion model generation system 100 generates a trained model for converting biometric data used for authentication so that, when authenticating the same biometric part of the same person, features indicating the individual's identity contained in the registered biometric data match features indicating the individual's identity contained in biometric data used for authentication and acquired by a method different from that used for enrollment. The biometric data conversion model generation system 100 includes a first data acquisition device P1, a second data acquisition device P2, and an information processing device S1. Note that the first data acquisition device P1 or the second data acquisition device P2 shown in FIG. 1 may be integrated with the information processing device S1. Furthermore, if the information processing device S1 has previously stored biometric data acquired by the same method as enrollment in the training data storage unit 13, or if it is capable of reading biometric data acquired and stored by the same method as enrollment from another device or storage medium, the first data acquisition device P1 is not essential and may be omitted.

[0018] In this disclosure, a method for generating a conversion model of biometric data when fingerprint authentication is performed will be described as an example of biometric authentication, but the biometric data used for biometric authentication is not limited to fingerprints. Biometric authentication may be, for example, vein authentication or palm print authentication, or authentication using any two or more biometric data of fingerprints, veins, and palm prints.

[0019] The information processing device S1 is connected to a first data acquisition device P1 and a second data acquisition device P2 so as to be able to communicate data with each other. The information processing device S1 is realized by a personal computer (hereinafter referred to as "PC"), a notebook PC, a tablet terminal, a cloud server, an on-premise server, or the like. The information processing device S1 generates a trained model for converting biometric data acquired by the second data acquisition device P2 into biometric data acquired by the first data acquisition device P1 and having characteristics that enable authentication (matching) with pre-registered biometric data. The information processing device S1 includes a communication unit 10, a processor 11, a memory 12, a training data storage unit 13, and a trained model storage unit 14.

[0020] The communication unit 10 is connected to the first data acquisition device P1 and the second data acquisition device P2 so as to be able to communicate with each other wirelessly or via wires. Note that the wireless communication referred to here refers to short-range wireless communication such as Bluetooth (registered trademark) or NFC (registered trademark), or communication via a wireless local area network (LAN) such as Wi-Fi (registered trademark).

[0021] The communication unit 10 outputs to the processor 11 the biometric data acquired by the first data acquisition device P1 (i.e., an image of a fingerprint captured with the fingertip in contact with an object (hereinafter referred to as a "contact image")) or the biometric data acquired by the second data acquisition device P2 (i.e., an image of a fingerprint captured in a non-contact state with the fingertip not in contact with an object (hereinafter referred to as a "non-contact image")).

[0022] The processor 11 is configured using, for example, a central processing unit (hereinafter referred to as "CPU"), a field programmable gate array (hereinafter referred to as "FPGA"), or a graphics processing unit (hereinafter referred to as "GPU"), and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11 references the programs and data stored in the memory 12 and executes the programs to realize the functions of the learning unit 111 and the feature point data generation unit 112, etc., and generate a trained model. Note that the feature point data generation unit 112 is not an essential component and may be omitted.

[0023] To enable authentication of features included in the biometric data acquired by the second data acquisition device P2 with features included in the biometric data acquired by the first data acquisition device P1, the learning unit 111 uses a learning model that converts the biometric data acquired by the second data acquisition device P2 to perform learning using multiple pairs of learning data stored in the learning data storage unit 13 (described later) or multiple pairs of learning data included in at least one learning data set. The learning unit 111 repeats learning with a loss of matching until the degree of matching between the features included in the biometric data acquired by the second data acquisition device P2 and the features included in the biometric data acquired by the first data acquisition device P1 satisfies a predetermined condition. When the learning unit 111 determines that the degree of matching between the features of the biometric data converted by the learning model and the features of the biometric data acquired by the first data acquisition device P1 satisfies the predetermined condition, it terminates learning of the learning model and outputs this learning model to the trained model storage unit 14 as a trained model for storage.

[0024] The feature point data generation unit 112 generates feature points of a body part from biometric data. When learning is performed using the feature points of a body part, the feature point data generation unit 112 generates the feature points of the body part and saves (stores) the generated feature points of the body part in the learning data storage unit 13 as learning data.

[0025] The memory 12 includes, for example, a random access memory (hereinafter referred to as "RAM") as a work memory used when executing each process of the processor 11, and a read only memory (hereinafter referred to as "ROM") that stores programs and data that define the operation of the processor 11. The RAM temporarily stores data or information generated or acquired by the processor 11. The ROM stores programs that define the operation of the processor 11.

[0026] The learning data storage unit 13 stores learning data used for learning. The learning data storage unit 13 stores (saves) information about the biometric data of different people acquired by the second data acquisition device P2 (for example, the biometric data itself or the file name of the biometric data) and information about the biometric data of different people acquired by the first data acquisition device P1 (for example, the biometric data itself or the file name of the biometric data). The learning data storage unit 13 may be configured separately from the information processing device S1 and realized by an external storage medium capable of data communication with the information processing device S1.

[0027] Here, the biometric data (learning data) stored in the learning data storage unit 13 and used to generate a learning model will be described. The learning data storage unit 13 associates one piece of biometric data acquired by the second data acquisition device P2 with one or more pieces of biometric data acquired by the first data acquisition device P1, and stores these pieces of biometric data as pairs of learning data to be used in one learning process. For example, in the example shown in FIG. 4, the learning data storage unit 13 stores biometric data having the file name "001-LEFT-INDEX-CL.png" and biometric data having the file name "001-LEFT-INDEX-C.png" as a first pair, and biometric data having the file name "002-RIGHT-RING-CL.png" and biometric data having the file name "002-RIGHT-RING-C.png" as a second pair.

[0028] Furthermore, the training data storage unit 13 may store multiple training data sets as one training data set including multiple training data pairs used for generating (training) a trained model. For example, the training data storage unit 13 in the example shown in FIG. 4 stores the above-described first pair and second pair as one training data set. Furthermore, multiple training data sets may be set; for example, a first training data set including three training data pairs and a second training data set including five training data pairs may be set. Note that the setting of the training data pairs and the setting of the training data sets may be performed by any person or may be performed automatically.

[0029] The trained model storage unit 14 stores (preserves) the trained model generated by the processor 11.

[0030] The first data acquisition device P1 is connected to the information processing device S1 so as to be able to communicate data with the information processing device S1. The first data acquisition device P1 captures and reads an image of a biological part in a state where the biological part is in contact with an object (e.g., a glass surface, etc.). The first data acquisition device P1 includes a communication unit 20, a processor 21, and a memory 22.

[0031] The communication unit 20 is connected to the information processing device S1 so as to be able to communicate wirelessly or via wire with the information processing device S1. The communication unit 20 transmits the biometric data (contact data) output from the processor 21 to the information processing device S1.

[0032] The processor 21 is configured using, for example, a CPU, FPGA, or GPU, and performs various processes and controls in cooperation with the memory 22. Specifically, the processor 21 references programs and data stored in the memory 22 and executes the programs to realize a function for acquiring biometric data of a biometric part in a state of contact with an arbitrary object. The processor 21 outputs the biometric data (contact data) output from the contact data acquisition unit 23 to the communication unit 20, and causes it to be transmitted to the information processing device S1.

[0033] The memory 22 includes, for example, a RAM as a work memory used when executing each process of the processor 21, and a ROM for storing programs and data that define the operation of the processor 21. The RAM temporarily stores data or information generated or acquired by the processor 21. The ROM has written therein programs that define the operation of the processor 21.

[0034] The contact data acquisition unit 23 is realized by, for example, a sensor or an optical system including a lens and an image sensor, and acquires biometric data of a biological part in contact with an arbitrary object. The contact data acquisition unit 23 outputs the acquired biometric data (contact data) to the processor 21. Note that, in the present disclosure, an example is shown in which the contact data acquisition unit 23 is mainly configured by an optical system and captures an image of a biological part, but a configuration using a sensor is not excluded.

[0035] The second data acquisition device P2 is connected to the information processing device S1 so as to be able to communicate data with it. The second data acquisition device P2 is realized, for example, by a visible light camera device or any device equipped with a camera (for example, a smartphone or a tablet terminal). The second data acquisition device P2 captures images of a biological part in a non-contact state where the biological part is not in contact with any object. The second data acquisition device P2 includes a communication unit 30, a processor 31, and a memory 32.

[0036] The communication unit 30 is connected to the information processing device S1 so as to be able to communicate wirelessly or via wires. The communication unit 30 transmits the biometric data (contactless data) output from the processor 31 to the information processing device S1.

[0037] The processor 31 is configured using, for example, a CPU, FPGA, or GPU, and performs various processes and controls in cooperation with the memory 32. Specifically, the processor 31 references the programs and data stored in the memory 32 and executes the programs to realize a function for acquiring biometric data of a body part that is in a non-contact state with any object. The processor 31 outputs the biometric data (non-contact data) output from the non-contact data acquisition unit 33 to the communication unit 30, and causes it to be transmitted to the information processing device S1.

[0038] The memory 32 includes, for example, a RAM as a work memory used when the processor 31 executes each process, and a ROM for storing programs and data that define the operation of the processor 31. The RAM temporarily stores data or information generated or acquired by the processor 31. The ROM stores programs that define the operation of the processor 31.

[0039] The non-contact data acquisition unit 33 is realized by, for example, an optical system including a lens and an image sensor, and captures biometric data of a biometric part that is in a non-contact state with any object. The non-contact data acquisition unit 33 outputs the acquired biometric data (non-contact data) to the processor 31.

[0040] In the following explanation, when biometric data acquired by different methods is used for enrollment and authentication of biometric authentication, a method for generating each trained model (specifically, a first trained model, a second trained model, and a third trained model) that converts contactless data so that the state of features contained in contactless data (biometric data) acquired by the same method as for authentication corresponds to the state of enrolled or contact data (biometric data) acquired by the same method as for enrollment will be described. In addition, in the following explanation, for ease of understanding, fingerprint authentication will be assumed, and an example in which the biometric part is the fingertip (fingerprint) will be specifically described.

[0041] Here, the first trained model is a trained model for converting an image of a fingerprint captured when the fingertip, which is a biological part, is not in contact with any object (hereinafter referred to as a "non-contact image"), and is a trained model for converting a non-contact image into an image of a fingerprint captured when the fingertip is in contact with any object (hereinafter referred to as a "contact image").

[0042] In addition, the second trained model is a trained model for converting contactless images, and is a trained model for converting contactless images into fingerprint feature points in a state where the fingerprint is in contact with an object (hereinafter referred to as "contact feature points").

[0043] In addition, the third trained model is a trained model for converting fingerprint feature points (hereinafter referred to as "non-contact feature points") when the fingertip, which is a biological part, is not in contact with any object, and is a trained model for converting non-contact feature points into contact feature points.

[0044] In the following explanation, the generation method of each trained model will be explained in the order of the first trained model, the second trained model, and the third trained model.

[0045] <How to generate the first trained model> First, a method for generating the first trained model will be described. The information processing device S1 uses a non-contact image and a contact image to generate a first trained model for converting an input non-contact image into a non-contact image that corresponds to the state of the contact image.

[0046] First, a method for generating the first trained model, that is, a method for generating training data Dt10 (see FIG. 4) used for training the training model MD, will be described with reference to FIG. 2. FIG. 2 is a flowchart showing the procedure for generating training data Dt10 to Dt30 by the information processing device S1. Note that the processing of steps St11 to St12 may be repeatedly executed based on the number of pairs of training data included in the training data Dt10.

[0047] Note that the procedures for generating each of the training data shown in FIG. 2 include procedures for generating each of the training data Dt10 to Dt30 used to generate the first to third trained models, respectively. However, here, only the method for generating the training data Dt10 used to generate the first trained model will be described.

[0048] The processor 11 of the information processing device S1 acquires at least one contact image Dt131-Dt13N acquired by the first data acquisition device P1 and a non-contact image Dt11 acquired by the second data acquisition device P2 (St11). Note that the timing of capturing each image by each data acquisition device may be arbitrary.

[0049] When generating the first trained model, the processor 11 associates the non-contact image Dt11 and the contact images Dt131 to Dt13N of the body parts of the same person, and stores them in the training data storage unit 13 (St12).

[0050] Here, processor 11 associates one non-contact image Dt11 with one or more contact images Dt131 to Dt13N, and stores the associated pair of training data in training data storage unit 13. That is, when there are multiple non-contact images, processor 11 generates multiple pairs of training data, each associated with one or more contact images Dt131 to Dt13N, and stores the pairs in training data storage unit 13. Processor 11 may also store two or more pairs of training data from the pairs of training data stored in training data storage unit 13 as a set of training data used to generate a first trained model.

[0051] As a result, the information processing device S1 generates learning data Dt10 including information on a plurality of pairs or sets of learning data. In the example shown in FIG. 4, the learning data Dt10 stored in the learning data storage unit 13 is managed as a pair of learning data, with a person ID capable of identifying the person whose biological part appears in the non-contact image and the contact image, information on the biological part appearing in the non-contact image and the contact image, and the file name of the non-contact image linked together. However, the learning data Dt10 is not limited to this example. It is sufficient that the file name of the non-contact image and the file name of the contact image are linked together.

[0052] Next, a method for generating a first trained model, that is, a training method using the training model MD, will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a flowchart showing the procedure for generating a first trained model of the information processing device S1. Fig. 4 is a diagram for explaining an example of generating a first trained model.

[0053] The learning unit 111 of the information processing device S1 performs learning using all pairs of learning data or all sets of learning data that are stored in the learning data storage unit 13 and that have been set in advance as learning data Dt10 to be used for learning (St20). Specifically, when N (an integer equal to or greater than 2) pairs of learning data or M (an integer equal to or greater than 1) sets of learning data are stored in the learning data storage unit 13, the learning unit 111 performs the processes of steps St21 to St23 for each of the N pairs of learning data or the M sets of learning data. That is, the learning unit 111 repeatedly performs the processes of steps St21 to St23 a number of times corresponding to the number of pairs or sets of learning data. Note that an example of learning processing using learning data Dt10 including N pairs of learning data will be described here.

[0054] The learning unit 111 acquires a non-contact image Dt11 of a biological part from the learning data Dt10 stored in the learning data storage unit 13, and at least one each of contact images Dt131 to Dt13N linked to the non-contact image Dt11 (St21).

[0055] The learning unit 111 uses the learning model MD to convert the non-contact image Dt11 into a non-contact image Dt12 so that the non-contact image Dt11 corresponds to the state of the biological part shown in the contact images Dt131 to Dt13N (for example, how the biological part is crushed, how the feature points or features change, etc.) (St22).

[0056] The learning unit 111 calculates the degree of match between the converted non-contact image Dt12 and each of the contact images Dt131 to Dt13N for each combination of non-contact feature points and contact feature points. The learning unit 111 executes learning with the calculated degree of match as a loss, and updates the learning model MD (St23). Note that the combinations referred to here refer to all three possible combinations of non-contact and contact images, for example, when one non-contact image is paired with three contact images.

[0057] 4, the learning data Dt10 includes information on N pairs of learning data. The learning unit 111 executes a first learning process (the processing of steps St21 to St23) using, as a first learning data pair, a non-contact image having the file name "001-LEFT-INDEX-CL.png" linked to information on the biological parts "LEFT (hand)" and "INDEX (finger)" of the person ID "001" and a contact image having the file name "001-LEFT-INDEX-C.png", and executes a second learning process (the processing of steps St21 to St23) using, as a second learning data pair, a non-contact image having the file name "002-RIGHT-RING-CL.png" linked to information on the biological parts "RIGHT (hand)" and "RING (finger)" of the person ID "002" and a contact image having the file name "002-RIGHT-RING-C.png". The learning unit 111 repeats this process to execute the Nth learning process (the processes of steps St21 to St23).

[0058] Based on all the degrees of match calculated in the processing of step St20 (i.e., N learning processes), the learning unit 111 determines whether the degrees of match satisfy predetermined conditions and whether the learning model MD is capable of generating a non-contact image Dt12 (St24).

[0059] The predetermined condition here is a condition that the average value of the degree of match is equal to or greater than a predetermined value. The predetermined value may be any value that is determined to be sufficiently high. This allows the learning unit 111 to generate a first trained model that can achieve conversion (inference) with higher consistency and stability for various contact images in the conversion (inference) process of non-contact images using the first trained model.

[0060] If the learning unit 111 determines in step St24 that the degree of coincidence satisfies a predetermined condition (St24, YES), it stores the updated learning model MD in the learned model storage unit 14 as a first learned model (St25).

[0061] On the other hand, if the learning unit 111 determines in step St24 that the degree of match does not satisfy the predetermined condition (St24, NO), it returns to the processing of step St20 and re-executes the processing of steps St21 to St23 using all pairs or sets of learning data included in the learning data Dt10, thereby updating the learning model MD again.

[0062] As a result, the information processing device S1 can generate and acquire a first trained model capable of generating a non-contact image (converted non-contact image) in which the state of a biological part shown in a non-contact image acquired in a non-contact state is transformed into a state corresponding to the state of the biological part shown in a contact image acquired in a contact state (e.g., how the biological part is crushed, how feature points or features change, etc.). Furthermore, because the first trained model of the present disclosure is trained using biometric data of multiple people, it is possible to infer biometric data corresponding to the three-dimensional shapes of various users' biological parts (e.g., fingers) and various contact states (e.g., pressure, direction of pressure, position of pressure).

[0063] That is, even if the three-dimensional shape of a biological part differs for each type of person or biological part and the biological part is deformed by any pressure applied to the biological part during enrollment, the information processing device S1 can use the first trained model to convert the biometric data (contactless image) acquired during authentication into data (contactless image) having biometric data corresponding to the state of the biometric data (contact image) registered during enrollment (for example, how the biological part is crushed, how the feature points or features change, etc.). Thus, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods.

[0064] <How to generate the second trained model> Next, a method for generating the second trained model will be described. The information processing device S1 generates contact feature points of a fingerprint from each contact image. The information processing device S1 generates a second trained model for converting an input non-contact image into non-contact feature points corresponding to the contact feature points generated from the contact image.

[0065] First, a method for generating a second trained model, that is, a method for generating training data Dt20 (see FIG. 6) used for training the training model MD, will be described with reference to FIG. 2. Note that the processing of steps St11 to St12 may be repeatedly executed based on the number of pairs of training data included in the training data Dt20.

[0066] The processor 11 of the information processing device S1 acquires at least one contact image Dt231-Dt23N acquired by the first data acquisition device P1 and a non-contact image Dt21 acquired by the second data acquisition device P2 (St11). Note that the timing of capturing each image by each data acquisition device may be arbitrary.

[0067] The processor 11 associates the non-contact image Dt21 and the contact images Dt231 to Dt23N, which are of the same person and show the same body part (for example, the middle finger of the right hand), and stores them in the learning data storage unit 13 (St12).

[0068] Here, processor 11 associates one non-contact image Dt21 with one or more contact images Dt231 to Dt23N and stores the associated pair of learning data in learning data storage unit 13. In other words, when there are multiple non-contact images, processor 11 associates each non-contact image with one or more contact images Dt231 to Dt23N and stores the associated pair of learning data in learning data storage unit 13. Processor 11 may also store two or more pairs of learning data among the pairs of learning data stored in learning data storage unit 13 as a set of learning data used to generate a second trained model.

[0069] As a result, the information processing device S1 generates learning data Dt20 including information on a plurality of pairs or sets of learning data. 6 shows an example in which the learning data Dt20 stored in the learning data storage unit 13 is managed as a pair of learning data, with a person ID capable of identifying the person who owns the biological part shown in the non-contact image and the contact image, information on the biological part shown in the non-contact image and the contact image, and the file name of the non-contact image and the file name of the contact image being linked together, but is not limited to this.

[0070] Processor 11 may perform image analysis on each of contact images Dt231 to Dt23N as shown in Fig. 6, and generate fingerprint contact feature point information from each of contact images Dt231 to Dt23N. In such a case, processor 11 generates training data (for example, training data Dt20 shown in Fig. 6) in which each contactless image is further associated with the file name of the contact feature points of the fingerprint (biological part) in a contact state, which is generated from at least one contact image, or generates training data in which the file name of the contactless image is associated with the file name of the contact feature points instead of the file name of the contact image, and stores this in training data storage unit 13. It is sufficient that the file name of the contactless image is associated with the file name of the contact image or the file name of the contact feature point information in training data Dt20.

[0071] Next, a method for generating a second trained model, that is, a training method using the training model MD, will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing the procedure for generating a second trained model of the information processing device S1. Fig. 6 is a diagram illustrating an example of generating a second trained model.

[0072] The learning unit 111 of the information processing device S1 performs learning using all pairs of learning data or all sets of learning data that are stored in the learning data storage unit 13 and that have been set in advance as learning data Dt20 to be used for learning (St30). Specifically, when N pairs of learning data or M sets of learning data are stored in the learning data storage unit 13, the learning unit 111 performs the processes of steps St31 to St34 for each of the N pairs of learning data or the M sets of learning data. That is, the learning unit 111 repeatedly performs the processes of steps St31 to St34 a number of times corresponding to the number of pairs or sets of learning data. Note that here, an example of learning processing using learning data Dt20 including N pairs of learning data will be described.

[0073] The learning unit 111 acquires a non-contact image Dt21 of a biological part from the learning data Dt20 stored in the learning data storage unit 13, and at least one each of contact images Dt231 to Dt23N linked to the non-contact image Dt21 (St31).

[0074] The learning unit 111 may acquire the information on the contact feature points when the information on the contact feature points generated from each of the contact images Dt231 to Dt23N is stored in the learning data Dt20 stored in the learning data storage unit 13. In such a case, the processing of step St33 may be omitted.

[0075] The learning unit 111 uses the learning model MD to convert the non-contact image Dt21 into non-contact feature points Dt22 so that the non-contact image Dt21 becomes non-contact feature points corresponding to the state of the contact feature points Dt241 to Dt24N (for example, a change in the feature points or a change in the positional relationship between the feature points, etc.) (St32).

[0076] In addition, here, in the processing of step St30, if the processing of step St32 has not yet been executed even once and contact feature points Dt241 to Dt24N have not yet been generated, the learning unit 111 estimates the states of the contact feature points and, based on the non-contact image and the estimated states of the contact feature points, converts the non-contact image into non-contact feature points having changes in feature points or changes in the positional relationships between feature points corresponding to the estimated states of the contact feature points. Furthermore, the learning unit 111 may execute the processing of step St32 after executing the processing of step St33.

[0077] The minutia data generator 112 generates fingerprint contact minutiae Dt241 to Dt24N from each of the contact images Dt231 to Dt23N (St33).

[0078] The learning unit 111 calculates the degree of match between the converted non-contact feature point Dt22 and each of the contact feature points Dt241 to Dt24N for each combination of a non-contact feature point and a contact feature point. The learning unit 111 performs learning in which all of the degrees of match calculated in the process of step St30 (i.e., N times of learning processing) are treated as losses, and updates the learning model MD (St34).

[0079] 6, the learning data Dt20 includes information on N pairs of learning data. The learning unit 111 executes the first learning process (the processes of steps St31 to St34) using, as the first learning data pair, a non-contact image having the file name "001-LEFT-INDEX-CL.png" linked to information on the biological parts "LEFT (hand)" and "INDEX (finger)" of the person ID "001", and a contact image having the file name "001-LEFT-INDEX-C.png" or contact feature point information having the file name "001-LEFT-INDEX-C.feat". Furthermore, the learning unit 111 executes a second learning process (the processing of steps St31 to St34) using, as a second learning data pair, a non-contact image having the file name "002-RIGHT-RING-CL.png" linked to information on the biological parts "RIGHT (hand)" and "RING (finger)" of the person ID "002", and a contact image having the file name "002-RIGHT-RING-C.png", or contact feature point information having the file name "002-RIGHT-RING-C.feat". The learning unit 111 repeats this to execute an Nth learning process (the processing of steps St31 to St34).

[0080] The learning unit 111 determines whether the degrees of match calculated in the process of step St30 (i.e., N times of learning process) satisfy a predetermined condition and whether the learning model MD is capable of generating the non-contact feature point Dt22 (St35). Note that the predetermined condition here is the same as the predetermined condition described in the generation of the first trained model, and therefore a description thereof will be omitted.

[0081] If the learning unit 111 determines in step St35 that the degree of coincidence satisfies a predetermined condition (St35, YES), it stores the updated learning model MD in the learned model storage unit 14 as a second learned model (St36).

[0082] On the other hand, if the learning unit 111 determines in step St35 that the degree of match does not satisfy the predetermined condition (St35, NO), it returns to the processing of step St30 and re-executes the processing of steps St31 to St34 using all pairs or sets of learning data included in the learning data Dt20, thereby updating the learning model MD again.

[0083] As a result, the information processing device S1 can generate and acquire a second trained model capable of generating non-contact feature points (converted non-contact feature points) by changing the state of a biological part shown in a non-contact image acquired in a non-contact state to a state corresponding to the state of a contact feature point acquired and generated in a contact state (e.g., a change in a feature point or a change in the positional relationship between feature points). Furthermore, because the second trained model of the present disclosure is trained using biometric data of multiple people, it becomes possible to infer biometric data corresponding to the three-dimensional shapes of various users' biological parts (e.g., fingers) and various contact states (e.g., pressure, direction of pressure, position of pressure).

[0084] That is, even if the three-dimensional shape of a biological part differs for each type of person or biological part and the biological part is deformed by any pressure applied to the biological part during enrollment, the information processing device S1 can use the second trained model to convert the biometric data (contactless image) acquired during authentication into biometric data (contactless feature points) corresponding to the state of the biometric data (contact feature points) registered during enrollment. Thus, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods.

[0085] <Third method for generating trained models> Next, a method for generating the third trained model will be described. The information processing device S1 generates non-contact feature points from each non-contact image, and generates contact feature points from each contact image. The information processing device S1 generates a third trained model for converting the states of the generated non-contact feature points into non-contact feature points equivalent to the states of the generated contact feature points.

[0086] First, a method for generating the third trained model, that is, a method for generating training data Dt30 (see FIG. 8) used for training the training model MD, will be described with reference to FIG. 2. Note that the processing of steps St11 to St12 may be repeatedly executed based on the number of pairs of training data included in the training data Dt30.

[0087] The processor 11 acquires at least one contact image Dt331-Dt33N acquired by the first data acquisition device P1 and a non-contact image (not shown) acquired by the second data acquisition device P2 (St11). Note that the timing of capturing each image by each data acquisition device may be arbitrary.

[0088] When generating the third trained model, processor 11 performs image analysis on each of the non-contact image (not shown) and contact images Dt331 to Dt33N, and generates information on fingerprint feature points (non-contact feature points and contact feature points) from each of the non-contact image (not shown) and contact images Dt331 to Dt33N (St13).

[0089] The processor 11 associates the non-contact feature point Dt31 and the contact feature points Dt341 to Dt34N, which are feature points of the same person, and stores them in the learning data storage unit 13 as a pair of learning data (St14).

[0090] As a result, the information processing device S1 generates learning data Dt30 including information on a plurality of pairs or sets of learning data. The learning data Dt30 stored in the learning data storage unit 13 shows an example in which a person ID capable of identifying the person who owns the biological part shown in the non-contact image and the contact image, information on the biological part shown in the non-contact image and the contact image, the file name of the non-contact image, and the file name of the contact image are linked together and managed as a single pair of learning data, but is not limited to this.

[0091] The training data used to generate the third trained model may be generated using the same procedure as for the first trained model, as long as the file names of the non-contact images and the contact images are linked together. In this case, the training unit 111 executes the process of generating non-contact feature points and contact feature points in the flowchart shown in FIG.

[0092] Furthermore, the learning data used to generate the third trained model may be linked to the file name of the non-contact image, the file name of the non-contact feature point information, the file name of the contact image, and the file name of the contact feature point information, as shown in FIG. 8.

[0093] Next, a method for generating a third trained model, that is, a training method using the training model MD, will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a flowchart showing the procedure for generating a third trained model of the information processing device S1. Fig. 8 is a diagram illustrating an example of generating a third trained model.

[0094] The learning unit 111 of the information processing device S1 performs learning using all pairs of learning data or all sets of learning data that are stored in the learning data storage unit 13 and that have been set in advance as learning data Dt30 to be used for learning (St40). Specifically, when N pairs of learning data or M sets of learning data are stored in the learning data storage unit 13, the learning unit 111 performs the processes of steps St41 to St44 for each of the N pairs of learning data or the M sets of learning data. That is, the learning unit 111 repeatedly performs the processes of steps St41 to St44 a number of times corresponding to the number of pairs or sets of learning data. Note that here, an example of learning processing using learning data Dt20 including N pairs of learning data will be described.

[0095] The learning unit 111 acquires a non-contact image (not shown) of a biological part from the learning data Dt30 stored in the learning data storage unit 13, and at least one contact image Dt331 to Dt33N linked to this non-contact image (not shown) (St41).

[0096] The feature data generation unit 112 generates non-contact feature points Dt32 of the fingerprint from the non-contact image (not shown) and contact feature points Dt341 to Dt34N of the fingerprint from the contact images Dt331 to Dt33N, respectively (St42). Note that the processing of step St42 may be executed only when the training data Dt30 does not include non-contact feature point information and contact point information.

[0097] The learning unit 111 uses the learning model MD to convert the non-contact feature point Dt31 into the non-contact feature point Dt32 corresponding to the state of the contact feature points Dt341 to Dt34N (for example, the three-dimensional shape of the biological part estimated from the contact feature point, how it is crushed, or how the feature points change) (St43).

[0098] The learning unit 111 calculates the degree of match between the converted non-contact feature point Dt32 and each of the contact feature points Dt341 to Dt34N for each combination of the non-contact feature point and the contact feature point. The learning unit 111 executes learning with the calculated degree of match as a loss, and updates the learning model MD (St44).

[0099] 8, the learning data Dt30 includes information on N pairs of learning data. The learning unit 111 executes a first learning process (the processing of steps St41 to St43) using, as a first learning data pair, a non-contact image having the file name "001-LEFT-INDEX-CL.png" linked to information on the biological parts "LEFT (hand)" and "INDEX (finger)" of the person ID "001" and a contact image having the file name "001-LEFT-INDEX-C.png", and executes a second learning process (the processing of steps St41 to St43) using, as a second learning data pair, a non-contact image having the file name "002-RIGHT-RING-CL.png" linked to information on the biological parts "RIGHT (hand)" and "RING (finger)" of the person ID "002" and a contact image having the file name "002-RIGHT-RING-C.png". The learning unit 111 repeats this process to execute the Nth learning process (the processes of steps St41 to St43).

[0100] The learning unit 111 determines whether the degrees of match calculated in the process of step St40 (i.e., N times of learning process) satisfy a predetermined condition and the learning model MD is capable of generating the non-contact feature point Dt32 (St45). Note that the predetermined condition here is the same as the predetermined condition described in the generation of the first trained model, and therefore a description thereof will be omitted.

[0101] If the learning unit 111 determines in step St45 that the degree of match satisfies a predetermined condition (St45, YES), it stores the updated learning model MD in the learned model storage unit 14 as a third learned model (St46).

[0102] On the other hand, if the learning unit 111 determines in step St45 that the degree of match does not satisfy the predetermined condition (St45, NO), it returns to the processing of step St40 and re-executes the processing of steps St41 to St44 using all pairs or sets of learning data included in the learning data Dt30, thereby updating the learning model MD again.

[0103] As a result, the information processing device S1 can generate and acquire a third trained model capable of generating non-contact feature points (converted non-contact feature points) obtained by transforming the state of a biological part indicated by a non-contact feature point acquired in a non-contact state into a state corresponding to the state of a biological part indicated by a contact feature point acquired in a contact state (e.g., how the biological part is crushed, how the feature point or feature changes, etc.). Furthermore, because the third trained model of the present disclosure is trained using biometric data of multiple people, it becomes possible to infer biometric data corresponding to the three-dimensional shapes of various users' biological parts (e.g., fingers) and various contact states (e.g., pressure, direction of pressure, position of pressure).

[0104] That is, even if the three-dimensional shape of a biometric part differs for each type of person or biometric part and the biometric part is deformed by any pressure applied to the biometric part during enrollment, the information processing device S1 can use the third trained model to convert the biometric data (non-contact feature points) acquired during authentication into biometric data (non-contact feature points) corresponding to the state of the biometric data (contact feature points) registered during enrollment. Thus, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods.

[0105] <Second use case example> The above explanation has described the methods for generating each of the first to third trained models. Therefore, in the following explanation, with reference to Figures 9 and 10, the overall configuration of biometric data conversion model generation system 100A when inferring (generating) a contactless image or contactless feature points used during authentication using each of the generated first to third trained models will be described, along with a method for inferring (generating) a contactless image or contactless feature points used during authentication.

[0106] 9 is a block diagram showing a second use case example of the biological data conversion model generation system 100A according to the embodiment. Note that the overall configuration of the biological data conversion model generation system 100A shown in FIG. 9 is just an example and is not intended to be limiting.

[0107] For example, the biological data conversion model generation system 100A may be realized by replacing the processor 11 of the information processing device S1 shown in Fig. 1 with the processor 11A of the information processing device S1A shown in Fig. 9, and configuring the information processing device S1 shown in Fig. 1 to be able to realize the functions of the processor 11A. In the description of the biological data conversion model generation system 100A shown in Fig. 9, the same reference numerals are assigned to configurations and functions similar to those of the biological data conversion model generation system 100 shown in Fig. 1, and description thereof will be omitted.

[0108] The biometric data conversion model generation system 100A includes a second data acquisition device P2 capable of acquiring biometric data used for authentication, and an information processing device S1A that infers (generates) biometric data used for authentication using a trained model generated in advance (at least one trained model from the first trained model to the third trained model) and has characteristics of a body part similar to or matching the biometric data at the time of enrollment. Note that the second data acquisition device P2 may be configured integrally with the information processing device S1A.

[0109] The information processing device S1A is connected to the second data acquisition device P2 so that data can be communicated therewith. The information processing device S1A is realized by a PC, a laptop PC, a tablet terminal, a cloud server, an on-premise server, or the like. The information processing device S1A includes a communication unit 10, a processor 11A, a memory 12, a trained model storage unit 14, a non-contact data acquisition unit 15, and a registered data storage unit 16. Note that the non-contact data acquisition unit 15 or the registered data storage unit 16 are not essential and may be omitted.

[0110] The processor 11A is configured using, for example, a CPU, FPGA, or GPU, and performs various processes and controls in cooperation with the memory 12. Specifically, the processor 11A references programs and data stored in the memory 12 and executes the programs to realize functions such as a feature point data generation unit 112, an inference unit 113, a registration unit 114, or an authentication unit 115, and performs inference of biometric data. Note that each of the feature point data generation unit 112, the registration unit 114, and the authentication unit 115 is not an essential component and may be omitted or realized by an external device communicably connected to the information processing device S1A.

[0111] The inference unit 113 uses the first learned model, the second learned model, or the third learned model that has been generated and saved (stored) in the learned model memory unit 14 to infer biometric data corresponding to the method of acquiring the biometric data at the time of registration from the biometric data acquired by the second data acquisition device P2 or the non-contact data acquisition unit 15.

[0112] The registration unit 114 functions when registering biometric data (contactless data) acquired by the second data acquisition device P2 or the contactless data acquisition unit 15 as biometric data at the time of registration to be compared with biometric data acquired at the time of authentication in authentication. The registration unit 114 associates the biometric data generated (inferred) by the inference unit 113 as the biometric data at the time of registration with information on the person and body part corresponding to this biometric data, and stores the associated data in the registration data storage unit 16.

[0113] The authentication unit 115 functions when performing biometric authentication using biometric data (contactless data) acquired by the second data acquisition device P2 or the contactless data acquisition unit 15. The authentication unit 115 outputs the biometric data generated (inferred) by the inference unit 113 as biometric data for authentication to an application or the like that executes authentication processing, and performs biometric authentication of the person corresponding to the biometric data for authentication by comparing it with the biometric data stored in the enrollment data storage unit 16.

[0114] The contactless data acquisition unit 15 is provided when the information processing device S1A itself acquires biometric data used for authentication, rather than the second data acquisition device P2. The contactless data acquisition unit 15 is realized by, for example, a visible light camera. The contactless data acquisition unit 15 outputs the captured contactless image (biometric data) to the processor 11A.

[0115] The enrollment data storage unit 16 stores (preserves) biometric data used for biometric authentication and registered before authentication is performed, in association with information on the person who owns the biometric data and the biometric body parts. The enrollment data storage unit 16 may be configured separately from the information processing device S1A and realized by an external storage medium capable of data communication with the information processing device S1A.

[0116] <Inference method using the first trained model> A method for inferring biometric data using the first trained model will be described with reference to Fig. 10. Note that the flowchart shown in Fig. 10 describes, as an example, an example in which the biometric data acquired by the second data acquisition device P2 or the non-contact data acquisition unit 15 is a non-contact image, and a non-contact image corresponding to the state of the biometric part shown in the contact image is generated (inferred) by conversion using the first trained model.

[0117] The inference unit 113 acquires the non-contact image (biometric data) acquired by the second data acquisition device P2 (St51).

[0118] The inference unit 113 converts the acquired contactless image using the first trained model (St52), and outputs the converted contactless image (biometric data) to the registration unit 114 or the authentication unit 115 (St53). Specifically, when the converted contactless image is to be stored in the registration data storage unit 16, the inference unit 113 outputs the converted contactless image to the registration unit 114, and when biometric authentication is to be performed using the converted contactless image, the inference unit 113 outputs the converted contactless image to the authentication unit 115.

[0119] The registration unit 114 adds or overwrites the converted contactless image as a contact image previously registered for biometric authentication (St54). The authentication unit 115 inputs the converted contactless image into an application or the like that executes authentication processing, and executes biometric authentication (St54).

[0120] As described above, the information processing device S1A can infer (generate) a contactless image corresponding to a contact image of a fingerprint in contact with an object based on a contactless image of a fingerprint captured without contacting any object by converting the contactless image using the first trained model. This allows the information processing device S1A to use the first trained model to convert a contactless image acquired using a method different from pre-registered biometric data (contact image), thereby adding or overwriting the converted contactless image as a new contact image for registration. The information processing device S1A can also perform biometric authentication using a contactless image acquired for authentication and the additionally registered biometric data (i.e., a contactless image converted using the first trained model).

[0121] <Inference method using the second trained model> A method for inferring biometric data using the second trained model will be described with reference to Fig. 10. Note that the flowchart shown in Fig. 10 describes, as an example, an example in which the biometric data acquired by the second data acquisition device P2 or the non-contact data acquisition unit 15 is a non-contact image, and non-contact feature points corresponding to the state of contact feature points are generated (inferred) by conversion using the second trained model.

[0122] The inference unit 113 acquires the non-contact image (biometric data) acquired by the second data acquisition device P2 (St61).

[0123] The inference unit 113 converts the acquired contactless image into contactless feature points corresponding to the contactless feature points using the second trained model (St63), and outputs the converted contactless feature points (biometric data) to the registration unit 114 or the authentication unit 115 (St64). Specifically, if the converted contactless feature points are to be stored in the registration data storage unit 16, the inference unit 113 outputs the converted contactless feature points to the registration unit 114, and if biometric authentication is to be performed using the converted contactless feature points, the inference unit 113 outputs the converted contactless feature points to the authentication unit 115.

[0124] The registration unit 114 adds or overwrites the converted non-contact feature points as contact feature points previously registered in biometric authentication (St65). The authentication unit 115 inputs the converted non-contact feature points into an application or the like that executes authentication processing, and executes biometric authentication (St65).

[0125] As described above, the information processing device S1A can infer (generate) non-contact feature points (biometric data) corresponding to the state of contact feature points generated from a fingerprint in a state of contact with any object, based on a contactless image in which the fingerprint is captured in a state of not contacting any object, by converting the non-contact feature points using the second trained model. Thus, the information processing device S1A can use the second trained model to convert a contactless image acquired in a manner different from the pre-registered biometric data (contact feature points), and add or overwrite the converted non-contact feature points as new contact feature points for registration. The information processing device S1A can also perform biometric authentication using a contactless image acquired for authentication and the additionally registered biometric data (i.e., non-contact feature points converted using the second trained model).

[0126] <Third inference method using trained model> A method for inferring biometric data using the third trained model will be described with reference to Fig. 10. Note that the flowchart shown in Fig. 10 describes, as an example, an example in which the biometric data acquired by the second data acquisition device P2 or the non-contact data acquisition unit 15 is a non-contact image, and non-contact feature points corresponding to the state of feature points of biometric parts included in the contact image are generated (inferred) by conversion using the third trained model.

[0127] The inference unit 113 acquires the non-contact image (biometric data) acquired by the second data acquisition device P2 (St61).

[0128] The feature data generation unit 112 generates non-contact feature points of the fingerprint from the acquired non-contact image using the second trained model (St62).

[0129] The inference unit 113 converts the acquired non-contact feature points into non-contact feature points corresponding to the contact feature points using the second trained model (St63), and outputs the converted non-contact feature points (biometric data) to the registration unit 114 or the authentication unit 115 (St64). Specifically, if the converted non-contact feature points are to be stored in the registration data storage unit 16, the inference unit 113 outputs the converted non-contact feature points to the registration unit 114, and if biometric authentication is to be performed using the converted non-contact feature points, the inference unit 113 outputs the converted non-contact feature points to the authentication unit 115.

[0130] The registration unit 114 adds or overwrites the converted non-contact feature points as contact feature points previously registered in biometric authentication (St65). The authentication unit 115 inputs the converted non-contact feature points into an application or the like that executes authentication processing, and executes biometric authentication (St65).

[0131] As described above, the information processing device S1A can infer (generate) non-contact feature points (biometric data) corresponding to contact feature points of a fingerprint in a contact state in contact with an object based on the non-contact feature points of a fingerprint in a non-contact state by converting the non-contact feature points using the third trained model. This allows the information processing device S1A to use the third trained model to convert non-contact feature points acquired by a method different from the pre-registered biometric data (contact feature points), thereby adding or overwriting the converted non-contact image as new contact feature points for registration. The information processing device S1A can also perform biometric authentication using the non-contact feature points acquired for authentication and the additionally registered biometric data (i.e., the non-contact feature points converted using the third trained model).

[0132] (Addendum) The above description of each embodiment discloses the following techniques.

[0133] (Technology 1) an acquisition unit (communication unit 10) that acquires one or more first biometric data (contact data, contact image, or contact feature points) of biometric regions of a plurality of persons to be authenticated acquired by a first acquisition method (contact state), and second biometric data (non-contact data, non-contact image, or non-contact feature points) of the biometric regions acquired by a second acquisition method (non-contact state) different from the first acquisition method (contact state); a data conversion unit (learning unit 111) that converts the second biometric data into biometric data equivalent to the first biometric data acquired by the first acquisition method (contact state) using a learning model MD; a calculation unit (learning unit 111) that calculates a degree of coincidence between the features of the body part included in the first biometric data and the features of the body part included in the converted second biometric data; a control unit (learning unit 111) that, when it is determined that the calculated degree of agreement does not satisfy a predetermined condition, performs learning on the learning model with the degree of agreement being a loss, and, when it is determined that the degree of agreement satisfies the predetermined condition, outputs the learning model MD as a trained model (first trained model, second trained model, or third trained model) that converts the second biometric data into biometric data acquired by the first acquisition method (contact state), A biological data conversion model generation device (information processing device S1). As a result, the information processing device S1 can generate and acquire a trained model capable of generating second biometric data in which the state of a biometric part indicated by the second biometric data acquired by the second acquisition method has been changed to a state corresponding to the state of a biometric part indicated by the first biometric data acquired by the first acquisition method (e.g., how the biometric part is crushed, how the feature points or features change, changes in the positional relationship between the feature points, etc.). Therefore, even if the three-dimensional shape of a biometric part differs for each type of person or biometric part and the biometric part is deformed by any pressure applied to the biometric part during enrollment, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods by using the trained model.

[0134] (Technology 2) the first acquisition method (contact state) is a method for acquiring biometric data of the body part of the person to be authenticated that is registered in advance in biometric authentication, The second acquisition method (contactless state) is a method for acquiring biometric data of the body part of the person to be authenticated in the biometric authentication. A biological data conversion model generation device (information processing device S1) according to (Technology 1). As a result of the above, the information processing device S1 can generate a trained model that can convert biometric data acquired using the acquisition method at the time of authentication (second acquisition method) into data equivalent to biometric data acquired (imaged) using the acquisition method at the time of registration (first acquisition method).

[0135] (Technology 3) the first acquisition method is a method in which the biometric data of the body part is acquired in a contact state in which the body part is in contact with an arbitrary object, the second acquisition method is a method in which the biometric data of the body part is acquired in a non-contact state in which the body part is not in contact with the object; A biological data conversion model generation device (information processing device S1) according to (Technology 1) or (Technology 2). As a result of the above, the information processing device S1 can generate a trained model that can convert biometric data acquired when the biological part is not in contact (second acquisition method) into data equivalent to biometric data acquired (imaged) when the biological part is in contact (first acquisition method).

[0136] (Technology 4) Each of the first biometric data and the second biometric data is biometric data obtained by capturing an image of the body part. A biological data conversion model generating device (information processing device S1) according to any one of (Technology 1) to (Technology 3). As a result of the above, the information processing device S1 can acquire a trained model (first trained model) that converts a non-contact image acquired (imaged) using the second acquisition method into image data equivalent to a contact image acquired (imaged) using the first acquisition method, and that can convert a biological part shown in image data acquired (imaged) using the second acquisition method into image data equivalent to a contact image acquired (imaged) using the first acquisition method.

[0137] (Technology 5) Each of the first biometric data and the second biometric data is feature point information of the body part. A biological data conversion model generating device (information processing device S1) according to any one of (Technology 1) to (Technology 3). As a result of the above, the information processing device S1 can acquire a trained model (third trained model) that converts non-contact feature points acquired by the second acquisition method into data equivalent to contact feature points acquired by the first acquisition method, and that can convert non-contact feature points acquired by the second acquisition method into feature points equivalent to contact feature points acquired (imaged) by the first acquisition method.

[0138] (Technology 6) a feature data generation unit (112) that generates first feature information of the biological part based on the first biometric data; the data conversion unit (learning unit 111) converts the second biometric data into second feature point information corresponding to the first feature point information based on the first biometric data acquired by the first acquisition method (contact state) using the learning model MD2; The calculation unit (processor 11, 11A) calculates a degree of coincidence between the first feature point information and the converted second feature point information. A biological data conversion model generating device (information processing device S1) according to any one of (Technology 1) to (Technology 3). As a result of the above, the information processing device S1 can acquire a trained model (second trained model) that converts a non-contact image acquired (imaged) using the second acquisition method into data corresponding to contact feature points generated from a contact image acquired using the first acquisition method, and that can convert a non-contact image acquired using the second acquisition method into feature point information corresponding to contact feature points acquired (imaged) using the first acquisition method.

[0139] (Technology 7) When the number of the first biometric data is one, the control unit (learning unit 111) determines whether the degree of match between the first biometric data and the second biometric data is equal to or greater than a predetermined value. A biological data conversion model generating device (information processing device S1) according to any one of (Technology 1) to (Technology 6). As described above, when the information processing device S1 uses one piece of biometric data (non-contact data) and one piece of biometric data (contact data) as learning data, by setting a higher matching value, it can generate a trained model that can achieve high reproducibility and higher conversion accuracy in the conversion (inference) process that converts the input biometric data (non-contact data) into biometric data (non-contact data) having characteristics equivalent to those of the biometric data (contact data).

[0140] (Technology 8) When there are a plurality of first biometric data, the control unit (learning unit 111) calculates the degree of match between each of the first biometric data and the second biometric data, and determines whether or not an average value of the calculated degrees of match is equal to or greater than a predetermined value. A biological data conversion model generating device (information processing device S1) according to any one of (Technology 1) to (Technology 6). As a result, when the information processing device S1 uses one piece of biometric data (non-contact data) and multiple pieces of biometric data (contact data) as training data, it can generate a trained model that can achieve conversion (inference) with higher consistency and higher stability for various biometric data (non-contact data) in the conversion (inference) process of the biometric data (non-contact data) described above.

[0141] (Technology 9) A method for generating a biological data conversion model performed by one or more processors (processors 11, 11A), Acquire one or more first biometric data (contact data, contact image, or contact feature points) of biometric regions of a plurality of subjects acquired by a first acquisition method (contact state), and acquire second biometric data (non-contact data, non-contact image, or non-contact feature points) of the biometric regions acquired by a second acquisition method (non-contact state) different from the first acquisition method (contact state), converting the second biometric data into biometric data equivalent to the first biometric data acquired by the first acquisition method (contact state) using a learning model MD; calculating a degree of coincidence between the features of the body part included in the first biometric data and the features of the body part included in the converted second biometric data; If it is determined that the calculated degree of agreement does not satisfy a predetermined condition, the learning model is subjected to learning in which the degree of agreement is treated as a loss; When it is determined that the calculated degree of match satisfies a predetermined condition, the learning model MD is output as a learned model (first learned model, second learned model, or third learned model) that converts the second biometric data into biometric data acquired by the first acquisition method (contact state). A method for generating a biomedical data transformation model. As a result, the information processing device S1 can generate and acquire a trained model capable of generating second biometric data in which the state of a biometric part indicated by the second biometric data acquired by the second acquisition method has been changed to a state corresponding to the state of a biometric part indicated by the first biometric data acquired by the first acquisition method (e.g., how the biometric part is crushed, how the feature points or features change, changes in the positional relationship between the feature points, etc.). Therefore, even if the three-dimensional shape of a biometric part differs for each type of person or biometric part and the biometric part is deformed by any pressure applied to the biometric part during enrollment, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods by using the trained model.

[0142] (Technology 10) a first acquisition device (first data acquisition device P1) that acquires first biometric data (contact data, contact images, or contact feature points) of multiple biometric parts of a person to be authenticated by a first acquisition method (contact state); a second acquisition device (second data acquisition device P2) that acquires second biometric data (non-contact data, non-contact image, or non-contact feature points) of the biometric part acquired by a second acquisition method (non-contact state) different from the first acquisition method (contact state); A biological data conversion model generation system (biological data conversion model generation system 100) including: a conversion model generation device (information processing device S1) capable of communicating with the first acquisition device (first data acquisition device P1) and the second acquisition device (second data acquisition device P2); The first acquisition device (first data acquisition device P1) acquiring one or more pieces of first biological data and transmitting them to the transformation model generation device (information processing device S1); The second acquisition device (second data acquisition device P2) acquiring the second biological data and transmitting it to the transformation model generation device (information processing device S1); The transformation model generation device (information processing device S1) converting the second biometric data into biometric data equivalent to the first biometric data acquired by the first acquisition method (contact state) using a learning model MD; calculating a degree of coincidence between the features of the body part included in the first biometric data and the features of the body part included in the converted second biometric data; If it is determined that the calculated degree of agreement does not satisfy a predetermined condition, the learning model is subjected to learning in which the degree of agreement is treated as a loss; When it is determined that the calculated degree of match satisfies a predetermined condition, the learning model MD is output as a learned model (first learned model, second learned model, or third learned model) that converts the second biometric data into biometric data acquired by the first acquisition method (contact state). A system for generating a biological data conversion model (biological data conversion model generation system 100). As described above, the biometric data conversion model generation system 100 can generate and acquire a trained model capable of generating second biometric data in which the state of a biometric part indicated by second biometric data acquired by a second acquisition method is transformed into a state corresponding to the state of a biometric part indicated by first biometric data acquired by a first acquisition method (e.g., how the biometric part is crushed, how feature points or features change, changes in the positional relationship between feature points, etc.). Therefore, even if the three-dimensional shape of a biometric part differs for each type of person or biometric part and the biometric part is deformed by any pressure applied to the biometric part during enrollment, the information processing device S1 can improve the authentication accuracy of biometric authentication using biometric data acquired by different acquisition methods by using the trained model.

[0143] Although various embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that those skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components of the various embodiments described above may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]

[0144] The present disclosure is useful as a biometric data conversion model generation device, a biometric data conversion model generation method, and a biometric data conversion model generation system that generate a trained model that converts biometric data acquired using a method different from that used at the time of enrollment into biometric data that can be biometrically authenticated. [Explanation of symbols]

[0145] 10, 20, 30 Communications Department 11, 11A, 21, 31 processors 12, 22, 32 memory 13 Learning data storage unit 14 Model storage section 15 Non-contact data acquisition unit 16 Registration data storage unit 23 Contact data acquisition unit 33 Non-contact data acquisition unit 100,100A Biological Data Conversion Model Generation System 111 Learning Department 112 Feature point data generation unit Dt10, Dt20, Dt30 training data Dt11, Dt12, Dt21 non-contact image Dt131, Dt13N, Dt231, Dt23N, Dt331, Dt33N Contact image Dt241, Dt24N, Dt341, Dt34N Contact feature points Dt22, Dt31, Dt32 Non-contact feature points MD learning model P1 First data acquisition device P2 Second data acquisition device S1, S1A Information processing device

Claims

1. an acquisition unit that acquires captured images of first biometric data of a plurality of biometric parts of a person to be authenticated, the captured images being acquired by a first acquisition method, and captured images of second biometric data of the biometric parts being acquired by a second acquisition method different from the first acquisition method; a data conversion unit that converts the captured image of the second biometric data into image data corresponding to the captured image of the first biometric data acquired by the first acquisition method using a learning model; a calculation unit that calculates a degree of coincidence between a feature of the body part included in the captured image of the first biometric data and a feature of the body part included in the captured image of the second biometric data after conversion; a control unit that, when it is determined that the calculated degree of agreement does not satisfy a predetermined condition, performs learning on the learning model with the degree of agreement being a loss, and, when it is determined that the degree of agreement satisfies the predetermined condition, outputs the learning model as a trained model that converts an image of the second biological data into the image data acquired by the first acquisition method. A device for generating biological data conversion models.

2. the first acquisition method is a method for acquiring biometric data of the body part of the person to be authenticated that is registered in advance in biometric authentication, the second acquisition method is a method for acquiring biometric data of the body part of the person to be authenticated in the biometric authentication. The biological data conversion model generating device according to claim 1 .

3. the first acquisition method is a method in which the biometric data of the body part is acquired in a contact state in which the body part is in contact with an arbitrary object, the second acquisition method is a method in which the biometric data of the body part is acquired in a non-contact state in which the body part is not in contact with the object; The biological data conversion model generating device according to claim 1 .

4. the captured image of the first biological data and the captured image of the second biological data include feature point information of the biological part; The biological data conversion model generating device according to claim 1 .

5. a feature point data generation unit that generates first feature point information of the biological part based on the captured image of the first biological data, the data conversion unit converts the captured image of the second biometric data into second feature point information corresponding to the first feature point information based on the captured image of the first biometric data acquired by the first acquisition method, using the learning model; the calculation unit calculates a degree of coincidence between the first feature point information and the converted second feature point information; The biological data conversion model generating device according to claim 1 .

6. when there is one captured image of the first biometric data, the control unit determines whether the degree of match between the captured image of the first biometric data and the captured image of the second biometric data is equal to or greater than a predetermined value; The biological data conversion model generating device according to claim 1 .

7. When there are a plurality of captured images of the first biometric data, the control unit calculates the degree of agreement between each of the captured images of the first biometric data and the captured image of the second biometric data, and determines whether an average value of the calculated degrees of agreement is equal to or greater than a predetermined value. The biological data conversion model generating device according to claim 1 .

8. 1. A method for generating a biological data transformation model performed by one or more processors, comprising: Acquire captured images of first biometric data of one or more biometric parts of a plurality of subjects acquired by a first acquisition method, and captured images of second biometric data of the biometric parts acquired by a second acquisition method different from the first acquisition method, converting the captured image of the second biometric data into image data corresponding to the captured image of the first biometric data acquired by the first acquisition method using a learning model; calculating a degree of coincidence between a feature of the body part included in the captured image of the first biometric data and a feature of the body part included in the captured image of the converted second biometric data; If it is determined that the calculated degree of agreement does not satisfy a predetermined condition, the learning model is subjected to learning in which the degree of agreement is treated as a loss; When it is determined that the calculated degree of match satisfies a predetermined condition, the learning model is output as a trained model that converts the captured image of the second biological data into the image data acquired by the first acquisition method. A method for generating a biomedical data transformation model.

9. a first acquisition device that acquires captured images of first biometric data of a plurality of biometric parts of a person to be authenticated by a first acquisition method; a second acquisition device that acquires a captured image of second biological data of the biological part acquired by a second acquisition method different from the first acquisition method; a conversion model generation device capable of communicating with the first acquisition device and the second acquisition device, The first acquisition device acquiring the captured image of one or more of the first biological data and transmitting it to the transformation model generation device; the second acquisition device, acquiring the captured image of the second biological data and transmitting it to the transformation model generation device; The transformation model generation device converting the second biometric data into image data corresponding to the captured image of the first biometric data acquired by the first acquisition method using a learning model; calculating a degree of coincidence between a feature of the body part included in the captured image of the first biometric data and a feature of the body part included in the captured image of the converted second biometric data; If it is determined that the calculated degree of agreement does not satisfy a predetermined condition, the learning model is subjected to learning in which the degree of agreement is treated as a loss; When it is determined that the calculated degree of match satisfies a predetermined condition, the learning model is output as a trained model that converts the captured image of the second biological data into the image data acquired by the first acquisition method. A system for generating biological data conversion models.

Images