Machine learning device, machine learning system, and machine learning method

Abstract

To achieve machine learning using biological information reflecting a real environment without collecting the biological information of a user in the real environment.SOLUTION: Updating a distribution of an attribute feature value on the basis of a real environment attribute feature value corresponding to biological information of a target user and extracting the attribute feature value from the updated distribution; Generating synthetic biometric information based on the extracted attribute feature value and a learning identifying feature value extracted from learning biometric information, and updating a machine learning model based on the synthetic biometric information and the learning biometric information, the machine learning model outputting information indicating a feature of a living body when the biometric information is input, the attribute feature value affecting authentication accuracy of biometric authentication and having a low correlation with the identifying feature value, the identifying feature value being used for collation in the biometric authentication.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a machine learning device, a machine learning system, and a machine learning method. [Background technology]

[0002] Biometric authentication technology is becoming more widespread, performing personal authentication based on images of faces, fingerprints, irises, etc. Personal authentication is a process of verifying whether a user using a system is the same person as a user already registered in the system.

[0003] Generally, in biometric authentication, a comparison is made between enrollment features extracted from biometric information acquired at the time of enrollment and authentication features extracted from biometric information acquired at the time of authentication, and if the similarity between the enrollment features and the authentication features is equal to or greater than a predetermined threshold, the enrolled user and the authenticated user are considered to be the same person.

[0004] Machine learning models are widely used to extract features for authentication from biometric information. Machine learning models are trained in advance using a large amount of training data. However, if the environment in which the training data was acquired differs significantly from the actual operating environment, the performance of the model will deteriorate significantly.

[0005] Japanese Patent Laid-Open Publication No. 2022-160144 (Patent Document 1) is a background art in this technical field that addresses such problems. The learning device in this publication includes an acquisition unit that acquires a first feature amount obtained by inputting an observation amount generated by a simulator into a neural network and a second feature amount obtained by inputting an observation amount obtained from a real environment into the neural network, and a learning unit that trains the neural network using an evaluation function that includes the difference between the first feature amount and the second feature amount. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2022-160144 Summary of the Invention [Problem to be solved by the invention]

[0007] The technology described in Patent Document 1 requires that observables obtained in a real environment be used as learning data. In other words, when using the technology described in Patent Document 1 to train a machine learning model for biometric authentication, it is necessary to collect and store biometric information obtained in a real environment. Because biometric information is sensitive personal information, from the perspective of privacy protection, it is undesirable to collect and store biometric information in a real environment. Furthermore, there is a possibility that users who are reluctant to provide their biometric information will stop using biometric authentication services.

[0008] Therefore, one aspect of the present invention realizes machine learning using biometric information that reflects the real environment, without collecting biometric information of a user in the real environment. [Means for solving the problem]

[0009] In order to solve the above problem, one aspect of the present invention employs the following configuration: a machine learning device including a processor and a memory, wherein the memory stores real-world attribute features that are attribute features corresponding to biometric information of a target user, distribution parameters that indicate a distribution of the attribute features, training biometric information that is the biometric information for training, and a machine learning model that outputs information indicating biometric features when the biometric information is input, wherein the attribute features are features that affect authentication accuracy of biometric authentication based on the biometric information and have a low correlation with discriminant features extracted from the biometric information based on a predetermined condition, the discriminant features are features extracted from the biometric information and used for matching in the biometric authentication, and the processor updates the distribution parameters based on the real-world attribute features, extracts attribute features from the updated distribution parameters, extracts training discriminant features that are the discriminant features for training from the training biometric information, generates synthetic biometric information based on the extracted attribute features and the training discriminant features, and updates the machine learning model based on the synthetic biometric information and the training biometric information. [Effects of the Invention]

[0010] According to one aspect of the present invention, machine learning can be realized using biometric information that reflects the real environment, without collecting biometric information of a user in the real environment.

[0011] Problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram illustrating a configuration example of a machine learning system according to a first embodiment. [Figure 2] 2 is a block diagram showing an example of the hardware configuration of a computer constituting each of a client terminal and a server in the first embodiment. FIG. [Figure 3] FIG. 10 is a sequence diagram illustrating an example of machine learning processing according to the first embodiment. [Figure 4]FIG. 10 is a sequence diagram illustrating an example of a biometric authentication process and a machine learning process according to a second embodiment. [Figure 5] FIG. 10 is a sequence diagram illustrating an example of a registration process for biometric authentication processing according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. In this embodiment, the same components are generally designated by the same reference numerals, and repeated explanations will be omitted. It should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. [Example]

[0014] The machine learning system in this embodiment includes a client terminal and a server. The client terminal shares biometric attribute features extracted from a user's biometric information with the server, updates attribute feature distribution parameters, generates synthetic biometric training information based on attribute features sampled from the attribute feature distribution parameters and discrimination features extracted from training biometric information, and performs machine learning using the training biometric information and the synthetic biometric training information.

[0015] 1 is a block diagram showing an example configuration of a machine learning system. The machine learning system includes, for example, a client terminal 1000 and a server 1100. The client terminal 1000 and the server 1100 are connected to each other via a network such as the Internet. Note that the machine learning system may include multiple client terminals 1000.

[0016] 1, the client terminal 1000 includes, for example, a biometric information acquisition unit 1010, a biometric attribute feature extraction unit 1020, an environmental attribute feature acquisition unit 1030, a user ID acquisition unit 1040, and an identification feature extraction unit 1050, all of which are functional units.

[0017] The biometric information acquisition unit 1010 acquires biometric information of a user through an input device such as a sensor or a camera. The biometric information includes, for example, an image showing the characteristics of a part of the user's body, such as a face image, a fingerprint image, a palm vein image, or an iris image.

[0018] The biometric attribute feature extraction unit 1020 extracts biometric attribute features from the biometric information acquired by the biometric information acquisition unit 1010. The biometric attribute features include features indicating, for example, the orientation and tilt of the face or fingers, skin color, whether or not an accessory is being worn and the position where it is worn, and the brightness and contrast of the biometric information in the image.

[0019] Although biometric attribute features are features extracted from biometric information, they are extracted separately from identification features. Furthermore, since biometric attribute features are features extracted from biometric information, some biometric attribute features may depend on the modality of the biometric information.

[0020] The environmental attribute feature acquisition unit 1030 acquires environmental attribute features that indicate the environment in which the biometric information was acquired. The environmental attribute features include features that indicate, for example, the illuminance of the surrounding environment where the client terminal 1000 is installed (the environment in which the biometric information was acquired), the acquisition date and time, location information of the client terminal 1000, the store where the client terminal 1000 is installed, and the ID of the client terminal 1000.

[0021] The environmental attribute feature is, for example, information acquired from a device installed in the environment where the biometric information is acquired. Specifically, for example, the environmental attribute feature is acquired from information registered in advance in the client terminal 1000, settings of a sensor (camera) that captures the biometric information, and a sensor (e.g., an illuminance sensor) different from the sensor that captures the biometric information. The environmental attribute feature is different from the biometric attribute feature in that it is acquired from an information source other than the biometric information.

[0022] Hereinafter, biometric information features and environmental attribute features may be collectively referred to simply as attribute features. Attribute features are features that affect the authentication accuracy of biometric authentication using biometric information acquired by the biometric information acquisition unit 1010. On the other hand, an individual cannot be identified based on attribute features alone. Specifically, for example, attribute features are features that have a low correlation with identification features described below based on a predetermined condition (for example, a correlation coefficient lower than a predetermined threshold (or a correlation coefficient lower than a predetermined first threshold and higher than a predetermined second threshold)). Note that attribute features extracted from the biometric information of multiple people with similar attributes have a high degree of similarity.

[0023] The user ID acquisition unit 1040 acquires a user ID that uniquely identifies a user, for example, in accordance with an input from the user to an input device. The user ID is written, for example, as a character string made up of alphanumeric characters.

[0024] The identification feature extraction unit 1050 extracts an identification feature for identifying an individual from the biometric information acquired by the biometric information acquisition unit 1010. The identification feature is a feature used for matching during biometric authentication, and is information with high individual identifiability. Generally, the similarity between identification features extracted from multiple pieces of biometric information of the same individual is higher than the similarity between identification features extracted from biometric information of different people.

[0025] The client terminal 1000 in the first embodiment does not necessarily have to include the user ID acquisition unit 1040 and the identification feature extraction unit 1050.

[0026] Server 1100, which is an example of a machine learning device, includes, for example, a parameter update unit 1110, an attribute feature sampling unit 1120, an identification feature extraction unit 1130, a biometric information generation unit 1140, a model update unit 1150, an attribute feature extraction unit 1160, and an identification feature matching unit 1170, all of which are functional units.

[0027] The server 1100 includes, for example, an attribute feature distribution parameter storage unit 1190, a learning biometric information storage unit 1191, a learning object model storage unit 1192, and a registered identification feature storage unit 1193, all of which are storage units in which information is stored.

[0028] The parameter update unit 1110 updates the attribute feature quantity distribution parameters based on the attribute feature quantities received from the client terminal 1000 and the attribute feature quantity distribution parameters stored in the attribute feature quantity distribution parameter storage unit 1190 .

[0029] The attribute feature distribution is a probability distribution in which the values ​​of the attribute feature are random variables. Any probability distribution can be adopted as the attribute feature distribution. Specifically, for example, a type of probability distribution may be predetermined for each type of attribute feature, and information indicating the type of probability distribution may also be stored in the attribute feature distribution parameter storage unit 1190. The attribute feature distribution parameters are parameters that determine the shape of the probability distribution. For example, when the attribute feature distribution is a multidimensional normal distribution, the attribute feature distribution parameters include a mean vector and a covariance matrix.

[0030] The attribute feature distribution parameters stored in the attribute feature distribution parameter storage unit 1190 may be, for example, parameters generated only according to the attribute features acquired by the client terminal 1000 (attribute features acquired in an actual operating environment), or may be parameters that are given predetermined initial values ​​and then updated using the attribute features acquired by the client terminal 1000.

[0031] Furthermore, when the client terminal 1000 extracts or acquires a plurality of types of attribute feature amounts, different attribute feature amount distribution parameters may be set for different types of attribute feature amounts.

[0032] Furthermore, the attribute feature quantity distribution parameters may be managed for each client terminal 1000 and / or for each user ID. That is, an attribute feature quantity distribution may be defined for each client terminal 1000, an attribute feature quantity distribution may be defined for each user ID, or an attribute feature quantity distribution may be defined for each combination of a client terminal 1000 and a user ID.

[0033] For example, in a case where attribute feature distribution parameters are managed for each ID of the client terminal 1000, for each location information of the client terminal 1000 if the client terminal 1000 is a stationary terminal, or for each store ID where the client terminal 1000 is installed, the server 1100 can obtain attribute features corresponding to the client terminal 1000 installed in a specific store.

[0034] If the server 1100 also receives information indicating the store in which the client terminal 1000 is installed in step S2510, which will be described later, it will be able to perform machine learning specialized for the actual environment of the store by executing the synthetic biometric information generation process in step S2570, which will be described later, using only the attribute features sampled from the attribute feature distribution determined by the attribute feature distribution parameters corresponding to the client terminal 1000 installed in the specific store.

[0035] The attribute feature sampling unit 1120 samples attribute features from the attribute feature distribution determined based on the attribute feature distribution parameters stored in the attribute feature distribution parameter storage unit 1190 .

[0036] For example, suppose that a multidimensional vector representing the brightness of biometric information exists as an attribute feature, the vector follows a multidimensional normal distribution, and a mean vector μ and a covariance matrix Σ are stored as attribute feature distribution parameters. In this case, the attribute feature sampling unit 1120 can sample the attribute feature by generating random numbers following a multidimensional normal distribution having a mean vector μ and a covariance matrix Σ.

[0037] The discrimination feature extraction unit 1130 extracts discrimination features from the training biometric information stored in the training biometric information storage unit 1191. The biometric information generation unit 1140 synthesizes the attribute features sampled by the attribute feature sampling unit 1120 and the discrimination features of the training biometric information extracted by the discrimination feature extraction unit 1130 to generate synthesized biometric information and stores the synthesized biometric information in the training biometric information storage unit 1191.

[0038] The model update unit 1150 updates the machine learning model based on the training biometric information including the synthetic biometric information stored in the training biometric information storage unit 1191 and the machine learning model stored in the learning model storage unit 1192. Any general machine learning model such as a linear regression model, a decision tree, or a neural network can be applied as the machine learning model.

[0039] The machine learning model stored in the learning model storage unit 1192 is a model that outputs information indicating biometric features when biometric information is input. Specifically, for example, the machine learning model may be a model that outputs a discrimination feature when biometric information is input, or a model that outputs a converted image of an image (a converted image including biometric features) when biometric information that is an image is input.

[0040] The attribute feature extraction unit 1160 extracts biometric attribute features and / or environmental attribute features from the biometric information stored in the learning biometric information storage unit 1191. The discrimination feature matching unit 1170 matches discrimination features included in enrollment templates registered in the enrollment discrimination feature storage unit 1193 with discrimination features extracted from the biometric information acquired by the client terminal 1000, and outputs an authentication result.

[0041] The server 1100 in the first embodiment does not necessarily have to include the discrimination feature matching unit 1170 and the registered discrimination feature storage unit 1193 .

[0042] 2 is a block diagram showing an example of the hardware configuration of a computer constituting each of the client terminal 1000 and the server 1100. The computer 8000 includes, for example, a CPU (Central Processing Unit) 8010, a memory 8020, an auxiliary storage device 8030, an input device 8040, an output device 8050, and a communication device 8060.

[0043] The CPU 8010 is an example of a processor, and executes programs stored in the memory 8020. The memory 8020 includes a ROM (Read Only Memory), which is a non-volatile storage element, and a RAM (Random Access Memory), which is a volatile storage element. The ROM stores immutable programs (e.g., a BIOS (Basic Input / Output System)). The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores programs executed by the CPU 8010 and data used when the programs are executed.

[0044] The auxiliary storage device 8030 is a large-capacity, non-volatile storage device such as a magnetic storage device (HDD (Hard Disk Drive)) or a flash memory (SSD (Solid State Drive)), and stores programs to be executed by the CPU 8010 and data to be used when the programs are executed. In other words, the programs are read from the auxiliary storage device 8030, loaded into the memory 8020, and executed by the CPU 8010.

[0045] A part or all of the programs executed by the CPU 8010 may be provided to the computer 8000 from a removable medium (such as a CD-ROM or flash memory) which is a non-transitory storage medium, or from an external computer equipped with a non-transitory storage device via a network, and may be stored in the non-volatile auxiliary storage device 8030 which is a non-transitory storage medium. For this reason, the computer 8000 may have an interface for reading data from removable media.

[0046] The CPU 8010 of the computer 8000 constituting the client terminal 1000 includes the aforementioned functional units of the client terminal 1000, namely, a biometric information acquisition unit 1010, a biometric attribute feature extraction unit 1020, an environmental attribute feature acquisition unit 1030, a user ID acquisition unit 1040, and an identification feature extraction unit 1050.

[0047] The CPU 8010 of the computer 8000 constituting the server 1100 includes the aforementioned functional units of the server 1100, namely, a parameter update unit 1110, an attribute feature sampling unit 1120, an identification feature extraction unit 1130, a biometric information generation unit 1140, a model update unit 1150, an attribute feature extraction unit 1160, and an identification feature matching unit 1170.

[0048] For example, the CPU 8010 of the computer 8000 constituting the client terminal 1000 functions as the biometric information acquisition unit 1010 by operating in accordance with a biometric information acquisition program loaded into the memory 8020 of the computer 8000 constituting the client terminal 1000, and functions as the biometric attribute feature extraction unit 1020 by operating in accordance with a biometric attribute feature extraction program loaded into the memory 8020. The same relationship between programs and functional units applies to other functional units included in the CPU 8010 of the computer 8000 constituting the client terminal 1000. The same relationship between programs and functional units applies to functional units included in the CPU 8010 of the computer 8000 constituting the server 1100.

[0049] The auxiliary storage device 8030 of the computer 8000 constituting the server 1100 provides a storage area for realizing the above-mentioned storage units of the server 1100, that is, the attribute feature distribution parameter storage unit 1190, the learning biometric information storage unit 1191, and the learning target model storage unit 1192. Data stored in each storage unit is accumulated as data on the auxiliary storage device 8030. Note that part or all of the information stored in each storage unit may be stored in the memory 8020, or may be stored in an external device connected to the computer 8000.

[0050] It should be noted that the information used by the machine learning system is not data structure dependent and may be represented in any data structure, for example, a table, a list, a database, or a queue, as appropriate, may store the information.

[0051] The input device 8040 includes devices that receive input from a user, such as a keyboard, a touch panel, a smart device, a mouse, etc. The input device 8040 also includes devices such as a biometric sensor, a scanner, and a camera that allow the biometric information acquisition unit 1010 to acquire biometric information.

[0052] The output device 8050 is a device that outputs the results of program execution in a format that can be viewed by a user, such as a display, printer, touch panel, or smart device. The communication device 8060 is a network interface device that controls communication with other devices in accordance with a predetermined protocol. The communication device 8060 may also include a serial interface such as a USB (Universal Serial Bus).

[0053] Each of the client terminal 1000 and the server 1100 is a computer system configured on a single physical computer 8000, or on multiple logically or physically configured computers 8000, and may operate in separate threads on the same computer 8000, or may operate on virtual computers constructed on multiple physical computer resources.

[0054] Fig. 3 is a sequence diagram showing an example of machine learning processing. In the example of Fig. 3, machine learning is performed that reflects attribute information of a real environment (for example, an environment in which the client terminal 1000 is used). In the first embodiment, the client terminal 1000 and the server 1100 share attribute features to perform machine learning.

[0055] Before the processing of Figure 3 starts, it is assumed that at least information indicating the type of attribute feature distribution is stored in the attribute feature distribution parameter storage unit 1190 (if the processing of Figure 3 has already been executed one or more times, the attribute feature distribution parameters are also stored), training biometric information is stored in the training biometric information storage unit 1191, and a machine learning model is stored in the training model storage unit 1192.

[0056] The biometric information acquisition unit 1010 of the client terminal 1000 acquires biometric information (biometric information of the same modality as the learning biometric information) from a user (an example of a target user) (S2010). The biometric information is data used for personal authentication, which is the target of this embodiment, and includes, for example, images showing the aforementioned physical information such as fingerprints, faces, irises, and veins, as well as information showing behavioral characteristics such as acceleration information, movement history, browsing history, and purchase history.

[0057] The acquisition of biometric information in step S2010 may be performed solely for the purpose of collecting attribute features, or may be performed for the purpose of authentication in addition to or instead of collecting attribute features. The client terminal 1000 may be a personal terminal such as a PC (Personal Computer), smartphone, or tablet terminal that is exclusively owned by one specific user, or may be a shared terminal installed in a specific store or the like and used by many users.

[0058] The biometric attribute feature extraction unit 1020 of the client terminal 1000 extracts biometric attribute features from the biometric information acquired in step S2010 (S2020).

[0059] Biometric attribute features refer to features based on biometric information. Biometric attribute features are extracted separately from identification features used in biometric authentication, and are features that make it difficult to identify an individual from biometric attribute features alone. Furthermore, among multiple biometric attribute features extracted from biometric information of multiple people acquired in the same environment, there is a tendency for there to be high similarities.

[0060] As described above, the biometric attribute feature amount includes, for example, feature amounts indicating the orientation and inclination of the face and fingers, whether or not accessories are worn, etc. Furthermore, when biometric information is acquired as an image, the biometric attribute feature amount may include feature amounts indicating the brightness and contrast of biometric parts (face, fingers, palm, iris, etc.) in the image.

[0061] Among these examples of biometric attribute features, the orientation and inclination of the face and fingers, and the brightness and contrast of biometric parts depend on the installation environment of the client terminal 1000 that acquires the biometric information, and therefore tend to have high similarity when acquired in the same environment. Also, hair and skin color often vary depending on the country or region, and are biometric attribute features that tend to have high similarity when acquired in the same environment.

[0062] The biometric attribute feature extraction unit 1020 narrows down the collection of biometric attribute features to only those biometric attribute features that are highly similar even between other people in the same environment, and updates the attribute distribution parameters, thereby updating the machine learning model in a manner that further eliminates personal identification. Note that the types of biometric attribute features with high similarity may be determined in advance or may be determined by the client terminal 1000. When the client terminal 1000 determines the types of biometric attribute features with high similarity, for example, it may calculate the average value of the distances between previously extracted values ​​for each of the same type of biometric attribute feature, and determine the type of biometric attribute feature with the smallest average value as the type of biometric attribute feature with high similarity.

[0063] The environmental attribute feature acquisition unit 1030 of the client terminal 1000 acquires environmental attribute features based on the environment in which the biometric information was acquired (S2030). Note that with regard to highly invariant information such as the terminal ID of the client terminal 1000 and location information (when the client terminal 1000 is a stationary terminal installed in a store, for example), the environmental attribute feature acquisition unit 1030 may not necessarily acquire this information each time step S2010 is performed, and may instead store the information acquired for the first time in the auxiliary storage device 8030. Note that the order in which the processing of step S2020 and the processing of step S2030 are performed does not matter. Also, it is possible that one of the processing of step S2020 and the processing of step S2030 is not executed, in which case the attribute feature corresponding to that one processing is not transmitted in step S2040.

[0064] The client terminal 1000 transmits the biometric attribute feature extracted in step S2020 and the environmental attribute feature acquired in step S2030 to the server 1100 (S2040). The biometric attribute feature and the environmental attribute feature acquired by the client terminal 1000 are examples of real environment attribute feature.

[0065] The server 1100 receives the biometric attribute feature amounts and the environmental attribute feature amounts transmitted from the client terminal 1000 in step S2040 (S2510).

[0066] The parameter update unit 1110 updates the attribute feature distribution parameters corresponding to each attribute feature, which are stored in the attribute feature distribution parameter storage unit 1190, based on the biometric attribute features and environmental attribute features received from the client terminal 1000 in step S2510, and stores the updated attribute feature distribution parameters in the attribute feature distribution parameter storage unit 1190 (S2520).

[0067] For example, when the attribute feature distribution parameters are managed separately for each type of attribute feature, such as the terminal ID or location information of the client terminal 1000, the parameter update unit 1110 updates the attribute feature distribution parameters in accordance with the type of attribute feature received from the client terminal 1000 in step S2510.

[0068] Furthermore, when attribute feature distribution parameters are managed for each user ID, the parameter update unit 1110 may use the authentication result of biometric authentication to update the attribute feature parameter distribution corresponding to the user ID having the attribute feature transmitted from the client terminal 1000. An example of using the authentication result of biometric authentication will be described later in a second embodiment with reference to FIG.

[0069] The attribute feature sampling unit 1120 selects an attribute feature to be sampled, and acquires the attribute feature distribution parameters of the selected attribute feature from the attribute feature distribution parameter storage unit 1190 (S2530).

[0070] At this time, the attribute feature sampling unit 1120 may acquire only attribute feature distribution parameters corresponding to the attribute feature of the type updated in step S2520, or may acquire only attribute feature distribution parameters other than those, or may acquire both. Furthermore, the attribute feature sampling unit 1120 may acquire only attribute feature distribution parameters corresponding to the attribute feature of the type specified by an administrator of the machine learning system, etc.

[0071] The attribute feature sampling unit 1120 samples an arbitrary number of attribute features (for example, the number may be specified by an administrator of the machine learning system or may be a randomly determined number) from each of the attribute feature distribution parameters acquired in step S2530 (S2540). Note that in step S2540, the attribute feature sampling unit 1120 may include the value of the attribute feature received in step S2510 in the sampling result in order to ensure that the value of the attribute feature received in step S2510 is reflected in updating the machine learning model.

[0072] The identification feature extraction unit 1130 of the server 1100 acquires the training biometric information from the training biometric information storage unit 1191, and extracts an identification feature from the acquired training biometric information (S2550).

[0073] The attribute feature extraction unit 1160 of the server 1100 acquires training biometric information from the training biometric information storage unit 1191, and extracts biometric attribute features and / or environmental attribute features from the acquired training biometric information (S2560). Note that only one of the biometric attribute features or the environmental attribute features may be extracted in step S2560. The types of attribute features to be extracted in step S2560 are designated, for example, by an administrator of the machine learning system.

[0074] The training biometric information to be acquired by the identification feature extraction unit 1130 and the attribute feature extraction unit 1160 may be all of the training biometric information stored in the training biometric information storage unit 1191, or may be training biometric information designated by an administrator of the machine learning system, etc. The order of execution of the process of step S2550 and the process of step S2560 does not matter.

[0075] Furthermore, if the attribute feature of the training biometric information is not used in the processing described later, the processing of step S2560 does not have to be executed. Specifically, for example, if only the environment attribute feature indicating whether or not an accessory is worn by 1 or 0 among the attribute feature is used in the processing described later, and if both the values ​​of 1 and 0 indicating whether or not an accessory is worn are obtained in the attribute feature sampling processing of step S2540, there is no need to acquire the value of the environment attribute feature indicating whether or not an accessory is worn from the training biometric information in step S2560, and therefore the processing of step S2560 can be omitted.

[0076] The biometric information generation unit 1140 generates synthetic biometric information by reflecting the values ​​of the attribute features extracted in step S2560 and the attribute features sampled in step S2540 for each of the discrimination features extracted in step S2550, and stores the generated synthetic biometric information in the training biometric information storage unit 1191 (S2570).

[0077] Specifically, for example, if the biometric information is a face image, in step S2570, the biometric information generation unit 1140 reflects each of the attribute features representing the face orientation, skin and hair color, shading on the face surface, and the presence or absence of specific accessories such as glasses and a mask to the discrimination features extracted from the face image (learning biometric information) of a certain person, thereby generating a face image of the person having a combination of the values ​​of each of these attributes.

[0078] Furthermore, the type of attribute feature sampled in step S2540 may be partially different from the type of attribute feature extracted in step S2560. Specifically, for example, if an attribute feature representing shading on the face surface is sampled in step S2540 and an attribute feature other than the attribute feature representing shading on the face surface is extracted from the training biometric information in step S2560, the biometric information generation unit 1140 can reflect only information regarding shading in the face image acquired in the actual operating environment (values ​​sampled from the attribute feature distribution) in the training biometric information in step S2570.

[0079] When an image generation model is used to generate the synthetic biometric information in step S2570, the attribute features that can be input to the image generation model vary depending on the characteristics of the image generation model. In this case, the system administrator can arbitrarily determine whether to use the attribute features sampled in step S2540 and / or the attribute features extracted from the training biometric information in step S2560 as the attribute features that can be input to the image generation model, depending on the purpose of training.

[0080] The model update unit 1150 selects and acquires biometric information to be used for learning from the learning biometric information storage unit 1191 (S2580). Specifically, for example, the model update unit 1150 randomly selects a predetermined number of pieces of learning biometric information from the learning biometric information storage unit 1191.

[0081] Furthermore, for example, the model update unit 1150 may calculate an evaluation value indicating the authentication accuracy for each piece of training biometric information stored in the training biometric information storage unit 1191, and may select training biometric information with a high evaluation value based on a predetermined condition (for example, training information with an evaluation value equal to or greater than a predetermined threshold, or a predetermined number of training biometric information in descending order of evaluation value), or may select training biometric information with a low evaluation value based on a predetermined condition (for example, training information with an evaluation value less than a predetermined threshold, or a predetermined number of training biometric information in descending order of evaluation value), or may select training selection information with an evaluation value within a predetermined range (for example, a range determined by a predetermined upper and lower limit).

[0082] Furthermore, for example, the model update unit 1150 may calculate an evaluation value based on the distance between pieces of training biometric information, or an evaluation value indicating the authentication accuracy for a set of training biometric information, and repeatedly exclude training biometric information based on the evaluation value (for example, it may exclude training biometric information with a high evaluation value based on predetermined conditions, or it may exclude training biometric information with a low evaluation value based on predetermined conditions, or it may exclude training biometric information with an evaluation value that is not within a predetermined range), thereby selecting training biometric information.

[0083] In addition, when the machine learning model is a model for extracting discrimination features, if the distance between the discrimination features extracted from biometric information and the representative discrimination features for each individual used during learning is too close, the effectiveness of learning is low, and if it is too far, learning becomes difficult. Therefore, the model update unit 1150 is expected to perform efficient and highly accurate learning by, for example, calculating the distance between the discrimination features extracted from the biometric information for learning and the representative discrimination features of the individual to which the biometric information belongs, and leaving only the data of individuals whose distance falls within a predetermined range.

[0084] Furthermore, for example, the model update unit 1150 may calculate an evaluation value indicating the authentication accuracy of the synthetic biometric information generated using the attribute features used when generating the synthetic biometric information, and may select training biometric information having a high evaluation value based on predetermined conditions, may select training biometric information having a low evaluation value based on predetermined conditions, or may select training selection information having an evaluation value within a predetermined range.

[0085] For example, if the performance of a machine learning model deteriorates when attribute features are changed, the performance of the model can be expected to improve by selectively sampling training biometric information that has attribute features that fall within the range of change.

[0086] Also, for example, the model update unit 1150 may include the synthetic biometric information generated in step S2570 in the selection result in step S2580 in order to ensure that the synthetic biometric information generated in step S2570 is reflected in the update of the machine learning model.

[0087] The model update unit 1150 updates the machine learning model acquired from the learning target model storage unit 1192 based on the machine learning model and the biometric information acquired in step S2580, and stores the updated machine learning model in the learning target model storage unit 1192 (S2590).

[0088] For example, if the machine learning model to be updated is a neural network that outputs discriminative features when a facial image is input, it would be ideal if all discriminative features obtained as output when any facial image of a single person is input into the model are the same.

[0089] In this case, the model updating unit 1150 can update the machine learning model by adopting the sum of the squares of the distances between identification features of the same person as a loss function and minimizing the value of the loss function. Alternatively, the model updating unit 1150 can generate a representative identification feature for each individual, define a loss function based on the distance between each of the identification features other than the representative identification feature for each individual, and update the machine learning model by minimizing the loss function.

[0090] The model update unit 1150 can generate a representative discrimination feature by calculating a predetermined statistical quantity such as the mean or median for a set of discrimination features generated by inputting multiple facial images of the same person into the machine learning model.

[0091] Furthermore, if the distance between the discrimination features of two people is small, it becomes difficult to discriminate between the two people using the discrimination features, so ideally the distance between any two people's discrimination features should be large (for example, equal to or greater than a predetermined value). Therefore, it is also effective for the model update unit 1150 to define a loss function whose value decreases as the distance between the discrimination features of different people or between representative discrimination features becomes smaller, and update the machine learning model by minimizing the value of the loss function.

[0092] When performing these learning processes in a manner suitable for biometric information that reflects attribute features obtained from an actual operating environment, one example is a method of minimizing a loss function based on the distance between a discrimination feature that has a small distance from other discrimination features of the same person and / or a discrimination feature that is easy to discriminate, and the discrimination feature that is output when the synthetic biometric information generated in step S2570 is used as input.

[0093] Furthermore, different machine learning models may be used depending on the environmental attribute features and the biometric attribute features. For example, by defining a machine learning model corresponding to a device ID included in the environmental attribute features, it is possible to learn and build a machine learning model specialized for biometric information having attribute features obtained from that device ID.

[0094] As described above, the machine learning system in this embodiment can perform machine learning based on attribute information obtained from biometric information in a real environment, without collecting the biometric information in the real environment acquired by the client terminal 1000 in the server 1100. In other words, the machine learning system in this embodiment can improve performance by adapting a machine learning model for personal authentication to a real environment, without collecting the user's biometric information itself. [Example]

[0095] The machine learning system of the first embodiment manages biometric attribute features and environmental attribute features obtained during biometric authentication independently of the authentication result, and uses them to update attribute feature distribution parameters for machine learning. On the other hand, there may be cases where it is desired to sample the attribute features of a specific user.

[0096] In addition, by defining dedicated attribute feature distribution parameters for each user ID, when a user with a low authentication success rate is found, the attribute feature distribution parameters of the user or the biometric information generated using attribute features sampled from the attribute feature distribution parameters can be used as auxiliary information to identify the cause of the low authentication success rate.

[0097] Under these circumstances, the machine learning system of this embodiment manages biometric attribute features, environmental attribute features, and attribute feature distribution parameters for each user ID based on the results of biometric authentication, and generates biometric information and updates the machine learning model. The machine learning system of this embodiment executes the processes of FIGS. 4 and 5 instead of the process of FIG. 3. The following mainly describes the differences between the process of this embodiment and the process of Example 1.

[0098] Fig. 4 is a sequence diagram showing an example of biometric authentication processing and machine learning processing. In the processing of Fig. 4, attribute information distribution parameters are managed for each user ID based on the biometric authentication matching result. Compared to the first embodiment, in this embodiment, processing for performing biometric authentication is added.

[0099] The user ID acquisition unit 1040 of the client terminal 1000 acquires the user ID (S3005). Specifically, for example, the user ID acquisition unit 1040 acquires the user ID through input by the user, such as keyboard input, reading of a two-dimensional code with a camera or scanner, or reading of an IC card.

[0100] If 1:N authentication is adopted as the biometric authentication method, which identifies all registered users without requiring input of a user ID or the like, the process of step S3005 does not need to be performed.

[0101] The processes of steps S3010, S3020, and S3030 are similar to the processes of steps S2010, S2020, and S2030 in the first embodiment, respectively.

[0102] The identification feature extraction unit 1050 of the client terminal 1000 extracts an identification feature from the biometric information acquired in step S3010 (S2035). For example, a multidimensional vector obtained by inputting the biometric information into a machine learning model such as a neural network trained for personal identification, and information on specific patterns and feature points extracted from an image containing the biometric information are both examples of an identification feature.

[0103] The client terminal 1000 transmits the user ID acquired in step S3005, the biometric attribute features acquired in step S3020, the environmental attribute features acquired in step S3030, and the identification features acquired in step S3035 to the server 1100 (S3040). The server 1100 receives the user ID, biometric attribute features, environmental attribute features, and identification features transmitted from the client terminal 1000 in step S3040 (S3510).

[0104] The discriminating feature matching unit 1170 of the server 1100 matches the discriminating feature received in step S3510 with the registered discriminating feature stored in the registered discriminating feature storage unit 1193 (S3610).

[0105] If 1:1 authentication is adopted as the biometric authentication method (if the processing of step S3010 is performed), then in step S3610 the discriminating feature matching unit 1170 compares the discriminating feature received in step S3510 with a registered discrimination feature associated with the user ID received in step S3510 among the registered discrimination features stored in the registered discrimination feature storage unit 1193. If 1:N authentication is adopted as the biometric authentication method (if the processing of step S3010 is not performed), then in step S3610 the discriminating feature matching unit 1170 compares, for example, the discriminating feature received in step S3510 with each of the registered discrimination features stored in the registered discrimination feature storage unit 1193.

[0106] In step S3610, the discriminant feature matching unit 1170 performs matching using, for example, a dissimilarity based on the distance between discriminant features. In this case, the discriminant feature matching unit 1170 calculates, for example, a Hamming distance or a Euclidean distance between discriminant features as the dissimilarity. The discrimination feature matching unit 1170 uses threshold processing or the like on the calculated dissimilarity to determine the authentication result indicating whether the authentication is successful or unsuccessful.

[0107] Note that the value calculated by the identification feature matching unit 1170 for matching is not limited to a continuous value such as similarity or dissimilarity. For example, if a template protection technology is applied to biometric information and a template is generated as an identification feature, the identification feature matching unit 1170 obtains either of two values, authentication success or authentication failure, as the matching result. In this case, the identification feature matching unit 1170 does not need to perform the threshold processing described above, and outputs the matching result as the authentication result.

[0108] The attribute feature sampling unit 1120 receives the authentication result output in step S3610, and if the authentication is successful, updates the attribute feature distribution parameters linked to the user ID of the successfully authenticated user using the biometric attribute features and environmental attribute features received in step S3510 (S3520). Through the above processing, the attribute feature distribution parameters can be managed and updated in association with the user ID, making it possible to sample attribute features on a per-user basis.

[0109] The processes of steps S3530, S3540, S3550, S3560, S3570, S3580, and S3590 are similar to steps S2530, S2540, S2550, S2560, S2570, S2580, and S2590 in the first embodiment, respectively.

[0110] As a result, in this embodiment, attribute information obtained in a real operating environment is reflected in existing biometric information that can be used for learning, and it is possible to train a high-performance machine learning model that captures differences in attribute features on a user-by-user basis.

[0111] If the authentication in step S3610 fails, the process in FIG. 4 may be terminated, or in step S3520, the attribute feature sampling unit 1120 may update the attribute feature distribution parameters, which are managed without distinguishing users, using the biometric attribute features and environmental attribute features received in step S3510, and then execute the processes from step S3530 onwards.

[0112] Fig. 5 is a sequence diagram showing an example of enrollment processing for biometric authentication processing. It is desirable that the processing of Fig. 5 be executed before the processing of Fig. 4. The processing of steps S4005, S4010, and S4035 in Fig. 5 is the same as the processing of steps S3005, S3010, and S3035 in Fig. 4, respectively. However, the discrimination feature extracted as the discrimination feature for enrollment in step S4035 may differ from the discrimination feature extracted in step S3035 during authentication. Furthermore, the processing of step S4005 is not omitted.

[0113] The client terminal 1000 transmits the user ID acquired in step S4005 and the discrimination feature acquired in step S4035 to the server 1100 (S4040). The server 1100 receives the discrimination feature and user ID transmitted from the client terminal 1000 in step S4040 (S4510), and stores the received discrimination feature and user ID in association with each other in the registered discrimination feature storage unit 1193 (S4520).

[0114] When the above-mentioned template is used as a target for matching with the discrimination feature during authentication, the server 1100 generates an enrollment template from the discrimination feature received in step S4510, associates the generated enrollment template with the user ID received in step S4510, and stores the associated enrollment template in the enrollment discrimination feature storage unit 1193. The enrollment template is information that can be matched with the discrimination feature obtained during authentication. When generating the template, the server 1100 may apply a predetermined template protection technology to prevent the original discrimination feature and biometric information from being leaked from the enrollment template.

[0115] 5 is executed, personal authentication can be performed based on the processing from step S3005 to step S3040 by the client terminal 1000 and the processing from step S3510 to step S3610 by the server 1100 in FIG.

[0116] The present invention is not limited to the above-described embodiments, but includes various modifications. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, delete, or replace part of the configuration of each embodiment with other configurations.

[0117] Furthermore, the above-described configurations, functions, processing units, processing means, etc. may be partially or entirely implemented in hardware, for example, by designing them as integrated circuits. The above-described configurations, functions, etc. may also be implemented in software, with a processor interpreting and executing a program that implements each function. Information such as the programs, tables, and files that implement each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0118] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0119] 1000 client terminal, 1020 biometric attribute feature extraction unit, 1030 environmental attribute feature acquisition unit, 1050 discrimination feature extraction unit, 1100 server, 1110 parameter update unit, 1120 attribute feature sampling unit, 1130 discrimination feature extraction unit, 1140 biometric information generation unit, 1150 model update unit, 1160 attribute feature extraction unit, 1170 discrimination feature matching unit, 1190 attribute feature distribution parameter storage unit, 1191 learning biometric information storage unit, 1192 learning target model storage unit, 1193 registered discrimination feature storage unit, 8000 computer, 8010 CPU, 8020 memory, 8030 auxiliary storage device, 8060 communication device

Claims

1. A machine learning device, comprising: a processor and a memory, The memory includes: real-world attribute features corresponding to biometric information of a target user; a distribution parameter indicating a distribution of the attribute feature quantity; training biometric information, which is the biometric information for training; a machine learning model that outputs information indicating characteristics of a living body when the biometric information is input; the attribute feature is a feature that affects authentication accuracy of biometric authentication based on the biometric information and has a low correlation with a discrimination feature extracted from the biometric information based on a predetermined condition, the identification feature is extracted from the biometric information and is used for matching in the biometric authentication, The processor: updating the distribution parameters based on the real environment attribute feature values; extracting attribute features from the updated distribution parameters; extracting, from the learning biometric information, learning discrimination features that are the discrimination features for learning; generating synthetic biometric information based on the extracted attribute features and the learning discrimination features; A machine learning device that updates the machine learning model based on the synthetic biometric information and the training biometric information.

2. The machine learning device according to claim 1 , The memory includes: an authentication identification feature that is the identification feature extracted from biometric information of the target user; the distribution parameters corresponding to each of a plurality of users including the target user; a registered identification feature that is the identification feature registered in advance corresponding to the target user; The processor: performing authentication of the target user by comparing the authentication identification feature with the registered identification feature; When it is determined that the authentication of the target user is successful, the machine learning device updates a distribution parameter corresponding to the target user based on the real environment attribute feature.

3. The machine learning device according to claim 1 , The memory includes: Information indicating the target store where the biometric information of the target user was acquired; The distribution parameters corresponding to each of a plurality of stores are stored; The processor updates a distribution parameter corresponding to the target store based on the actual environment attribute feature.

4. The machine learning device according to claim 1 , The attribute feature indicates an environment in which the biometric information is acquired, and includes an environmental attribute feature acquired from a device installed in the environment in which the biometric information is acquired.

5. The machine learning device according to claim 4, A machine learning device, wherein the environmental attribute feature includes at least one of the date and time when the biometric information was acquired, the illuminance when the biometric information was acquired, and location information when the biometric information was acquired.

6. The machine learning device according to claim 1 , The attribute feature includes a biometric attribute feature extracted from the biometric information.

7. The machine learning device according to claim 6, When the biometric information is an image of a part of a user's body, the biometric attribute feature includes at least one of the orientation of the part of the body included in the image, the tilt of the part of the body included in the image, the color of the part of the body included in the image, the presence or absence of accessories included in the image, the position at which accessories are worn included in the image, the brightness of the part of the body included in the image, and the contrast of the part of the body included in the image.

8. The machine learning device according to claim 6, The biometric attribute feature is a feature that has a high similarity based on predetermined conditions when extracted from the biometric information acquired in the same environment.

9. The machine learning device according to claim 1 , The machine learning device is configured to output the discrimination feature when the biometric information is input to the machine learning model.

10. 1. A machine learning system, comprising: A client terminal and a server are provided, the client terminal holds real-world attribute features that are attribute features corresponding to biometric information of a target user; The server a distribution parameter indicating a distribution of the attribute feature quantity; training biometric information, which is the biometric information for training; a machine learning model that outputs information indicating characteristics of a living body when the biometric information is input; the attribute feature is a feature that affects authentication accuracy of biometric authentication based on the biometric information and has a low correlation with a discrimination feature extracted from the biometric information based on a predetermined condition, the identification feature is extracted from the biometric information and is used for matching in the biometric authentication, the client terminal transmits the real environment attribute feature to the server; The server updating the distribution parameters based on the real environment attribute feature values; extracting attribute features from the updated distribution parameters; extracting, from the learning biometric information, learning discrimination features that are the discrimination features for learning; generating synthetic biometric information based on the extracted attribute features and the learning discrimination features; A machine learning system that updates the machine learning model based on the synthetic biometric information and the training biometric information.

11. A machine learning method using a machine learning device, the machine learning device includes a processor and a memory; The memory includes: real-world attribute features corresponding to biometric information of a target user; a distribution parameter indicating a distribution of the attribute feature quantity; training biometric information, which is the biometric information for training; a machine learning model that outputs information indicating characteristics of a living body when the biometric information is input; the attribute feature is a feature that affects authentication accuracy of biometric authentication based on the biometric information and has a low correlation with a discrimination feature extracted from the biometric information based on a predetermined condition, the identification feature is extracted from the biometric information and is used for matching in the biometric authentication, The machine learning method includes: the processor updates the distribution parameters based on the real environment attribute feature values; the processor extracts attribute features from the updated distribution parameters; the processor extracts, from the training biometric information, training discrimination features that are the discrimination features for training; the processor generates synthetic biometric information based on the extracted attribute features and the learning discrimination features; A machine learning method in which the processor updates the machine learning model based on the synthetic biometric information and the training biometric information.

Images