Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises an estimation unit for learning and an updating unit. The estimation unit for learning uses an estimation model for estimating ellipse parameters pertaining to the contour or edge of the pupil and the iris which are included in an iris image and estimates the ellipse parameter pertaining an iris image for learning. The updating unit uses the estimated ellipse parameters and correct ellipse parameters that are correct answers of the ellipse parameters estimated for the iris image for learning, and updates model parameters included in the estimation model. The ellipse parameters include a shared parameter shared by the pupil and the iris included in the iris image.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing device, an information processing method, and a recording medium.

[0002] For example, Patent Document 1 describes that for robust feature detection using an eye shape model, pupil detection can be improved by removing erroneous pupil boundary points. In Patent Document 1, erroneous pupil boundary points can be created when the eyelid partially occludes the pupil. The erroneous pupil boundary points reflect the position of the eyelid rather than the true boundary of the pupil.

[0003] The patent document 1 describes that once erroneous pupil boundary points have been identified and removed, an ellipse may be fitted to the pupil using the remaining pupil boundary points, and that algorithms that may be implemented for such ellipse fitting include differential and integral operators, least squares methods, random sample consensus (RANSAC), or ellipse or curve fitting algorithms.

[0004] Special Publication No. 2022-538669

[0005] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above.

[0006] The information processing device according to the present disclosure includes: a training estimation means for estimating ellipse parameters relating to a training iris image using an estimation model for estimating the ellipse parameters relating to the outline or edge of a pupil and an iris included in the iris image; and an update means for updating model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image.

[0007] The information processing method in the present disclosure includes, by one or more computers, estimating ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters for the contour or edge of the pupil and iris included in the iris image, and updating model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image.

[0008] The information processing method of the present disclosure is a recording medium on which a program is recorded, which causes one or more computers to estimate ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters for the contour or edge of the pupil and iris included in the iris image, and update model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image.

[0009] 1 is a diagram illustrating an example configuration of a first information processing system according to the present disclosure. FIG. 2 is a block diagram illustrating an example configuration of a first information processing device according to the present disclosure. FIG. 3 is a flowchart illustrating an example processing operation of a first information processing device according to the present disclosure. FIG. 4 is a diagram illustrating an example of an iris image according to the present disclosure. FIG. 5 is a diagram illustrating an example of an ellipse specifying parameter and an example of ellipse parameters using contraction parameter examples 1 and 2 according to the present disclosure. FIG. 6 is a diagram illustrating an example of ellipse parameters including a gaze direction represented in two dimensions as an auxiliary parameter according to the present disclosure. FIG. 7 is a diagram illustrating an example of key points related to an eyelid according to the present disclosure. FIG. 8 is a diagram illustrating an example of key points related to a pupil and an iris according to the present disclosure. FIG. 9 is a diagram illustrating an example of ellipse parameters including positions of n key points as auxiliary parameters according to the present disclosure. FIG. 10 is a diagram illustrating another example of an iris image according to the present disclosure, showing an example of an iris image in which the distance between the upper and lower eyelids is narrower than in FIG. 4 (so-called half-closed eyes). FIG. 11 is a block diagram illustrating an example configuration of a first update unit according to the present disclosure. FIG. 12 is a flowchart illustrating an example processing operation of a first update unit according to the present disclosure. FIG. 13 is a diagram illustrating an example physical configuration of a first information processing device according to the present disclosure. FIG. 14 is a block diagram illustrating an example configuration of a second information processing device according to the present disclosure. FIG. 15 is a block diagram illustrating an example configuration of a second information processing device according to the present disclosure. 1 is a block diagram showing a configuration example of a third information processing device according to the present disclosure. FIG. 1 is a block diagram showing a configuration example of a third learning estimator according to the present disclosure. FIG. 2 is a flowchart showing an example of a processing operation of the third learning estimator according to the present disclosure. FIG. 3 is a block diagram showing a configuration example of a first ellipse specifying parameter estimator according to the present disclosure. FIG. 4 is a block diagram showing a configuration example of a first loss calculation unit according to the present disclosure. FIG. 5 is a flowchart showing an example of a processing operation of the first loss calculation unit according to the present disclosure. FIG. 6 is a block diagram showing a configuration example of a fourth information processing device according to the present disclosure. FIG. 7 is a flowchart showing an example of a processing operation of the fourth information processing device according to the present disclosure. FIG. 8 is a block diagram showing a configuration example of a fifth information processing device according to the present disclosure. FIG. 9 is a flowchart showing an example of a processing operation of the fifth information processing device according to the present disclosure. FIG. 10 is a block diagram showing a configuration example of a fifth update unit according to the present disclosure.1 is an example of an iris image in a state where a part of the iris is hidden by an eyelid according to the present disclosure; FIG. 2 is a diagram showing an example of the configuration of a third information processing system according to the present disclosure; FIG. 3 is a block diagram showing an example of the configuration of a sixth information processing device according to the present disclosure; FIG. 4 is a flowchart showing an example of the processing operation of the sixth information processing device according to the present disclosure.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In all drawings, similar components are denoted by similar reference numerals, and descriptions thereof will be omitted as appropriate. In the present disclosure, the drawings relate to one or more embodiments.

[0011] First Embodiment (Overview) Generally, the iris and pupil included in an iris image are often elliptical rather than perfectly circular.

[0012] Patent Document 1 describes that, as described above, an algorithm may be used to fit an ellipse to the pupil using the remaining pupil boundary points after removing the erroneous pupil boundary points.

[0013] However, there is no description of a technique for estimating the iris and pupil contained in an iris image. Furthermore, the algorithm for fitting an ellipse to the pupil is merely described as a list of its constituent elements. Therefore, even if the technique described in Patent Document 1 is used, it may not be possible to accurately estimate the iris and pupil contained in an iris image.

[0014] One of the objectives of the present disclosure is to accurately estimate the iris and pupil contained in an iris image.

[0015] (Configuration Example of Information Processing System S1) As shown in FIG. 1, the information processing system S1 includes a training data storage device 90 that stores training iris images and correct answer data related to the training iris images, and an information processing device 100.

[0016] (Configuration Example of Information Processing Device 100) As shown in FIG. 2, the information processing device 100 includes a learning estimation unit 110 and an update unit 120.

[0017] The training estimation unit 110 estimates ellipse parameters for the training iris image using an estimation model for estimating ellipse parameters for the outline or edge of the pupil and iris included in the iris image.

[0018] The update unit 120 updates the model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image.

[0019] The ellipse parameters include common parameters that are common to the pupil and iris included in the iris image.

[0020] According to the information processing system S1, it is possible to construct an estimation model that estimates the shapes of the iris and pupil included in the iris image as ellipses, thereby enabling the iris and pupil included in the iris image to be estimated with high accuracy.

[0021] According to the information processing device 100, it is possible to construct an estimation model that estimates the shapes of the iris and pupil included in the iris image as an ellipse, thereby enabling the iris and pupil included in the iris image to be estimated with high accuracy.

[0022] (Example of Processing Operation of Information Processing Device 100) The information processing device 100 executes information processing as shown in FIG.

[0023] The training estimation unit 110 estimates ellipse parameters for the training iris image using an estimation model for estimating ellipse parameters for the outline or edge of the pupil and iris included in the iris image (step S110).

[0024] The update unit 120 updates the model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image (step S120).

[0025] The ellipse parameters include common parameters that are common to the pupil and iris included in the iris image.

[0026] This information processing makes it possible to construct an estimation model that estimates the shapes of the iris and pupil contained in the iris image as ellipses, thereby enabling the iris and pupil contained in the iris image to be estimated with high accuracy.

[0027] (Detailed Example) The training data storage device 90 and the information processing device 100 are connected to each other via a communication network NT. As a result, the training data storage device 90 and the information processing device 100 transmit and receive information to each other via the communication network NT. The communication network NT may be wired, wireless, or a combination of these.

[0028] (Training Data Storage Device 90) For example, training data is stored in advance in the training data storage device 90. The training data includes, for example, training iris images and correct answer data related to the training iris images.

[0029] The training iris images are iris images used for training the estimation model.

[0030] An iris image is an image that includes the pupil and iris of a subject. In the following, an example in which the subject is a human will be described. Note that the subject is not limited to a human, and may be an animal such as a dog, cat, pig, cow, bird, or snake.

[0031] 4 is a diagram showing an example of an iris image, which includes the upper and lower eyelids and the white of the eye in addition to the pupil and iris.

[0032] As described above, the iris image is not limited to the iris image shown in FIG. 4 as long as it includes the pupil and iris. That is, the iris image may include only the pupil and iris. For example, the iris image may also include the pupil and iris, and at least one of the upper and lower eyelids, the white of the eye including the conjunctiva and ocular membrane, the eyelashes, the eyebrows, etc. For example, the iris image may also include both eyes. For example, the iris image may also include parts of the face, such as the forehead, between the eyebrows, temples, nose, cheeks, mouth, and hair, or the entire face, or other parts or the entire body.

[0033] The iris image may be, for example, an image obtained by photographing the pupil and iris of a subject in a wavelength band in the visible light region or the near-infrared region. The iris image may be, for example, an image obtained by processing the image obtained by photographing the pupil and iris of a subject. For example, the processing may involve general image processing that can substantially maintain the shape of the pupil and iris included in the iris image. In more detail, for example, the processing may include, but is not limited to, downsampling, cutting out a region including the iris, etc.

[0034] The number of training iris images may be one, but typically multiple. Generally, it is difficult to obtain a large number of training iris images by photographing them all. Therefore, the training iris images may be iris images that have been processed from photographed images, etc.

[0035] The correct answer data will be described in detail later.

[0036] The information processing system S1 may include an imaging device connected to the communication network NT instead of or in addition to the training data storage device 90. This imaging device may be, for example, a camera or the like, and is used to capture images of the pupil and iris of a subject. When an imaging device is included, an iris image of the pupil and iris captured by the imaging device may be used as the training iris image. Furthermore, the training data storage device 90 may have a function for performing image processing to obtain training iris images, and may store the training iris images obtained by this image processing.

[0037] A storage unit for storing training data may be provided in the information processing device 100. In this case, the information processing system S1 does not need to include a training data storage device 90 separate from the information processing device 100.

[0038] (Regarding the training estimation unit 110) The training estimation unit 110 estimates ellipse parameters relating to the contours or edges of the pupil and iris included in the training iris image using an estimation model currently being trained. The estimation model is, for example, a machine learning model for estimating ellipse parameters relating to the contours or edges of the pupil and iris included in the iris image. The estimation model may estimate one or more parameters. The estimation model includes one or more model parameters and is configured using, for example, a neural network.

[0039] In more detail, for example, the training estimation unit 110 includes an estimation model. The training estimation unit 110 acquires training iris images in response to, for example, a user instruction. For example, the training estimation unit 110 acquires the training iris images from the training data storage device 90.

[0040] For example, the training estimation unit 110 inputs the acquired training iris image to the estimation model. In response to this input, the estimation model outputs ellipse parameters related to the input training iris image. In this way, the training estimation unit 110 estimates ellipse parameters related to the training iris image.

[0041] When training the estimation model using multiple training iris images, the training estimation unit 110 may acquire multiple training iris images. In this case, the training estimation unit 110 may, for example, input each of the acquired training iris images to the estimation model. For example, the training estimation unit 110 may input each of the training iris images to the estimation model sequentially. In response to this input, the estimation model outputs ellipse parameters for each of the input training iris images. In this way, the training estimation unit 110 may estimate ellipse parameters for each of the training iris images.

[0042] An example of training an estimation model using multiple training iris images is so-called mini-batch training, which is a machine learning model training method that repeatedly trains an estimation model using a group of images (mini-batch) consisting of, for example, 128 training iris images.

[0043] Note that the batch size used in mini-batch learning (i.e., the number of training iris images constituting a mini-batch) is not limited to the above-mentioned 128, and may be determined as appropriate. Furthermore, the method for learning the estimation model is not limited to mini-batch learning.

[0044] (Regarding Ellipse Parameters) Ellipse parameters are parameters relating to an ellipse that represents the shape of the iris and pupil included in an iris image. The ellipse is, for example, an ellipse that represents the outline or edge of the iris and pupil. The outline or edge of the iris may also be referred to as the boundary between the iris and the white of the eye, the boundary between the cornea and the sclera, etc. The outline or edge of the pupil may also be referred to as the boundary between the iris and the pupil, etc. The ellipse parameters may be expressed using vectors, for example.

[0045] The ellipse parameters include at least one shared parameter, which is a parameter shared with respect to the pupil and the iris included in the iris image, and which may be correlated with the shapes of the pupil and the iris included in the iris image.

[0046] The shared parameters may include contraction parameters and / or auxiliary parameters.

[0047] The contraction parameter is a parameter that contracts the pupil shape parameter and the iris shape parameter. The number of pupil shape parameters may be one or more. The number of iris shape parameters may be one or more.

[0048] The auxiliary parameters are parameters that are added when training an estimation model. There may be one or more auxiliary parameters.

[0049] (Pupil Shape Parameter and Iris Shape Parameter) The pupil shape parameter is a parameter for specifying a pupil ellipse, which is an ellipse that represents the shape of the pupil included in the iris image.

[0050] The iris shape parameters are parameters for specifying the iris ellipse, and the pupil ellipse is an ellipse that represents the shape of the iris included in the iris image.

[0051] In general, an ellipse can be represented by, for example, a center position, two axis radii (major axis radius and minor axis radius), and an ellipse rotation angle. Also, for example, an ellipse can be represented by, for example, a center position, a major axis radius (or a minor axis radius), an ellipticity, and an ellipse rotation angle.

[0052] The center position is the position of the center of the ellipse. The center position may be expressed using, for example, the x-coordinate and the y-coordinate of the center of the ellipse.

[0053] The major axis radius and the minor axis radius mean half the length of the major axis and the minor axis of the ellipse, respectively. An ellipse with equal major and minor radii corresponds to a circle. Note that the major axis diameter and the minor axis diameter may be used instead of the major axis radius and the minor axis radius, respectively.

[0054] Ellipticity is the ratio of the major axis radius to the minor axis radius, and is generally a value obtained using the formula "minor axis radius / major axis radius." An ellipse with an ellipticity of 1 corresponds to a circle. Note that the minor axis diameter and major axis diameter may be used instead of the minor axis radius and major axis radius, respectively. Instead of ellipticity, for example, "(major axis radius - minor axis radius) / major axis radius," "(major axis radius - minor axis radius) / minor axis radius," the reciprocals of these values, the reciprocal of ellipticity, etc. may be used.

[0055] The ellipse rotation angle is an angle indicating the rotation angle of an ellipse. The ellipse rotation angle may be expressed, for example, as the angle of the major axis (or minor axis) of the ellipse relative to a reference direction.

[0056] When these examples are applied to each of the pupil shape parameters and the iris shape parameters, each of the pupil shape parameters and the iris shape parameters includes five parameters.

[0057] Here, the parameters including the pupil shape parameters and the iris shape parameters are also referred to as "ellipse-specifying parameters." That is, the ellipse-specifying parameters include the pupil shape parameters and the iris shape parameters. If the pupil shape parameters and the iris shape parameters each include five parameters, the ellipse-specifying parameters include ten parameters.

[0058] In more detail, for example, suppose the pupil shape parameters include five parameters (P1, P2, P3, P4, P5). Also, suppose the iris shape parameters include five parameters (Q1, Q2, Q3, Q4, Q5). The contents represented by P1 to P5 and Q1 to Q5 are, for example, as follows:

[0059] P1: x-coordinate of the pupil center P2: y-coordinate of the pupil center P3: semi-major axis radius of the pupil ellipse P4: ellipticity of the pupil ellipse P5: ellipse rotation angle of the pupil ellipse

[0060] Q1: x coordinate of the iris center Q2: y coordinate of the iris center Q3: semi-major axis radius of the iris ellipse Q4: ellipticity of the iris ellipse Q5: ellipse rotation angle of the iris ellipse

[0061] P1 to P5 and Q1 to Q10 shown here are examples of when the iris ellipse and pupil ellipse are expressed using the center position, major axis radius, ellipticity, and ellipse rotation angle. This example will also be used in the following explanation.

[0062] Note that the iris ellipse and the pupil ellipse may each be represented using a center position, two axis radii (major axis radius and minor axis radius), and an ellipse rotation angle, or may be represented by other methods. For example, the iris ellipse and the pupil ellipse may be represented by different methods. For example, the ellipse parameters may be represented by a rectangle inscribed in the ellipse, and more specifically, by the coordinates of two diagonal vertices of a rectangle inscribed or circumscribed in the ellipse and the ellipse rotation angle. Furthermore, for example, one of the pupil shape parameters and the iris shape parameters may be represented by the amount of deviation based on the other.

[0063] (Regarding Contraction Parameters) As described above, the contraction parameters are parameters that contract the pupil shape parameters and the iris shape parameters. When the ellipse parameters include a contraction parameter, the contraction parameter may be one or more parameters configured from one or more types of contraction parameters.

[0064] Examples of the types of contraction parameters include the following contraction parameter examples 1 and 2. Note that the types of contraction parameters are not limited to contraction parameter examples 1 and 2.

[0065] Example 1 of contracted parameters: Predetermined parameters included in pupil shape parameters and iris shape parameters

[0066] Example 2 of reduced parameters: Parameters representing components whose dimensions have been reduced by performing statistical analysis processing on pupil shape parameters and iris shape parameters

[0067] That is, the contraction parameters may include at least one of predetermined parameters included in the pupil shape parameters and the iris shape parameters, and parameters representing components whose dimensionality has been reduced by performing statistical analysis processing on the pupil shape parameters and the iris shape parameters.

[0068] Contraction parameter examples 1 and 2 are described below.

[0069] (Example 1 of the contraction parameter) The contraction parameter may be a predetermined parameter included in the pupil shape parameter and the iris shape parameter. The parameter predetermined as the contraction parameter may be, for example, a parameter having a correlation between the pupil shape parameter and the iris shape parameter.

[0070] A parameter having a correlation between a pupil shape parameter and an iris shape parameter means, for example, a parameter having a relationship in which when the value of one of the pupil shape parameter and the iris shape parameter changes, the value of the other also changes.

[0071] The degree of correlation can be expressed, for example, by a correlation coefficient. A parameter having a correlation between a pupil shape parameter and an iris shape parameter may be a parameter whose correlation coefficient is equal to or greater than a predetermined threshold. The threshold here may be, for example, 0.2 or greater, or may be 0.2 or greater and 0.8 or less, but is not limited to these. Note that the value representing the degree of correlation is not limited to the correlation coefficient.

[0072] Examples of parameters that have a correlation between pupil shape parameters and iris shape parameters include ellipticity and ellipse rotation angle. This is because, in general, there is often a high correlation between the ellipticity of the pupil ellipse and the iris ellipse. Also, there is often a high correlation between the ellipticity of the pupil ellipse and the iris ellipse.

[0073] Therefore, the contraction parameter may be, for example, a parameter corresponding to at least one of the ellipticity and the ellipse rotation angle of the pupil ellipse and the iris ellipse. Note that the parameter predetermined as the contraction parameter is not limited to at least one of the ellipticity and the ellipse rotation angle.

[0074] For example, when the ellipticity C1 and the ellipse rotation angle C2 are used as contraction parameters, the ellipse parameters include eight parameters (C1, C2, P1, P2, P3, Q1, Q2, Q3).

[0075] For example, a value corresponding to either P4 or Q4 may be used for C1. As a result, P4 and Q4 are contracted to C1. For example, a value corresponding to either P5 or Q5 may be used for C2. As a result, P5 and Q5 are contracted to C2.

[0076] P1, P2, and P3 included in the ellipse parameters are pupil-specific parameters that are specific to the pupil. In Example 1 of the contracted parameters, the pupil-specific parameters correspond to the parameters included in the pupil shape parameters that are not represented by the contracted parameters.

[0077] Q1, Q2, and Q3 included in the ellipse parameters are iris-specific parameters that are specific to the iris. In Example 1 of Contracted Parameters, the iris-specific parameters correspond to the parameters included in the iris shape parameters that are not expressed in the contracted parameters.

[0078] Here, as described above, the pupil-specific parameter is a parameter specific to the pupil, and the same applies hereinafter. As described above, the iris-specific parameter is a parameter specific to the iris, and the same applies hereinafter. Such pupil-specific parameters and iris-specific parameters can also be said to be parameters that do not have a high correlation for representing the pupil ellipse and the iris ellipse, respectively. Not having a high correlation may mean, for example, that the correlation coefficient between the pupil-specific parameter and the iris-specific parameter is less than a predetermined threshold (e.g., less than 0.2). Here, the predetermined threshold is not limited to 0.2. The value representing the degree of correlation between the pupil-specific parameter and the iris-specific parameter is not limited to the correlation coefficient.

[0079] Note that in Example 1, the parameters predetermined as contraction parameters are not limited to the ellipticity and the ellipse rotation angle. In Example 1, the number of contraction parameters may be one or more. In Example 1, the number of parameters included in the pupil-specific parameters is, for example, the number of parameters included in the pupil shape parameters minus the number of contraction parameters, and is not limited to three. Similarly, the number of parameters included in the iris-specific parameters is, for example, the number of parameters included in the iris shape parameters minus the number of contraction parameters, and is not limited to three. The number of pupil-specific parameters may be one or more. The number of iris-specific parameters may be one or more.

[0080] (Example 2 of Reduced Parameter) The reduced parameter may be a parameter representing a component whose dimension has been reduced by performing a statistical analysis process on the pupil shape parameter and the iris shape parameter. The statistical analysis process is, for example, a process of reducing the dimension of the pupil shape parameter and the iris shape parameter. The component whose dimension has been reduced using the statistical analysis process may be, for example, a component that has a correlation with at least one parameter included in each of the pupil shape parameter and the iris shape parameter.

[0081] An example of statistical analysis processing is principal component analysis. That is, the contraction parameters may be parameters representing eigenvalues ​​of principal components identified by performing principal component analysis on pupil shape parameters and iris shape parameters. The principal components used in the contraction parameters may be one or more.

[0082] In more detail, for example, assume that a first principal component and a second principal component representing all of the pupil shape parameters and the iris shape parameters are obtained by performing principal component analysis. If the parameters representing the first principal component and the second principal component as contraction parameters are S1 and S2, the ellipse parameters include two components (S1, S2). Note that the number of principal components after dimension reduction by principal component analysis is not limited to two. For example, the number of principal components may be determined in advance, or may be determined by setting a threshold value for the contribution rate of each principal component.

[0083] 5 is a diagram showing examples of ellipse-specifying parameters (pupil shape parameters and iris shape parameters) and examples of ellipse parameters using contraction parameter examples 1 and 2. As can be seen from the diagram, by using contraction parameters as ellipse parameters, the number of parameters for expressing the shapes of the iris and pupil included in the iris image as an ellipse can be reduced compared to the number of ellipse-specifying parameters.

[0084] The statistical analysis process is not limited to principal component analysis. When components obtained by the statistical analysis process are used as contraction parameters, the number of components is not limited to two, and may be one, or three or more.

[0085] (Auxiliary Parameters) As described above, auxiliary parameters are parameters that are added when training an estimation model. That is, auxiliary parameters may be output from an estimation model currently being trained and used to update model parameters, which will be described later. On the other hand, when a trained estimation model is used, the trained estimation model does not need to output auxiliary parameters. Note that the above-mentioned contraction parameters may be output not only from an estimation model currently being trained, but also from a trained estimation model.

[0086] The auxiliary parameter may be, for example, a parameter that is correlated with at least one parameter included in each of the pupil shape parameter and the iris shape parameter. By using the auxiliary parameter, it is possible to improve the estimation accuracy of a component that is correlated with the auxiliary parameter among the components included in each of the pupil shape parameter and the iris shape parameter.

[0087] When the ellipse parameters include auxiliary parameters, the auxiliary parameters may be one or more parameters made up of one or more types of auxiliary parameters.

[0088] Examples of types of auxiliary parameters include the following auxiliary parameter examples 1 and 2. Note that the types of auxiliary parameters are not limited to auxiliary parameter examples 1 and 2.

[0089] Auxiliary parameter example 1: Parameter representing gaze direction

[0090] Auxiliary parameter example 2: A parameter that represents the position of one or more keypoints

[0091] That is, the auxiliary parameters may include at least one of a parameter representing a gaze direction and a parameter representing the positions of one or more key points.

[0092] Examples 1 and 2 of the auxiliary parameters will be described below.

[0093] (Example 1 of Auxiliary Parameter) The auxiliary parameter may be a parameter representing the line of sight.

[0094] A general technique may be used for estimating the gaze direction from an image (gaze estimation). For example, a model-based method, an appearance-based method, or the like may be used for gaze estimation. The model-based method is a technique for estimating the gaze direction using a geometric eyeball model. The appearance-based method is a technique for estimating the gaze direction based on image features of the eye region using an SVM (Support Vector Machine), a multi-layer neural network, a Gaussian process, or the like.

[0095] The gaze direction may be expressed, for example, by a two-dimensional vector (SRx, SRy) or a three-dimensional vector (SRx, SRy, SRz). SRx, SRy, and SRz are parameters corresponding to, for example, the horizontal direction, the vertical direction, and the photographing direction of the photographing device that generated the iris image, respectively. The coordinate system for expressing SRx, SRy, and SRz is not limited to the one exemplified here.

[0096] FIG. 6 is a diagram showing an example of ellipse parameters including a line of sight direction expressed two-dimensionally as an auxiliary parameter.

[0097] Generally, the gaze direction has a strong correlation with the center-to-center distance, ellipticity, and ellipse rotation angle of the iris ellipse and pupil ellipse. Therefore, by using the gaze direction as an auxiliary parameter, it is possible to improve the accuracy of estimating the center-to-center distance, ellipticity, and ellipse rotation angle of the iris ellipse and pupil ellipse.

[0098] The line of sight direction may be used as a contraction parameter. Assume that the line of sight direction is represented by, for example, a two-dimensional vector (SRx, SRy). In this case, the ellipse parameters may include, for example, nine components: (SRx, SRy, D1, D2, D3, P1, P2, P3, Q3).

[0099] In this example of ellipse parameters, SRx, SRy, D1, D2, and D3 are contraction parameters. D1 is an example of a parameter for specifying an ellipticity shared by the pupil and iris using the gaze direction (SRx, SRy). D2 is an example of a parameter for specifying an ellipse rotation angle shared by the pupil and iris using, for example, the gaze direction (SRx, SRy). D3 is an example of a parameter for specifying the iris center using the gaze direction (SRx, SRy) and the pupil center (P1, P2). P1, P2, and P3 are examples of pupil-specific parameters. Q3 is an example of an iris-specific parameter.

[0100] (Example 2 of Auxiliary Parameter) The auxiliary parameter may be a parameter that represents the positions of one or more key points. Key points are points related to the eyes, and are generally also called feature points, landmarks, or the like.

[0101] The key points may include, for example, points relating to at least one of the eyelid, pupil, and iris.

[0102] 7 is a diagram showing examples of key points related to eyelids. In the example shown in the figure, the key points related to the eyelids include the outer corner A of the eye, the inner corner B of the eye, and multiple points on the upper and lower eyelids. The key point for the upper eyelid includes the upper end of the upper eyelid (upper end of the eye) C. The key point for the lower eyelid includes the lower end of the lower eyelid (lower end of the eye) D.

[0103] 8 is a diagram showing examples of key points related to the pupil and iris. In the example shown in the figure, the key points related to the pupil include multiple points at the boundary between the pupil and the iris. In the example shown in the figure, the key points related to the iris include multiple points at the boundary between the iris and the white of the eye.

[0104] If there are n keypoints, the position of the i-th keypoint may be expressed as (Pxi, Pyi), for example, where n is an integer greater than or equal to 1, and i is an integer greater than or equal to 1 and less than or equal to n.

[0105] FIG. 9 is a diagram showing an example of ellipse parameters including the positions of n key points as auxiliary parameters.

[0106] Here, as described above, the auxiliary parameters are output from the estimation model during training. The auxiliary parameters may be output from the trained estimation model, but need not be output. Therefore, the ellipse parameters shown in the figure are examples of ellipse parameters output from the estimation model during training. Furthermore, if the trained estimation model outputs ellipse parameters including auxiliary parameters, the ellipse parameters shown in the figure are also examples of ellipse parameters output from the trained estimation model.

[0107] In general, in an iris image, the pupil ellipse and iris ellipse have a strong correlation with the shape of the eyelid, pupil, and iris, as can be seen by comparing the iris image shown in Fig. 4 with the iris image of half-closed eyes shown in Fig. 10. Therefore, by using key points related to at least one of the eyelid, pupil, and iris as auxiliary parameters, the estimation accuracy of the iris ellipse and pupil ellipse can be improved.

[0108] It should be noted that the key points are not limited to those exemplified here, and the key points may be used as contraction parameters.

[0109] (Regarding the Correct Answer Data) The correct answer data is data including correct answers for ellipse parameters related to training iris images. The correct answer data may be expressed using vectors, for example.

[0110] The correct answer data is used for training the estimation model. Therefore, the correct answer data may include correct answers for ellipse parameters output from the estimation model during training. The correct answers for the ellipse parameters are also referred to as correct ellipse parameters. For example, if the ellipse parameters output from the estimation model during training include auxiliary parameters, the correct answer data may include correct answers for the auxiliary parameters.

[0111] When there are multiple training iris images, the correct answer data includes correct answers for the ellipse parameters associated with each of the training iris images. For example, information for identifying the training iris image (image ID (identification)) may be used to associate the correct answers with the training iris images. Note that the method for associating the correct answers with the training iris images is not limited to this.

[0112] (Regarding the updating unit 120) The updating unit 120 updates the model parameters included in the estimation model using the ellipse parameters estimated by the training estimation unit 110 and ground truth data related to the training iris image. The number of model parameters may be one or more. By updating the model parameters, the estimation model can be trained.

[0113] The update unit 120 includes, for example, a loss calculation unit 121 and a parameter update unit 122 as shown in FIG.

[0114] The loss calculation unit 121 calculates a weighted loss using the ellipse parameters estimated by the training estimation unit 110, ground truth data related to the training iris image, and weights for each corresponding component between the estimated ellipse parameters and the ground truth data.

[0115] The parameter updater 122 updates the model parameters included in the estimation model using the weighted loss calculated by the loss calculator 121 .

[0116] The update unit 120 executes an update process (step S120) as shown in FIG. 12, for example.

[0117] The loss calculation unit 121 calculates a weighted loss using the ellipse parameters estimated in step S110, ground truth data related to the training iris image, and weights for each corresponding component between the estimated ellipse parameters and the ground truth data (step S121).

[0118] The parameter update unit 122 updates the model parameters included in the estimation model using the weighted loss calculated in step S121 (step S122).

[0119] Below, detailed examples of the loss calculation unit 121 and the parameter update unit 122 will be described.

[0120] (Loss Calculation Unit 121) As described above, the loss calculation unit 121 calculates the weighted loss.

[0121] The weighted loss is a loss calculated using the ellipse parameters estimated by the training estimation unit 110 for the training iris image, the correct answer data including the correct answer for the training iris image, and the weights for each corresponding component between the estimated ellipse parameters and the correct answer data.

[0122] The loss is, for example, a value indicating the degree of deviation between the ellipse parameters and the correct answer included in the correct answer data. The weighted loss is a loss calculated using a value indicating the degree of deviation for each component between the estimated ellipse parameters and the correct answer included in the correct answer data and the weight of the corresponding component.

[0123] In more detail, for example, the weighted loss may be the sum of values ​​obtained by multiplying a value indicating the degree of deviation for each component between the estimated ellipse parameters and the correct answer included in the correct answer data by the weight of the corresponding component. Alternatively, for example, the weighted loss may be a value obtained by dividing the sum of values ​​obtained by multiplying a value indicating the degree of deviation for each component by the weight of the corresponding component by the number of parameters included in the ellipse parameters. Note that the number of parameters included in the estimated ellipse parameters is the same as the number of correct answers included in the correct answer data.

[0124] The value indicating the degree of deviation for each component is, for example, the absolute value of the difference for each component, the square of the difference for each component, etc. Note that the value indicating the degree of deviation for each component is not limited to the examples given here.

[0125] The weight for each component may be a predetermined value. The weight for each component may be determined based on a predetermined relational expression, a predetermined condition, etc. An example of a method for determining the weight for each component will be described in detail later.

[0126] The loss calculation unit 121 may calculate the loss using the ellipse parameters estimated by the training estimation unit 110 and ground truth data related to the training iris image. This loss may be, for example, the L1 norm (sum of absolute differences), the L2 norm (sum of squared differences), the L1 mean obtained by dividing the L1 norm by the number of components, or the L2 mean obtained by dividing the L2 norm by the number of components. The L1 mean is also referred to as the mean absolute error, etc. The L2 mean is also referred to as the mean squared error, etc. The parameter update unit 122 may update the model parameters included in the estimation model using the loss calculated by the loss calculation unit 121.

[0127] Here, when mini-batch learning is performed, the loss calculation unit 121 may calculate the weighted loss (or loss) using the ellipse parameters estimated by the learning estimation unit 110 for each of the multiple training iris images that make up the mini-batch and correct answer data including correct answers for the multiple training iris images.

[0128] In this case, the loss calculation unit 121 may calculate the average value of the weighted losses (or losses) calculated for each of the multiple training iris images that make up the mini-batch as the loss for that mini-batch. That is, when the loss is for a mini-batch, it may be, for example, the sum of the weighted losses (or losses) calculated for each of the multiple training iris images that make up the mini-batch divided by the number of training iris images that make up the mini-batch. Alternatively, the loss may be the sum of the weighted losses (or losses) calculated for each of the multiple training iris images that make up the mini-batch itself.

[0129] (Regarding the parameter update unit 122) The parameter update unit 122 calculates a gradient using, for example, the weighted loss calculated by the loss calculation unit 121. The parameter update unit 122 updates each of the model parameters using, for example, the calculated gradient and a predetermined learning rate.

[0130] In more detail, for example, the parameter update unit 122 calculates the gradient for each model parameter by backpropagation. Note that a method other than backpropagation may be used to calculate the gradient.

[0131] The parameter update unit 122 may update each of the model parameters included in the estimation model, for example, by using a value obtained by multiplying the gradient of each model parameter by a learning rate. The learning rate may be a predetermined value or may be changed from an initial value depending on the learning stage.

[0132] The learning rate or its initial value may be determined using an appropriate common method such as an LR range test, which determines the learning rate based on the change in the accuracy rate or loss when the learning rate is gradually increased by a predetermined amount. When changing the learning rate, an appropriate common method such as a CLR (Cyclic Learning Rate), which changes the learning rate for each batch between an upper limit learning rate and a lower limit learning rate, may be used. The parameter update unit 122 may include at least one of a function for determining such a learning rate or its initial value, and a function for changing the learning rate.

[0133] As described above, the estimation model can be learned by executing the estimation process (step S120) performed by the learning estimation unit 110 and the update process (step S120) performed by the update unit 120.

[0134] When multiple training iris images are used, steps S110 to S120 may be repeated for each training iris image. When mini-batch training is used, steps S110 to S120 may be repeated for each mini-batch.

[0135] The condition for terminating the learning of the estimation model (termination condition) may be, for example, repeating steps S110 to S120 a predetermined number of times. The termination condition may be, for example, that the change in the value of the weighted loss (or loss) is equal to or less than a predetermined value. The termination condition may be, for example, that steps S110 to S120 are repeated for all training iris images stored in the training data storage device 90. Note that the termination conditions are not limited to those exemplified here. Furthermore, multiple termination conditions may be combined.

[0136] By training the estimation model, a trained estimation model can be constructed. For example, by training the estimation model, the trained estimation model is stored in the training estimation unit 110. The trained estimation model can estimate the pupil and iris included in the iris image as ellipses. Therefore, the trained estimation model can be used to accurately estimate the pupil and iris included in the iris image.

[0137] The model parameters of the trained estimation model may be stored in the training estimation unit 110 together with the estimation model, or may be stored in a storage unit (not shown) for storing model parameters. This storage unit may be included in the information processing device 100 or in an external device (not shown). In this case, the training estimation unit 110 may estimate the ellipse parameters using the model parameters stored in the storage unit. Furthermore, the update unit 120 may update the model parameters stored in the storage unit.

[0138] (Example of physical configuration of information processing device 100) The information processing device 100 physically includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070, as shown in FIG.

[0139] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0140] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0141] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0142] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing device 100. The processor 1020 reads each of these program modules into the memory 1030 and executes them to realize the function corresponding to the program module.

[0143] The network interface 1050 is an interface for connecting the information processing device 100 to a network. The network is a communication network for transmitting and receiving information to and from other devices (not shown), and may be wired, wireless, or a combination of these.

[0144] The input interface 1060 is an interface for the user to input information, and is composed of, for example, a touch panel, a keyboard, a mouse, and the like.

[0145] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0146] Although the example in which the information processing device 100 is physically configured as one device (e.g., a computer, etc.) has been described, the information processing device 100 may also be configured as a plurality of devices (e.g., computers, etc.) that transmit and receive information to each other via, for example, a communication network NT. In this case, the plurality of devices may cooperate to perform information processing. The training data storage device 90 may also be physically configured in the same way as the information processing device 100. Furthermore, the training data storage device 90 and the information processing device 100 may also be physically configured as a single device (e.g., a computer, etc.).

[0147] (Actions and Effects) As described above, according to this embodiment, the information processing device 100 includes the training estimation unit 110 and the updating unit 120. The training estimation unit 110 estimates ellipse parameters related to the training iris image using an estimation model for estimating ellipse parameters related to the outline or edge of the pupil and iris included in the iris image. The updating unit 120 updates the model parameters included in the estimation model using the estimated ellipse parameters and ground truth data related to the training iris image. The ellipse parameters include shared parameters that are shared for the pupil and iris included in the iris image.

[0148] This makes it possible to construct an estimation model that accurately estimates the shapes of the iris and pupil contained in the iris image as ellipses, thereby enabling accurate estimation of the iris and pupil contained in the iris image.

[0149] According to this embodiment, the shared parameters include at least one of auxiliary parameters and contraction parameters. The auxiliary parameters are parameters that are added when training an estimation model. The contraction parameters are parameters that contract a pupil shape parameter for specifying an ellipse that represents the shape of a pupil included in an iris image and an iris shape parameter for specifying an ellipse that represents the shape of an iris included in the iris image.

[0150] According to this, by using the auxiliary parameters, it is possible to construct an estimation model that estimates the shapes of the iris and pupil contained in the iris image as ellipses with even greater accuracy, thereby making it possible to estimate the iris and pupil contained in the iris image with even greater accuracy.

[0151] Furthermore, by using contracted parameters, it is possible to reduce the number of parameters included in the ellipse parameters while constructing an estimation model that accurately estimates the shapes of the iris and pupil included in the iris image as an ellipse, thereby enabling high-speed and high-accuracy estimation of the iris and pupil included in the iris image.

[0152] According to this embodiment, the contraction parameters include at least one of predetermined parameters included in the pupil shape parameters and the iris shape parameters, and parameters representing components whose dimensions have been reduced by performing statistical analysis processing on the pupil shape parameters and the iris shape parameters.

[0153] This makes it possible to construct an estimation model that accurately estimates the shapes of the iris and pupil included in an iris image as an ellipse while reducing the number of parameters included in the ellipse parameters, thereby enabling the iris and pupil included in an iris image to be estimated quickly and accurately.

[0154] According to this embodiment, the auxiliary parameter has a correlation with at least one parameter included in each of the pupil shape parameters and the iris shape parameters.

[0155] According to this, by using the auxiliary parameters, it is possible to construct an estimation model that estimates the shapes of the iris and pupil contained in the iris image as ellipses with even greater accuracy, thereby making it possible to estimate the iris and pupil contained in the iris image with even greater accuracy.

[0156] According to this embodiment, the auxiliary parameters include at least one of a parameter representing a gaze direction and a parameter representing the positions of one or more key points, the one or more key points including predetermined locations on at least one of an eyelid, a pupil, and an iris.

[0157] According to this, by using the auxiliary parameters, it is possible to construct an estimation model that estimates the shapes of the iris and pupil contained in the iris image as ellipses with even greater accuracy, thereby making it possible to estimate the iris and pupil contained in the iris image with even greater accuracy.

[0158] According to this embodiment, the update unit 120 includes a loss calculation unit 121 and a parameter update unit 122. The loss calculation unit 121 calculates a weighted loss using estimated ellipse parameters, ground truth data, and weights for each corresponding component in the estimated ellipse parameters and the ground truth data. The parameter update unit 122 updates the model parameters included in the estimation model using the calculated weighted loss.

[0159] According to this, by using the weight for each component, it is possible to update the model parameters using an appropriate loss. Therefore, it is possible to construct an estimation model that estimates the shapes of the iris and pupil contained in the iris image with higher accuracy as an ellipse. Therefore, it is possible to estimate the iris and pupil contained in the iris image with higher accuracy.

[0160] [Embodiment 2] In this embodiment, an example will be described in which an information processing device creates correct answer data to be used for training an estimation model when ellipse parameters include one or more contraction parameters. Note that, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0161] 14, the information processing system S2 includes a training data storage device 91 and an information processing device 200. The training data storage device 91 stores training iris images and ellipse-specific parameters (pupil shape parameters and iris shape parameters) related to the training iris images.

[0162] For example, training data is stored in advance in the training data storage device 91. In this embodiment, the training data includes, for example, one or more training iris images and ellipse-specific parameters (pupil shape parameters and iris shape parameters) for each of the one or more training iris images.

[0163] (Configuration Example of Information Processing Device 200) As shown in FIG. 15, for example, the information processing device 200 includes a supervised data generating unit 230, the learning estimation unit 110, and the update unit 120 described above.

[0164] The correct data generating unit 230 calculates correct data from the pupil shape parameters and iris shape parameters related to the training iris image.

[0165] The correct answer data includes one or more contracted parameters, a pupil-specific parameter specific to the pupil, and an iris-specific parameter specific to the iris. The training estimation unit 110 estimates ellipse parameters including one or more contracted parameters, the pupil-specific parameter, and the iris-specific parameter.

[0166] (Example of Processing Operation of Information Processing Device 200) The information processing device 200 executes information processing as shown in FIG.

[0167] The correct data generating unit 230 calculates correct data from the pupil shape parameters and iris shape parameters related to the training iris image (step S230).

[0168] The correct answer data includes one or more contracted parameters, a pupil-specific parameter specific to the pupil, and an iris-specific parameter specific to the iris. The training estimation unit 110 estimates ellipse parameters including one or more contracted parameters, the pupil-specific parameter, and the iris-specific parameter.

[0169] The above-described steps S110 to S120 are executed.

[0170] (Regarding the supervised data generating unit 230) The supervised data generating unit 230, for example, acquires ellipse-specific parameters (including pupil shape parameters and iris shape parameters) from the training data storage device 91. The supervised data generating unit 230 generates supervised data using, for example, the acquired ellipse-specific parameters.

[0171] In detail, for example, in the case of the contracted parameters of the above-mentioned "Contracted Parameter Example 1," the correct answer data generation unit 230 may generate the contracted parameters by deleting parameters corresponding to parameters that are predetermined as contracted parameters.

[0172] More specifically, for example, assume that the ellipse parameters are (C1, C2, P1, P2, P3, Q1, Q2, Q3) as described above. In this case, the correct data generating unit 230 may generate contracted parameters by deleting P4 and P5 (or Q4 and Q5) from the ellipse specifying parameters (P1, P2, P3, P4, P5, Q1, Q2, Q3, Q4, Q5).

[0173] Furthermore, for example, in the case of the contraction parameters of the above-mentioned "Contraction Parameter Example 2," the correct answer data generation unit 230 may generate the contraction parameters by performing principal component analysis. For example, the correct answer data generation unit 230 may generate the contraction parameters by performing principal component analysis using the ellipse-specifying parameters for each of the multiple learning images.

[0174] More specifically, for example, the correct data generation unit 230 may perform principal component analysis on any combination of ellipse-specific parameters (P1, P2, P3, P4, P5, Q1, Q2, Q3, Q4, Q5). Then, the correct data generation unit 230 may determine, as contraction parameters, principal components whose cumulative contribution ratios are equal to or greater than a predetermined threshold. In this case, the principal components may be determined as contraction parameters so that the number of parameters included in the ellipse parameters is minimized by using the principal components as contraction parameters.

[0175] The correct data generating unit 230 may calculate correct data including contracted parameters corresponding to the determined principal components, pupil-specific parameters, and iris-specific parameters. The pupil-specific parameters correspond to, for example, pupil shape parameters that are not expressed by contracted parameters corresponding to the principal components among the ellipse-specific parameters. The iris-specific parameters correspond to, for example, iris shape parameters that are not expressed by contracted parameters corresponding to the principal components among the ellipse-specific parameters.

[0176] For example, when there are multiple training images, the supervised data generating unit 230 may calculate multiple supervised data for each of the multiple training images. Each of the multiple supervised data may be calculated using the ellipse-specifying parameters for each of the multiple training images.

[0177] The training estimation unit 110 may estimate ellipse parameters including one or more contraction parameters, a pupil-specific parameter, and an iris-specific parameter. That is, the estimation model may output ellipse parameters configured from parameters corresponding to the correct answer included in the correct answer data.

[0178] (Actions and Effects) As described above, according to this embodiment, the information processing device 200 further includes a correct data generation unit 230 that calculates correct data from pupil shape parameters and iris shape parameters related to the training iris image. The correct data includes correct answers for one or more contraction parameters, correct answers for pupil-specific parameters specific to the pupil, and correct answers for iris-specific parameters specific to the iris. The estimation model estimates ellipse parameters including one or more contraction parameters, pupil-specific parameters, and iris-specific parameters.

[0179] [Embodiment 3] In embodiment 3, an example will be described in which ellipse parameters are corrected using ellipse-specific parameters. The ellipse-specific parameters used for correction may be stored in advance, as described in embodiment 2, or may be estimated from a training iris image. In this embodiment, an example will be described in which an information processing device estimates ellipse-specific parameters (pupil shape parameters and iris shape parameters) from a training iris image. Note that descriptions that overlap with other embodiments will be omitted as appropriate for the sake of brevity.

[0180] The information processing system S1 may include an information processing device 300 instead of the information processing device 100 .

[0181] (Configuration Example of Information Processing Device 300) As shown in FIG. 17, for example, the information processing device 300 includes a learning estimation unit 310 and the above-described update unit 120.

[0182] The training estimation unit 310 estimates and corrects ellipse parameters for the training iris images.

[0183] The learning estimation unit 310 includes, for example, an ellipse parameter estimation unit 311, an ellipse specifying parameter estimation unit 312, and a correction unit 313, as shown in FIG.

[0184] The ellipse parameter estimation unit 311 uses the estimation model to estimate ellipse parameters for the training iris image.

[0185] The ellipse-specific parameter estimation unit 312 estimates ellipse-specific parameters for the training iris image using an ellipse-specific parameter estimation model for estimating ellipse-specific parameters included in the iris image. The ellipse-specific parameters are pupil shape parameters and iris shape parameters.

[0186] The correction unit 313 corrects the estimated ellipse parameters using the estimated ellipse-specifying parameters.

[0187] (Example of Processing Operation of Information Processing Device 300) The information processing device 300 executes information processing as shown in FIG. 19, for example.

[0188] The training estimation unit 310 estimates and corrects ellipse parameters for the training iris image (step S310).

[0189] The learning estimation unit 310 executes an estimation process (step S310) as shown in FIG. 20, for example.

[0190] The ellipse parameter estimation unit 311 uses the estimation model to estimate ellipse parameters for the training iris image (step S311).

[0191] The ellipse-specific parameter estimation unit 312 estimates ellipse-specific parameters for the training iris image using an ellipse-specific parameter estimation model for estimating ellipse-specific parameters included in the iris image (step S312). The ellipse-specific parameters are pupil shape parameters and iris shape parameters.

[0192] The correction unit 313 corrects the estimated ellipse parameters using the estimated ellipse-specifying parameters (step S313).

[0193] The above-described step S120 is executed.

[0194] (Ellipse Parameter Estimation Unit 311) The ellipse parameter estimation unit 311 estimates ellipse parameters for the training iris image using an estimation model. The ellipse parameter estimation unit 311 may be the same as the training estimation unit 110 described above.

[0195] (Ellipse-specific parameter estimation unit 312) The ellipse-specific parameter estimation unit 312 estimates ellipse-specific parameters for the training iris image using an ellipse-specific parameter estimation model.

[0196] The ellipse-specific parameter estimation model is, for example, a machine learning model for estimating ellipse-specific parameters included in an iris image. The ellipse-specific parameter estimation model includes one or more model parameters and is configured using, for example, a neural network. The estimation model and the ellipse-specific parameter estimation model may be configured using a series of neural networks or may be configured using individual neural networks.

[0197] For example, the ellipse-specific parameter estimation unit 312 may input a training iris image that has been subjected to image processing including cutting out an iris region from the training iris image into an ellipse-specific parameter estimation model, and estimate an ellipse-specific parameter for the training iris image.

[0198] More specifically, as shown in FIG. 21, the ellipse specific parameter estimating unit 312 includes an image processing unit 312a and a processed image parameter estimating unit 312b.

[0199] The image processing unit 312a performs image processing, including cutting out an iris region including the pupil and iris from the training iris image using the estimated ellipse parameters. This image processing may include, in addition to cutting out, downsampling to an image of a predetermined size. Furthermore, the cut-out iris image may be normalized to a predetermined size using the size of the iris ellipse. Normalization may involve, for example, converting the iris region included in the training iris image into an image of a predetermined shape, such as a rectangle.

[0200] The processed image parameter estimation unit 312b inputs the training iris image that has been image processed by the image processing unit 312a into an ellipse specifying parameter estimation model, and estimates ellipse specifying parameters for the training iris image.

[0201] (Regarding Correction Unit 313) The correction unit 313 corrects the estimated ellipse parameters using the estimated ellipse-specifying parameters.

[0202] For example, the correction unit 313 corrects a contraction parameter included in the estimated ellipse parameters with a parameter included in the estimated ellipse-specifying parameters that corresponds to the contraction parameter. This correction may be, but is not limited to, replacement, correction of a deviation amount, or the like.

[0203] In more detail, for example, the contraction parameters include an ellipticity and an ellipse rotation angle. The correction unit 313 may, for example, use the ellipticities of the pupil and the iris included in the ellipse-specifying parameters to correct the ellipticity shared by the pupil and the iris as a contraction parameter in the estimated ellipse parameters. Furthermore, for example, the correction unit 313 may use the ellipse rotation angles of the pupil and the iris included in the ellipse-specifying parameters to correct the ellipse rotation angles shared by the pupil and the iris as a contraction parameter in the estimated ellipse parameters.

[0204] (Operations and Effects) As described above, according to this embodiment, the learning estimation unit 310 includes the ellipse parameter estimation unit 311 , the ellipse specifying parameter estimation unit 312 , and the correction unit 313 .

[0205] An ellipse parameter estimation unit 311 estimates ellipse parameters for the training iris image using an estimation model. An ellipse-specific parameter estimation unit 312 estimates pupil shape parameters and iris shape parameters for the training iris image using an ellipse-specific parameter estimation model for estimating pupil shape parameters and iris shape parameters included in the iris image. A correction unit 313 corrects the estimated ellipse parameters using the estimated pupil shape parameters and iris shape parameters.

[0206] This corrects the estimated ellipse parameters, thereby improving the accuracy of estimating the ellipse parameters, and therefore enabling the iris and pupil contained in the iris image to be estimated with even greater accuracy.

[0207] According to this embodiment, the ellipse specific parameter estimating unit 312 includes an image processing unit 312a and a processed image parameter estimating unit 312b.

[0208] The image processing unit 312a performs image processing, including cutting out an iris region including the pupil and iris from the training iris image, using the estimated ellipse parameters. The processed image parameter estimation unit 312b inputs the training iris image that has undergone image processing into an ellipse-specific parameter estimation model, and estimates ellipse-specific parameters for the training iris image.

[0209] This reduces the data size of the image input to the ellipse-specific parameter estimation model, allowing for high-speed estimation of the ellipse-specific parameters. The estimated ellipse parameters are then corrected using the ellipse-specific parameters, improving the accuracy of the ellipse parameter estimation. This makes it possible to quickly construct an estimation model that accurately estimates ellipse parameters related to the contours or edges of the iris and pupil contained in an iris image.

[0210] Fourth Embodiment In a fourth embodiment, a detailed example of the loss calculation unit 121 (see, for example, FIG. 11 ) will be described. Note that, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0211] (Example of Configuration of Loss Calculation Unit 121) As shown in FIG. 22, the loss calculation unit 121 includes a weight determination unit 121a and a weighted loss calculation unit 121b.

[0212] The weight determination unit 121a determines a weight for a second component included in the supervised data based on the magnitude of a parameter of the first component included in the supervised data. The second component is a component different from the first component.

[0213] The weighted loss calculation unit 121b calculates the weighted loss using the ellipse parameters estimated by the weight determination unit 121a, the correct answer data, and the weights determined by the weight determination unit 121a.

[0214] (Example of Processing Operation of Loss Calculation Unit 121) The loss calculation unit 121 executes a loss calculation process (step S121) as shown in FIG. 23, for example.

[0215] The weight determination unit 121a determines a weight for the second component included in the supervised data based on the magnitude of the parameter of the first component included in the supervised data (step S121a). The second component is a component different from the first component.

[0216] The weighted loss calculation unit 121b calculates a weighted loss using the ellipse parameters estimated by the weight determination unit 121a, the ground truth data, and the weights determined by the weight determination unit 121a (step S121b).

[0217] Regarding the Weight Determining Unit 121a: For example, the weight determining unit 121a may determine a larger weight for the second component as the first component becomes larger. The first component and the second component may be components that have a correlation.

[0218] In detail, for example, the first component may be the ellipticity. The second component may be the elliptic rotation angle. The smaller the ellipticity, the closer the ellipse is to a circle. Therefore, in general, for an elliptical shape, the greater the ellipticity, the greater the impact of the elliptic rotation angle on the estimation accuracy of the elliptic shape. In this case, the weight determination unit 121a may use, for example, a predetermined formula or the like to determine a weight for the elliptic rotation angle that is greater the ellipticity included in the correct answer data. The predetermined formula may be, for example, a formula representing a monotonically increasing, differentiable function. The weight for the elliptic rotation angle may be determined by substituting the ellipticity into this formula.

[0219] (Weighted Loss Calculation Unit 121b) The weighted loss calculation unit 121b may calculate the weighted loss using, for example, the ellipse parameters estimated for the training iris image, the ground truth data, and weights for each component, as described above. The weights for each component may include the weights determined by the weight determination unit 121a.

[0220] (Actions and Effects) As described above, according to this embodiment, the loss calculation unit 121 includes a weight determination unit 121a and a weighted loss calculation unit 121b. The weight determination unit 121a determines a weight for the second component included in the supervised data based on the magnitude of the parameter of the first component included in the supervised data. The weighted loss calculation unit 121b calculates a weighted loss using the ellipse parameters estimated by the weight determination unit 121a, the supervised data, and the weight determined by the weight determination unit 121a. The second component is a component different from the first component.

[0221] This allows the model parameters to be updated using an appropriate loss, making it possible to construct an estimation model that more accurately estimates the shapes of the iris and pupil contained in the iris image as ellipses.As a result, it becomes possible to more accurately estimate the iris and pupil contained in the iris image.

[0222] Fifth Embodiment In a fifth embodiment, an example in which the line of sight direction is used as a common parameter will be described. Note that, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0223] The information processing system S1 may include an information processing device 500 instead of the information processing device 100 .

[0224] The information processing device 500 includes, for example, a learning estimation unit 510 and an update unit 520 as shown in FIG.

[0225] The training estimation unit 510 estimates ellipse parameters including the gaze direction for the training iris image using an estimation model for estimating the gaze direction and ellipse parameters of the eye included in the iris image.

[0226] The update unit 520 updates the model parameters included in the estimation model using the ellipse parameters including the estimated gaze direction and the ground truth data related to the training iris image.

[0227] The correct answer data includes correct answers for ellipse parameters including gaze direction for the training iris image.

[0228] (Example of Processing Operation of Information Processing Device 500) The information processing device 500 executes information processing as shown in FIG.

[0229] The training estimation unit 510 uses the estimation model to estimate ellipse parameters including the gaze direction for the training iris image (step S510).

[0230] The update unit 520 updates the model parameters included in the estimation model using the ellipse parameters including the estimated gaze direction and the ground truth data related to the training iris image (step S520).

[0231] The correct answer data includes correct answers for ellipse parameters including gaze direction for the training iris image.

[0232] (Regarding the training estimation unit 510) The training estimation unit 510, for example, inputs a training iris image into an estimation model and estimates ellipse parameters related to the training iris image. The ellipse parameters include the gaze direction as a shared parameter. The training estimation unit 510 outputs ellipse parameters including the gaze direction as a shared parameter. The gaze direction may be included in the ellipse parameters as an auxiliary parameter, or may be included in the ellipse parameters as a contraction parameter. In other words, the auxiliary parameter may include a parameter representing the gaze direction. The contraction parameter may include a parameter representing the gaze direction.

[0233] (Regarding the updating unit 520) The updating unit 520 updates the model parameters included in the estimation model using, for example, ellipse parameters including an estimated gaze direction and correct answer data. The correct answer data includes, for example, correct ellipse parameters related to a training iris image. The correct ellipse parameters include the gaze direction as a shared parameter. For example, if the ellipse parameters estimated by the estimation model include the gaze direction as an auxiliary parameter, the correct ellipse parameters include the correct gaze direction as an auxiliary parameter. For example, if the ellipse parameters estimated by the estimation model include the gaze direction as a contraction parameter, the correct ellipse parameters include the correct gaze direction as a contraction parameter.

[0234] The updating unit 520 may calculate a weighted loss using, for example, ellipse parameters including the gaze direction estimated by the training estimation unit 510, ground truth data related to the training iris image, and weights for each corresponding component in the estimated ellipse parameters and the ground truth data. The updating unit 520 may update model parameters included in the estimation model using the calculated weighted loss. Such an updating unit 520 may include, for example, a loss calculation unit that calculates the weighted loss and a parameter update unit that updates the model parameters.

[0235] The update unit 520 may calculate the loss using ellipse parameters including the gaze direction estimated by the training estimation unit 510 and ground truth data related to the training iris image. The update unit 520 may update the model parameters included in the estimation model using the calculated loss.

[0236] As described above, the estimation process (step S510) performed by the learning estimation unit 510 and the update process (step S520) performed by the update unit 520 are performed, thereby enabling learning of an estimation model that estimates the gaze direction and ellipse parameters.

[0237] If multiple training iris images are used, steps S510 to S520 may be repeated for each training iris image. If mini-batch training is used, steps S510 to S520 may be repeated for each mini-batch.

[0238] The condition for terminating the learning of the estimation model (termination condition) may be, for example, repeating steps S510 to S520 a predetermined number of times. The termination condition may be, for example, that the change in the value of the weighted loss (or loss) is equal to or less than a predetermined value. The termination condition may be, for example, that steps S510 to S520 are repeated for all training iris images stored in the training data storage device 90. Note that the termination conditions are not limited to those exemplified here. Furthermore, multiple termination conditions may be combined.

[0239] This allows the estimation model to be trained and a trained estimation model to be constructed. The gaze direction often affects the ellipse parameters such as the ellipticity and the ellipse rotation angle θ.

[0240] Therefore, by using the gaze direction estimation result as a contraction parameter, it is possible to construct an estimation model that accurately estimates the shapes of the iris and pupil included in the iris image as an ellipse while reducing the number of parameters included in the ellipse parameters. Therefore, it is possible to quickly and accurately estimate the iris and pupil included in the iris image.

[0241] Furthermore, by using the gaze direction as an auxiliary parameter, it is possible to construct an estimation model that can estimate the iris contained in the iris image with even greater accuracy.Using the trained estimation model, it is possible to estimate the iris contained in the iris image with even greater accuracy.

[0242] (Operations and Effects) As described above, according to this embodiment, the information processing device 500 includes the learning estimation unit 510 and the update unit 520 .

[0243] The learning estimation unit 510 uses the estimation model to estimate ellipse parameters including the gaze direction of the eye included in the iris image.

[0244] The update unit 520 updates the model parameters included in the estimation model using the estimated ellipse parameters and the correct answer data. The correct answer data includes correct answers for the ellipse parameters, including the gaze direction, for the training iris image.

[0245] This makes it possible to construct an estimation model for estimating ellipse parameters that include the gaze direction as a common parameter.

[0246] By using the gaze direction as an auxiliary parameter, it is possible to construct an estimation model that can estimate the iris contained in the iris image with even greater accuracy.Using the trained estimation model, it is possible to estimate the iris contained in the iris image with even greater accuracy.

[0247] Furthermore, by using the gaze direction as a contraction parameter, it is possible to construct an estimation model that accurately estimates the shapes of the iris and pupil included in the iris image as an ellipse while reducing the number of parameters included in the ellipse parameters, thereby enabling the iris and pupil included in the iris image to be estimated quickly and accurately.

[0248] Sixth Embodiment In a sixth embodiment, an example will be described in which the positions of key points are used as the shared parameters. As described above, the key points are one or more predetermined locations related to the eyes. Note that, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0249] In this embodiment, the shared parameters include parameters representing the positions of one or more key points. Similar to the above-mentioned gaze direction, the positions of the one or more key points may be included in the ellipse parameters as auxiliary parameters, or may be included in the ellipse parameters as contraction parameters. In other words, the auxiliary parameters may include parameters representing the positions of one or more key points. The contraction parameters may include parameters representing the positions of one or more key points.

[0250] The information processing system S1 may include an information processing device 600 instead of the information processing device 100 .

[0251] The information processing device 600 includes a learning estimation unit 610 and an update unit 620, as shown in FIG. 26, for example.

[0252] The training estimator 610 uses the estimation model to estimate ellipse parameters, including keypoint locations, for the training iris images.

[0253] The update unit 620 updates the model parameters included in the estimation model using the ellipse parameters including the positions of the estimated key points and the ground truth data related to the training iris image.

[0254] The ground truth data includes ground truths for ellipse parameters, including keypoint locations, for training iris images.

[0255] (Example of Processing Operation of Information Processing Device 600) The information processing device 600 executes information processing as shown in FIG.

[0256] The training estimation unit 610 uses the estimation model to estimate ellipse parameters including the positions of key points for the training iris image (step S610).

[0257] The update unit 620 updates the model parameters included in the estimation model using the ellipse parameters including the positions of the estimated key points and the ground truth data related to the training iris image (step S520).

[0258] The ground truth data includes ground truths for ellipse parameters, including keypoint positions, for the training iris image.

[0259] (Regarding the training estimation unit 610) The training estimation unit 610, for example, inputs a training iris image into an estimation model and estimates ellipse parameters related to the training iris image. The ellipse parameters include the positions of one or more keypoints as shared parameters. The training estimation unit 610 outputs ellipse parameters including the positions of one or more keypoints as shared parameters. The positions of one or more keypoints may be included in the ellipse parameters as auxiliary parameters, or may be included in the ellipse parameters as contraction parameters. In other words, the auxiliary parameters may include parameters representing the positions of one or more keypoints. The contraction parameters may include parameters representing the positions of one or more keypoints.

[0260] (Regarding the updating unit 620) The updating unit 620 updates the model parameters included in the estimation model using, for example, ellipse parameters including the estimated positions of one or more keypoints and correct answer data. The correct answer data includes, for example, correct answers for ellipse parameters including the positions of one or more keypoints as shared parameters for a training iris image. The correct answers for the ellipse parameters include the positions of one or more keypoints as shared parameters. For example, if the ellipse parameters estimated by the estimation model include the positions of one or more keypoints as auxiliary parameters, the correct answers for the positions of one or more keypoints include the correct answers for the positions of one or more keypoints as auxiliary parameters. For example, if the ellipse parameters estimated by the estimation model include the positions of one or more keypoints as contraction parameters, the correct answers for the ellipse parameters include the correct answers for the positions of one or more keypoints as contraction parameters.

[0261] The updating unit 620 may calculate a weighted loss using, for example, ellipse parameters including the positions of one or more keypoints estimated by the training estimation unit 610, ground truth data related to the training iris image, and weights for each corresponding component between the estimated ellipse parameters and the ground truth data. The updating unit 620 may update the model parameters included in the estimation model using the calculated weighted loss.

[0262] 28, the update unit 620 may include a loss calculation unit 621 and the above-described parameter update unit 122. The loss calculation unit 621 may include, for example, a weight determination unit 621a and the above-described weighted loss calculation unit 121b.

[0263] The weight determination unit 621a determines weights for one or more predetermined key points based on whether or not the one or more key points are hidden by the eyelid.

[0264] The pupil and part of the iris may be hidden by the eyelid. FIG. 29 shows an example of an iris image in which part of the iris is hidden by the eyelid. The dotted line in the figure indicates the outline of the eyelid. It may be difficult to estimate the position of a keypoint hidden by the eyelid. Therefore, the weight determination unit 621a may determine the weight for a keypoint based on whether the keypoint is hidden by the eyelid.

[0265] For example, the weight determination unit 621a may determine the weight of a key point hidden by an eyelid to be a first predetermined value. The weight determination unit 621a may determine the weight of a key point not hidden by an eyelid to be a second predetermined value. The first predetermined value is, for example, 0. The second predetermined value is, for example, 1. Note that the first and second predetermined values ​​are not limited to 0 and 1 as mentioned here, and may be determined in advance as appropriate.

[0266] As described above, the weighted loss calculation unit 121b calculates the weighted loss using the estimated ellipse parameters, the ground truth data, and the determined weights. By using the weights determined depending on whether or not a keypoint is hidden by an eyelid, an estimation model that estimates ellipse parameters with high accuracy can be constructed. For example, by setting the weight of a keypoint hidden by an eyelid to 0, learning can be performed using only keypoints that are not hidden by an eyelid and whose positions are easy to estimate, and an estimation model that estimates ellipse parameters with even higher accuracy can be constructed.

[0267] Note that the updating unit 620 may update the model parameters included in the estimation model by performing a process in which the gaze direction in the updating unit 520 described above is replaced with one or more key points. That is, for example, the updating unit 620 may calculate a weighted loss using ellipse parameters including one or more key points estimated by the training estimation unit 510, ground truth data related to the training iris image, and weights for each corresponding component in the estimated ellipse parameters and the ground truth data. The updating unit 520 may update the model parameters included in the estimation model using the calculated weighted loss.

[0268] (Operations and Effects) As described above, according to this embodiment, the information processing device 600 includes the learning estimation unit 610 and the update unit 620 .

[0269] The training estimator 610 uses the estimation model to estimate ellipse parameters including keypoints for the training iris image.

[0270] The update unit 620 updates the model parameters included in the estimation model using the estimated ellipse parameters and ground truth data related to the training iris image.

[0271] The ground truth data includes ground truths for ellipse parameters, including keypoint positions, for the training iris image.

[0272] This allows us to construct an estimation model that estimates ellipse parameters, including the keypoint positions as common parameters.

[0273] By using ellipse parameters with the positions of key points as auxiliary parameters, it is possible to construct an estimation model that can estimate the iris contained in an iris image with even greater accuracy. Therefore, it is possible to estimate the iris contained in an iris image with even greater accuracy using a trained estimation model.

[0274] Furthermore, by using the keypoints as contraction parameters, it is possible to reduce the number of parameters (dimensions) included in the ellipse parameters while constructing an estimation model that accurately estimates the shapes of the iris and pupil included in the iris image as an ellipse, thereby enabling high-speed and high-accuracy estimation of the iris and pupil included in the iris image.

[0275] According to this embodiment, the auxiliary parameters include parameters representing the positions of one or more keypoints. The update unit 620 includes a loss calculation unit 621 and a parameter update unit 622. The loss calculation unit 621 calculates a weighted loss using the estimated ellipse parameters, ground truth data, and weights for each corresponding component in the estimated ellipse parameters and the ground truth data. The parameter update unit 622 updates the model parameters included in the estimation model using the calculated weighted loss.

[0276] The loss calculation unit 621 determines weights for one or more predetermined key points based on whether the one or more key points are hidden by the eyelid. The loss calculation unit 621 calculates a weighted loss using the estimated ellipse parameters, the ground truth data, and the determined weights.

[0277] This allows us to build an estimation model that accurately estimates ellipse parameters by using weights determined depending on whether the iris is hidden by the eyelid. Therefore, it is possible to estimate the iris contained in the iris image with even greater accuracy using the trained estimation model.

[0278] [Embodiment 7] In embodiment 7, an example of performing iris authentication using ellipse parameters will be described as an example of using a trained estimation model. Note that, for the sake of brevity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0279] (Configuration Example of Information Processing System S7) The information processing system S7 includes, for example, a photographing device 95, the training data storage device 90 described above, and an information processing device 700, as shown in FIG.

[0280] The photographing device 95 photographs an object and generates an iris image. The photographing device 95 is, for example, a camera. The photographing device 95 photographs an object to be authenticated and generates an iris image including the iris of the object to be authenticated. The photographing device 95 may also be used to obtain a learning iris image.

[0281] (Configuration example of information processing device 700) As shown in Fig. 31 , the information processing device 700 includes an estimation unit 740 and an authentication unit 750. Note that the information processing device 700 may further include the above-mentioned learning estimation unit 110 and update unit 120.

[0282] The estimation unit 740 uses the trained estimation model to estimate ellipse parameters for an iris image including the iris of the authentication target.

[0283] The authentication unit 750 authenticates the authentication target using the ellipse parameters estimated for the authentication target.

[0284] (Example of Processing Operation of Information Processing Device 700) The information processing device 700 executes information processing as shown in FIG.

[0285] The estimation unit 740 uses the trained estimation model to estimate ellipse parameters for the iris image including the iris of the authentication target (step S740).

[0286] The authentication unit 750 authenticates the authentication target using the ellipse parameters estimated for the authentication target (step S750).

[0287] (Regarding the Estimation Unit 740) The estimation unit 740 includes, for example, a trained estimation model. The estimation unit 740 acquires an authentication iris image captured in response to, for example, a user instruction, from the imaging device 95. The authentication iris image is an iris image that includes the iris of the authentication target.

[0288] For example, the estimation unit 740 inputs the acquired authentication iris image into the trained estimation model. In response to this input, the trained estimation model outputs ellipse parameters related to the input authentication iris image. As a result, the estimation unit 530 estimates ellipse parameters related to the authentication iris image.

[0289] When the ellipse parameters include auxiliary parameters, as described above, the ellipse parameters output by the estimation model during training include the auxiliary parameters, but the ellipse parameters output by the trained estimation model do not need to include the auxiliary parameters. In other words, when the ellipse parameters include auxiliary parameters, the estimation unit 740 may estimate ellipse parameters including the auxiliary parameters for the authentication iris image using the trained estimation model, and output the ellipse parameters excluding the auxiliary parameters.

[0290] In detail, for example, when the gaze direction is included as an auxiliary parameter in the ellipse parameters, the estimation unit 740 may estimate ellipse parameters including the gaze direction using a trained estimation model, and output ellipse parameters excluding the gaze direction.Furthermore, for example, when the positions of one or more key points are included as auxiliary parameters in the ellipse parameters, the estimation unit 740 may estimate ellipse parameters including the positions of one or more key points using a trained estimation model, and output ellipse parameters excluding the positions of the one or more key points.

[0291] The function of the estimation unit 740 corresponds to, for example, the function of the learning estimation units 110, 310, 510, and 610 after learning an estimation model to use an authentication iris image as input to the estimation model instead of training data (learning iris image). The estimation unit 740 may acquire the authentication iris image from a storage unit (not shown) that stores the authentication iris image instead of or together with the imaging device 95.

[0292] Furthermore, the model parameters of the trained estimation model may be stored in the training estimation unit 110, 310, 410, 510, or 610 together with the estimation model, or may be stored in a storage unit (not shown) for storing model parameters. This storage unit may be included in the information processing device 100, 200, 300, 400, 500, 600, or 700, or may be included in an external device (not shown). In this case, the training estimation unit 110, 310, 410, 510, or 610 may estimate ellipse parameters using the model parameters stored in the storage unit. Furthermore, the update unit 120, 220, 320, or 420 may update the model parameters stored in the storage unit. The estimation unit 740 may estimate ellipse parameters, etc., by using the estimation model with reference to the model parameters stored in the storage unit after training.

[0293] (Regarding the Authentication Unit 750) The authentication unit 750, for example, acquires an iris image for authentication and the ellipse parameters estimated by the estimation unit 740. The authentication unit 750, for example, uses the ellipse parameters estimated by the estimation unit 740 to cut out an iris portion from the iris image for authentication. The authentication unit 750, for example, uses the cut-out iris portion to extract iris feature amounts. The iris feature amounts may be, for example, a feature vector. The authentication unit 750, for example, acquires the feature vector by inputting the cut-out iris portion into an extraction model that has been trained to extract the feature vector from the image of the iris portion.

[0294] The authentication unit 750, for example, compares the acquired feature vector with a pre-registered feature vector and performs authentication based on the comparison result. In detail, for example, the authentication unit 750 may determine that authentication is successful when the similarity (e.g., cosine similarity) between the acquired feature vector and the pre-registered feature vector is equal to or greater than a threshold. The authentication unit 750 may determine that authentication is unsuccessful when the similarity is less than the threshold.

[0295] Note that authentication may be performed using any common technique and is not limited to the method described here. Furthermore, the use of the trained estimation model is not limited to iris authentication.

[0296] (Operations and Effects) As described above, according to this embodiment, the information processing device 700 includes the estimation unit 740 that uses a trained estimation model to estimate ellipse parameters related to an iris image that includes the iris of the authentication target.

[0297] This allows iris authentication and the like to be performed using highly accurate ellipse parameters, thereby enabling highly accurate iris authentication and the like.

[0298] According to this embodiment, the ellipse parameters include auxiliary parameters, and the trained estimation model outputs the ellipse parameters excluding the auxiliary parameters for the authentication iris image.

[0299] This allows iris authentication and the like to be performed using accurate ellipse parameters output by an estimation model constructed by learning using auxiliary parameters, thereby enabling iris authentication and the like to be performed with even greater accuracy.

[0300] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0301] In addition, although the flowcharts used in the above description show a sequence of steps (processes), the order of steps executed in each embodiment is not limited to the sequence shown in the flowcharts. In each embodiment, the order of steps shown in the diagrams can be changed as long as it does not cause any problems in terms of the content.

[0302] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes: 1. An information processing device comprising: a training estimation means that estimates ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters related to the outline or edge of a pupil and iris included in an iris image; and an update means that updates model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image. 2. The information processing device described in 1., wherein the shared parameters include at least one of auxiliary parameters that are added when training the estimation model, and contraction parameters that contract a pupil shape parameter for specifying an ellipse representing the shape of a pupil included in the iris image and an iris shape parameter for specifying an ellipse representing the shape of an iris included in the iris image. 3. The information processing device according to 2., further comprising correct answer calculation means for calculating the correct ellipse parameters from the pupil shape parameters and the iris shape parameters related to the training iris image, wherein the correct ellipse parameters include one or more of the contracted parameters, a pupil-specific parameter specific to the pupil, and an iris-specific parameter specific to the iris, and the training estimation means estimates the ellipse parameters including the one or more contracted parameters, the pupil-specific parameter, and the iris-specific parameter. 4. The information processing device according to 2. or 3, wherein the contracted parameters include at least one of a predetermined parameter included in the pupil shape parameters and the iris shape parameters, and a parameter representing a component obtained by reducing the dimension by performing statistical analysis processing on the pupil shape parameters and the iris shape parameters.5. The information processing device described in 4., wherein the training estimation means includes: ellipse parameter estimation means that estimates the ellipse parameters for the training iris image using the estimation model; ellipse-specific parameter estimation means that estimates the pupil shape parameters and the iris shape parameters for the training iris image using an ellipse-specific parameter estimation model for estimating the pupil shape parameters and the iris shape parameters included in the iris image; and correction means that corrects the estimated ellipse parameters using the estimated pupil shape parameters and iris shape parameters. 6. The information processing device described in 5., wherein the ellipse-specific parameter estimation means includes: image processing means that performs image processing including cutting out an iris region including the pupil and iris from the training iris image using the estimated ellipse parameters; and processed image parameter estimation means that inputs the image-processed training iris image to the ellipse-specific parameter estimation model and estimates the ellipse-specific parameters for the training iris image. 7. the updating means includes: loss calculation means for calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct ellipse parameters, and weights for each corresponding component in the estimated ellipse parameters and the correct answer data; parameter updating means for updating a model parameter included in the estimation model using the calculated weighted loss; the loss calculation means includes: weight determination means for determining the weight for a second component included in the correct answer data based on a magnitude of a parameter of a first component included in the correct answer data; and weighted loss calculation means for calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weight, the second component being a component different from the first component. 8. The information processing device according to any one of 2. to 7., wherein the auxiliary parameters include at least one of a parameter representing a gaze direction and a parameter representing a position of one or more key points, and the one or more key points include predetermined locations with respect to at least one of an eyelid, a pupil, and an iris.9. The information processing device according to 8., wherein the auxiliary parameters include parameters representing positions of the one or more keypoints, and the updating means includes: loss calculation means for calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct ellipse parameters, and weights for each component corresponding to the estimated ellipse parameters and the correct answer data, and parameter updating means for updating model parameters included in the estimation model using the calculated weighted loss, and the loss calculation means includes: weight determination means for determining the weights for the one or more predetermined keypoints based on whether the one or more keypoints are hidden by the eyelid, and weighted loss calculation means for calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weights. 10. The information processing device according to any one of 2. to 9., further comprising estimation means for estimating the ellipse parameters for an authentication iris image including the iris of an authentication target using the trained estimation model. 11. The information processing device according to 10., wherein the ellipse parameters include the auxiliary parameters, and the trained estimation model outputs the ellipse parameters excluding the auxiliary parameters for the authentication iris image. 12. An information processing method, including one or more computers estimating the ellipse parameters for a training iris image using an estimation model for estimating ellipse parameters for the contour or edge of a pupil and iris included in an iris image, and updating model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, and the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image.13. The information processing method according to 12., wherein the shared parameters include at least one of: auxiliary parameters added when training the estimation model; and contraction parameters that contract a pupil shape parameter for specifying an ellipse representing the shape of a pupil included in the iris image and an iris shape parameter for specifying an ellipse representing the shape of an iris included in the iris image. 14. The information processing method according to 13., further comprising: calculating the correct ellipse parameters from the pupil shape parameters and the iris shape parameters related to the training iris image, the correct ellipse parameters including one or more of the contraction parameters, a pupil-specific parameter specific to the pupil, and an iris-specific parameter specific to the iris, and estimating the ellipse parameters includes estimating the ellipse parameters including the one or more contraction parameters, the pupil-specific parameter, and the iris-specific parameter. 15. The information processing method according to 13. or 14., wherein the contracted parameters include at least one of predetermined parameters included in the pupil shape parameters and the iris shape parameters, and parameters representing components obtained by reducing the dimensions of the pupil shape parameters and the iris shape parameters by performing statistical analysis processing on the pupil shape parameters and the iris shape parameters. 16. The information processing method according to 15., wherein the estimating the ellipse parameters includes: estimating the ellipse parameters for the training iris image using the estimation model; estimating the pupil shape parameters and the iris shape parameters for the training iris image using an ellipse-specific parameter estimation model for estimating the pupil shape parameters and the iris shape parameters included in the iris image; and correcting the estimated ellipse parameters using the estimated pupil shape parameters and iris shape parameters. 17. 16. The information processing method according to item 16, wherein estimating the pupil shape parameters and the iris shape parameters comprises: performing image processing including cutting out an iris region including the pupil and the iris from the training iris image using the estimated ellipse parameters; and inputting the training iris image that has undergone the image processing into the ellipse specifying parameter estimation model to estimate the ellipse specifying parameters related to the training iris image.18. The information processing method according to any one of 13. to 17., wherein updating the model parameters includes: calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct answer ellipse parameters, and weights for each corresponding component between the estimated ellipse parameters and the correct answer data; updating the model parameters included in the estimation model using the calculated weighted loss; and calculating the weighted loss includes determining the weight for a second component included in the correct answer data based on the magnitude of a parameter of a first component included in the correct answer data; and calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weight, wherein the second component is a component different from the first component. 19. The information processing method according to any one of 13. to 18., wherein the auxiliary parameters include at least one of a parameter representing a gaze direction and a parameter representing the position of one or more key points, and the one or more key points include predetermined locations on at least one of an eyelid, a pupil, and an iris. 20. The information processing method according to 19., wherein the auxiliary parameters include parameters that represent positions of the one or more keypoints, and updating the model parameters includes calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct ellipse parameters, and weights for each component corresponding to the estimated ellipse parameters and the correct answer data, updating the model parameters included in the estimation model using the calculated weighted loss, and calculating the weighted loss includes determining the weights for the one or more predetermined keypoints based on whether the one or more keypoints are hidden by the eyelid, and calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weights. 21. The information processing method according to any one of 13. to 20., further comprising estimating the ellipse parameters for an authentication iris image that includes an iris of an authentication target using the trained estimation model.22. The information processing method according to 21., wherein the ellipse parameters include the auxiliary parameters, and the trained estimation model estimates the ellipse parameters excluding the auxiliary parameters for the authentication iris image. 23. A program causing one or more computers to execute the information processing method according to any one of 12. to 22. 24. A recording medium having recorded thereon a program causing one or more computers to execute the information processing method according to any one of 12. to 22.

[0303] 90, 91 Training data storage device 95 Imaging device 100, 200, 300, 500, 600, 700 Information processing device 110, 310, 510, 610 Learning estimation unit 120, 520, 620 Update unit 121, 621 Loss calculation unit 121a, 621a Weight determination unit 121b Weighted loss calculation unit 122, 622 Parameter update unit 123 Parameter update unit 230 Correction data generation unit 311 Ellipse parameter estimation unit 312 Ellipse specific parameter estimation unit 312a Image processing unit 312b Processed image parameter estimation unit 313 Correction unit 740 Estimation unit 750 Authentication unit

Claims

1. An information processing device comprising: a training estimation means for estimating ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters for the outline or edge of the pupil and iris included in the iris image; and an update means for updating model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and the iris included in the iris image.

2. The information processing device according to claim 1, wherein the shared parameters include at least one of: auxiliary parameters added when training the estimation model; and contraction parameters that contract a pupil shape parameter for specifying an ellipse representing the shape of a pupil included in the iris image; and an iris shape parameter for specifying an ellipse representing the shape of an iris included in the iris image.

3. An information processing device as described in claim 2, further comprising a correct answer calculation means for calculating the correct ellipse parameters from the pupil shape parameters and the iris shape parameters related to the training iris image, wherein the correct ellipse parameters include one or more of the contraction parameters, a pupil-specific parameter specific to the pupil, and an iris-specific parameter specific to the iris, and the training estimation means estimates the ellipse parameters including the one or more contraction parameters, the pupil-specific parameter, and the iris-specific parameter.

4. The information processing device according to claim 2 or 3, wherein the contracted parameters include at least one of predetermined parameters included in the pupil shape parameters and the iris shape parameters, and parameters representing components whose dimensionality has been reduced by performing statistical analysis processing on the pupil shape parameters and the iris shape parameters.

5. The information processing device according to claim 4, wherein the learning estimation means includes: ellipse parameter estimation means for estimating the ellipse parameters for the learning iris image using the estimation model; ellipse specific parameter estimation means for estimating the pupil shape parameters and the iris shape parameters for the learning iris image using an ellipse specific parameter estimation model for estimating the pupil shape parameters and the iris shape parameters included in the iris image; and correction means for correcting the estimated ellipse parameters using the estimated pupil shape parameters and iris shape parameters.

6. The information processing device according to claim 5, wherein the ellipse specific parameter estimation means includes: image processing means that performs image processing including cutting out an iris region including the pupil and iris from the training iris image using the estimated ellipse parameters; and processed image parameter estimation means that inputs the training iris image that has undergone the image processing into the ellipse specific parameter estimation model and estimates the ellipse specific parameter for the training iris image.

7. The information processing device according to any one of claims 2 to 6, wherein the updating means includes: loss calculation means for calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct answer ellipse parameters, and weights for each corresponding component in the estimated ellipse parameters and the correct answer data; parameter updating means for updating model parameters included in the estimated model using the calculated weighted loss; and the loss calculation means includes: weight determination means for determining the weight for a second component included in the correct answer data based on the magnitude of a parameter of a first component included in the correct answer data; and weighted loss calculation means for calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weight; and the second component is a component different from the first component.

8. An information processing device according to any one of claims 2 to 7, wherein the auxiliary parameters include at least one of a parameter representing a gaze direction and a parameter representing the position of one or more key points, and the one or more key points include predetermined locations on at least one of the eyelid, pupil, and iris.

9. The information processing device according to claim 8, wherein the auxiliary parameters include parameters representing positions of the one or more keypoints, and the updating means includes: loss calculation means for calculating a weighted loss using the estimated ellipse parameters, correct answer data including the correct ellipse parameters, and weights for each component corresponding to the estimated ellipse parameters and the correct answer data; parameter updating means for updating model parameters included in the estimation model using the calculated weighted loss, and the loss calculation means includes: weight determination means for determining the weight for the one or more predetermined keypoints based on whether or not the one or more keypoints are hidden by the eyelid, and weighted loss calculation means for calculating the weighted loss using the estimated ellipse parameters, the correct answer data, and the determined weights.

10. The information processing device according to any one of claims 2 to 9, further comprising an estimation means for estimating the ellipse parameters for an iris image for authentication that includes the iris of an object to be authenticated, using the trained estimation model.

11. The information processing device according to claim 10, wherein the ellipse parameters include the auxiliary parameters, and the trained estimation model outputs the ellipse parameters excluding the auxiliary parameters for the authentication iris image.

12. An information processing method, comprising: one or more computers estimating ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters for the contour or edge of the pupil and iris included in the iris image; and updating model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and iris included in the iris image.

13. A recording medium having recorded thereon a program that causes one or more computers to estimate ellipse parameters for a training iris image using an estimation model for estimating the ellipse parameters for the contour or edge of the pupil and iris included in the iris image, and update model parameters included in the estimation model using the estimated ellipse parameters and correct ellipse parameters that are correct solutions to the ellipse parameters estimated for the training iris image, wherein the ellipse parameters include shared parameters that are shared for the pupil and iris included in the iris image.

Images