Information processing system, information processing method and program

Abstract

This information processing system (100) comprises a first processing unit (102), a normalization unit (103), a second processing unit (104), and a correcting unit (105). The first processing unit (102) performs a first process using a first neural network that uses a subject image as an input. The normalization unit (103) performs a normalization process using first output information that is the result of the first process, and generates a normalized image pertaining to the subject image. The second processing unit (104) performs a second process using a second neural network using the normalized image as an input, and extracts an image feature amount pertaining to the normalized image. The correcting unit (105) corrects a first parameter that is a parameter used in the first neural network, on the basis of information pertaining to the normalized image.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and program Regarding. [Background technology]

[0002] For example, Patent Document 1 discloses an image processing method that can distinguish and recognize whether the objects corresponding to iris images are the same object, for example, whether the iris images are of the same person, based on the iris features corresponding to the iris images.

[0003] Patent Document 1 describes that the iris position, pupil position, and corresponding iris image may be input to a neural network that performs iris segmentation, and the neural network may output a mask map corresponding to the iris region in the iris image. The neural network that performs iris segmentation can determine the iris region in the iris image through training and generate the corresponding mask map.

[0004] Furthermore, according to the description in Patent Document 1, the obtained iris image and the mask map of the detected iris region may be different in size. Therefore, Patent Document 1 describes that once the iris position in the iris image and the mask map of the iris region (segmentation result) are obtained, the image region and mask map corresponding to the iris position may be normalized, and the normalized image region and mask map may be adjusted to a preset standard.

[0005] Furthermore, Patent Document 1 describes that multi-scale feature processing may be performed on an image region corresponding to the normalized iris position, further improving feature accuracy. According to the description in Patent Document 1, this multi-scale feature processing is a processing procedure similar to multi-scale feature extraction, and by performing multi-scale feature extraction, etc., it is possible to obtain an iris feature map corresponding to an iris image. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2022-511217 Summary of the Invention [Problem to be solved by the invention]

[0007] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above. [Means for solving the problem]

[0008] According to one aspect of the present invention, a first processing means for performing a first processing using a first neural network with a target image as an input; a normalization unit that performs a normalization process using first output information that is a result of the first process to generate a normalized image related to the target image; a second processing means for performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; and a correction means for correcting a first parameter, which is a parameter used in the first neural network, based on information about the normalized image. 、 The information about the normalized image includes a normalized gradient that is a local gradient in the normalization process and is calculated based on the image feature amount so as to reduce loss. An information processing system is provided.

[0009] According to one aspect of the present invention, One or more computers performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; Correcting first parameters that are parameters used in the first neural network based on information about the normalized image. death, The information about the normalized image includes a normalized gradient that is a local gradient in the normalization process and is calculated based on the image feature amount so as to reduce loss. A method for processing information is provided.

[0010] According to one aspect of the present invention, On one or more computers, performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; modifying first parameters used in the first neural network based on information about the normalized image; and Let it run 、 The information about the normalized image includes a normalized gradient that is a local gradient in the normalization process and is calculated based on the image feature amount so as to reduce loss. Program Mu Provided. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating an overview of an information processing system according to a first embodiment. [Figure 2] 1 is a flowchart showing an outline of information processing according to the first embodiment. [Figure 3] 1 is a diagram illustrating an example of the configuration of an information processing system according to a first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a target image according to the first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of development processing according to the first embodiment. [Figure 6] 1 is a diagram illustrating an example of the physical configuration of an information processing device according to a first embodiment. [Figure 7] 10 is a flowchart showing a detailed example of a correction process according to the first embodiment. [Figure 8] FIG. 4 is a diagram for explaining a process for determining a first loss gradient based on a second loss gradient and a normalized gradient in the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of the functional configuration of a parameter correction unit according to the second embodiment. [Figure 10] 10 is a flowchart showing a detailed example of a parameter correction process according to the second embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of the configuration of an information processing system according to a third embodiment. [Figure 12] 11 is a flowchart showing an example of information processing for authentication according to the third embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of the configuration of an information processing system according to a fourth embodiment. [Figure 14] FIG. 13 is a diagram illustrating an example of the configuration of an information processing system according to a sixth embodiment. [Figure 15] FIG. 20 is a diagram illustrating an example of the functional configuration of a normalization unit according to a sixth embodiment. [Figure 16] 13 is a flowchart showing an example of information processing according to the sixth embodiment. [Figure 17] 13 is a flowchart showing an example of normalization processing according to the sixth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0013] <Embodiment 1> (overview) 1 is a diagram showing an overview of an information processing system 100 according to embodiment 1. The information processing system 100 includes a first processing unit 102, a normalization unit 103, a second processing unit 104, and a correction unit 105.

[0014] The first processing unit 102 performs a first process using a first neural network with the target image as input.

[0015] The normalization unit 103 performs normalization processing using the first output information that is the result of the first processing, and generates a normalized image for the target image.

[0016] The second processing unit 104 performs a second process using a second neural network with the normalized image as input, and extracts image features related to the normalized image.

[0017] The modifying unit 105 modifies the first parameters, which are parameters used in the first neural network, based on information about the normalized image.

[0018] According to this information processing system 100, the first parameter is modified based on information about the normalized image. Therefore, the first parameter can be modified using the chain rate of partial derivatives to reduce the loss calculated using a common loss function for the first neural network and the second neural network. Since machine learning can be performed simultaneously on the first neural network and the second neural network using a common loss function, the first parameter can be modified to extract image features suitable for authentication. Therefore, it is possible to improve the accuracy of authentication using a target image.

[0019] FIG. 2 is a flowchart showing an outline of information processing according to the first embodiment.

[0020] The first processing unit 102 performs a first process using a first neural network with a target image as an input (step S101).

[0021] The normalization unit 103 performs normalization processing using the first output information that is the result of the first processing, and generates a normalized image for the target image (step S102).

[0022] The second processing unit 104 performs a second process using a second neural network with the normalized image as an input, and extracts image features related to the normalized image (step S103).

[0023] The modifying unit 105 modifies the first parameters, which are parameters used in the first neural network, based on the information about the normalized image (step S104).

[0024] According to this information processing, the first parameter is modified based on information about the normalized image. Therefore, the first parameter can be modified using the chain rate of partial derivatives to reduce the loss calculated using a common loss function for the first neural network and the second neural network. Since machine learning can be performed simultaneously on the first neural network and the second neural network using a common loss function, the first parameter can be modified to extract image features suitable for authentication. Therefore, it is possible to improve the accuracy of authentication using a target image.

[0025] A detailed example of the first embodiment will be described below.

[0026] (Detailed example) For example, a neural network that performs iris segmentation is used in the image processing method described in the above-mentioned Patent Document 1. In addition, neural networks are generally also often used in processing for obtaining an iris feature map (multi-scale feature processing).

[0027] In this way, in order to perform authentication using a target image (an iris image in Patent Document 1), a plurality of different neural networks may be connected and performed consecutively, and there is a demand for improving the accuracy of authentication using a target image. In view of these circumstances, one example of the purpose of this disclosure is to provide an information processing system, an information processing method, a recording medium, etc. that solve the problem of improving the accuracy of authentication using a target image.

[0028] Here, "authentication using a target image" means confirming the reliability, legitimacy, etc. of the target shown in the target image. Authentication using a target image is performed, for example, by determining whether the target shown in the target image is the same as a predetermined target (for example, whether the person shown in an iris image is the same as a person registered in advance).

[0029] (Example of functional configuration of information processing system 100) 3 is a diagram showing an example of the configuration of an information processing system 100 according to embodiment 1. The information processing system 100 includes an information processing device 101. Functionally, the information processing device 101 includes a first processing unit 102, a normalization unit 103, a second processing unit 104, and a correction unit 105.

[0030] The first processing unit 102 performs a first processing using a first neural network with a target image as input. The target image is an image obtained by photographing a target. The first processing unit 102 outputs first output information that is the result of the first processing.

[0031] 4 is a diagram showing an example of a target image according to embodiment 1. In this embodiment, the target is a human eye, and the target image is an eye image obtained by photographing the human eye. The eye image is an image showing the eye, and includes, for example, one or more images of the pupil, iris, white of the eye, etc.

[0032] The target image is not limited to an eye image, but may be, for example, a facial image (i.e., an image showing a face) of a target face, an image of a target object, or the like. The eye image and facial image are not limited to a person, but may be an image showing the eyes and face of an animal. When the target image is an eye image, iris authentication can be performed. When the target image is a facial image, facial authentication can be performed. In this way, authentication according to the target image (for example, various types of biometric authentication) can be performed.

[0033] The first neural network may be an appropriate neural network such as a convolutional neural network.

[0034] The first processing unit 102 may acquire a target image in order to input the target image to the first neural network. There are various methods for the first processing unit 102 to acquire the target image. For example, the first processing unit 102 may acquire the target image from a camera (not shown). For example, the first processing unit 102 may acquire the target image from another device (not shown) via a network. For example, the first processing unit 102 may acquire the target image from a built-in or externally connected storage unit (not shown).

[0035] The first processing is, for example, processing for detecting predetermined feature points of the target in the target image, and the first output information in this case is information relating to the feature points (feature point information).

[0036] In this embodiment, an example will be described in which the first process is an eye detection process for detecting an iris in an eye image. Feature points according to this embodiment are, for example, the iris and pupil.

[0037] The first output information according to the present embodiment includes, as feature point information, the position of the iris (iris position) detected in the first process. The iris position includes, for example, the pupil center position, pupil radius, iris center position, and iris radius.

[0038] The pupil center position is information indicating the position of the pupil center. The pupil radius is information indicating the radius of the pupil. The iris center position is information indicating the position of the iris center. The iris radius is information indicating the radius of the iris.

[0039] The first process is not limited to eye detection process, and may be various other processes. Examples of the first process other than eye detection process will be described in other embodiments. Furthermore, the feature points are not limited to the pupil and iris, and may be, for example, the outer corner or inner corner of the eye. The feature point information for the outer corner and inner corner of the eye is, for example, information indicating the position of the outer corner of the eye and information indicating the position of the inner corner of the eye.

[0040] The normalization unit 103 performs normalization processing using first output information that is a result of the first processing, and generates a normalized image for the target image. The normalization unit 103 according to this embodiment performs normalization processing using the position of the iris detected in the first processing, and generates a normalized image for the iris included in the eye image.

[0041] The normalization process may include, for example, a process of nonlinearly normalizing the target image (nonlinear normalization process). The nonlinear normalization process may be, for example, an expansion process that converts a circular image into a rectangular image. A circular image is an image surrounded by two roughly concentric circles. In this case, the nonlinear normalization process may, for example, convert the polar coordinate system into a Cartesian coordinate system for the circular image.

[0042] 5 is a diagram showing an example of the unfolding process according to embodiment 1. The annular image according to this embodiment is, for example, an image of an iris. In this case, the unfolding process converts the polar coordinate system, which is formed by the length R in the radial direction with the center of the iris as the origin and the angle θ between the reference direction and the radius, into an XY coordinate system, for the image of the iris.

[0043] In this case, in the normalization process, the normalization unit 103 cuts out an image of the iris from the target image based on the first output information and performs unfolding process on the image of the iris. As a result, the normalization unit 103 generates a normalized image related to the target image.

[0044] The second processing unit 104 performs a second process using a second neural network with the normalized image as input to extract image features related to the normalized image. These image features may be used for authentication, for example.

[0045] The second neural network may be an appropriate neural network such as a convolutional neural network.

[0046] The modifying unit 105 modifies the first parameter based on information about the normalized image. The modifying unit 105 may further modify the second parameter.

[0047] The information about the normalized image includes, for example, normalized gradients, which are local gradients in the normalization process.

[0048] The first parameter is a parameter used in the first neural network. The first parameter is usually a plurality of parameters, but may be a single parameter.

[0049] The second parameter is a parameter used in the second neural network. The second parameter is usually a plurality of parameters, but may be a single parameter.

[0050] In detail, for example, the correction unit 105 functionally includes a loss calculation unit 111, a gradient calculation unit 112, and a parameter correction unit 113.

[0051] The loss calculation unit 111 calculates a loss function for calculating a loss based on the extracted image feature amount and the correct answer data.

[0052] In detail, the loss calculation unit 111 acquires the correct answer data based on instructions from the user or the like. The correct answer data may be prepared in advance. There are various methods for the loss calculation unit 111 to acquire the correct answer data. For example, the loss calculation unit 111 may acquire the correct answer data from another device (not shown) via a network. For example, the loss calculation unit 111 may acquire the correct answer data from a built-in or externally connected storage unit (not shown).

[0053] The loss calculation unit 111 obtains a loss function for calculating the error (loss) between the image feature amount and the correct answer data. For example, cross entropy (error), mean square error, etc. may be applied to the error (loss).

[0054] The gradient calculation unit 112 calculates a normalized gradient based on the normalized image. For example, the gradient calculation unit 112 calculates a normalized gradient based on the normalized image so as to reduce the loss. This loss is a value that can be calculated using the loss function calculated by the loss calculation unit 111.

[0055] The gradient calculation unit 112 according to this embodiment may further calculate at least one of the second gradient and the first gradient so as to reduce the loss.

[0056] The second gradient is a local gradient in the second neural network. Gradient calculation unit 112 calculates the second gradient at each node constituting the second neural network based on, for example, information about the second neural network.

[0057] The first gradient is a local gradient in the first neural network. Gradient calculation unit 112 calculates the first gradient at each node constituting the first neural network based on, for example, information about the first neural network.

[0058] The parameter modification unit 113 modifies the first parameter based on the normalized gradient. The parameter modification unit 113 may further obtain a first loss gradient and a second loss gradient. The parameter modification unit 113 may further modify the second parameter based on the second gradient.

[0059] The first loss gradient is the gradient of the loss function to apply to the first neural network. In other words, the first loss gradient is the gradient of the loss function L with respect to the first parameter x (∂L / ∂x). The second loss gradient is the gradient of the loss function to apply to the second neural network. In other words, the second loss gradient is the gradient of the loss function L with respect to the second parameter y (∂L / ∂y).

[0060] Here, L represents the loss function, x is a parameter applied to the first neural network, and y is a parameter applied to the second neural network.

[0061] In detail, for example, the parameter correction unit 113 corrects the second parameter based on the gradient of the loss function L and the second gradient. To correct the second parameter, the parameter correction unit 113 may use, for example, backpropagation.

[0062] Furthermore, for example, the parameter correction unit 113 calculates the second loss gradient (∂L / ∂y) using, for example, backpropagation. The parameter correction unit 113 calculates the first loss gradient (∂L / ∂x) based on, for example, the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x).

[0063] The parameter modification unit 113 modifies the first parameter based on the first loss gradient (∂L / ∂x) and the first gradient. To modify the first parameter, the parameter modification unit 113 may use, for example, backpropagation. Here, the first loss gradient is calculated based on the normalized gradient as described above. Therefore, the parameter modification unit 113 modifies the first parameter based on the normalized gradient.

[0064] (Physical configuration of information processing system 100) The information processing system 100 according to this embodiment is physically configured from an information processing device 101, which is, for example, one PC (Personal Computer).

[0065] 6 is a diagram showing an example of the physical configuration of the information processing device 101 according to embodiment 1. The information processing device 101 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0066] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and other components to each other is not limited to bus connection.

[0067] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0068] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0069] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing device 101. The processor 1020 reads each of these program modules into the memory 1030 and executes them, thereby realizing the function corresponding to the program module.

[0070] The network interface 1050 is an interface for connecting the information processing device 101 to a network N that is configured by wired or wireless communication or a combination of these.

[0071] The input interface 1060 is an interface for the user to input information, and is composed of, for example, a touch panel, a keyboard, a mouse, and the like.

[0072] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0073] The information processing device 101 may be physically configured from a plurality of devices (e.g., computers) having the physical configuration exemplified in Fig. 6. In this case, the plurality of devices may be connected to each other so as to be able to transmit and receive information to and from each other via a network configured, for example, by wire, wirelessly, or a combination thereof.

[0074] (Operation of information processing system 100) 2, the information processing system 100 executes information processing. In detail, the information processing device 101 executes the information processing.

[0075] In the information processing, the information processing device 101 performs machine learning of a first neural network using a target image and supervised answer data. In the information processing according to this embodiment, the information processing device 101 further performs machine learning of a second neural network using the target image and supervised answer data. The information processing is started, for example, when the information processing device 101 receives a predetermined start instruction from a user.

[0076] The first processing unit 102, the normalization unit 103, and the second processing unit 104 execute steps S101, S102, and S103, respectively. Then, the correction unit 105 corrects the first parameters, which are parameters used in the first neural network, based on information about the normalized image, as described above (step S104).

[0077] FIG. 7 is a flowchart showing a detailed example of the correction process (step S104) according to the first embodiment.

[0078] The loss calculation unit 111 calculates a loss function for calculating a loss based on the extracted image feature amount and the correct answer data (step S141).

[0079] In detail, for example, the loss calculation unit 111 acquires supervised data prepared in advance based on a user instruction, etc. The loss calculation unit 111 obtains a loss function for calculating the error (loss) between the image feature amount and the supervised data.

[0080] The gradient calculation unit 112 calculates a gradient so that the loss obtained from the loss function calculated in step S141 becomes small (step S142).

[0081] In detail, for example, the gradients calculated in step S142 include the second gradient, the normalized gradient, and the first gradient.

[0082] The parameter correction unit 113 corrects the parameters using the gradient calculated in step S142 (step S143).

[0083] The parameters corrected in step S143 according to this embodiment are, for example, a first parameter and a second parameter. In detail, for example, the parameter correction unit 113 uses the gradient calculated in step S142 to calculate a first update value and a second update value that reduce the loss. The first update value is a value for updating the first parameter. The second update value is a value for updating the second parameter. The parameter correction unit 113 updates the first parameter and the second parameter using the first update value and the second update value.

[0084] More specifically, for example, the parameter correction unit 113 determines a second update value and a second loss gradient (∂L / ∂y) that reduce the loss based on the gradient of the loss function L and the second gradient.

[0085] The parameter correction unit 113 calculates a first loss gradient (∂L / ∂x) that reduces the loss based on the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x). FIG. 8 is a diagram for explaining the process of calculating the first loss gradient based on the second loss gradient and the normalized gradient. FIG. 8 shows an example in which the first neural network and the second neural network are configured with a plurality of nodes N. The parameter correction unit 113 calculates, for example, the product of the normalized gradient (∂y / ∂x) calculated based on the normalized image and the second loss gradient (∂L / ∂y). As a result, the gradient calculation unit 112 calculates the first loss gradient (∂L / ∂x).

[0086] Parameter correction unit 113 determines a first update value that reduces the loss based on the first loss gradient (∂L / ∂x) and the first gradient.

[0087] After executing step S143, the parameter correction unit 113 returns to the information processing (see FIG. 2) and ends the information processing.

[0088] In this way, by calculating the first loss gradient based on the second loss gradient and the normalized gradient, the first parameter and the second parameter can be updated simultaneously (end-to-end) using a loss function common to the first neural network and the second neural network.

[0089] (Actions and Effects) As described above, according to this embodiment, the information processing system 100 includes a first processing unit 102, a normalization unit 103, a second processing unit 104, and a correction unit 105. The first processing unit 102 performs a first processing using a first neural network with a target image as input. The normalization unit 103 performs a normalization processing using first output information that is the result of the first processing, to generate a normalized image related to the target image. The second processing unit 104 performs a second processing using a second neural network with the normalized image as input, to extract image features related to the normalized image. The correction unit 105 corrects first parameters, which are parameters used in the first neural network, based on information related to the normalized image.

[0090] In this way, the first parameter is modified based on information about the normalized image. Therefore, the first parameter can be modified using the chain rate of partial derivatives to reduce the loss calculated using a common loss function for the first neural network and the second neural network. Since machine learning can be performed simultaneously on the first neural network and the second neural network using a common loss function, the first parameter can be modified so that image features suitable for authentication can be extracted. Therefore, it is possible to improve the accuracy of authentication using a target image.

[0091] According to this embodiment, the normalization process includes a process of non-linearly normalizing the target image.

[0092] This makes it possible to obtain an image suitable for extracting image features, which in turn makes it possible to extract image features suitable for authentication, thereby improving the accuracy of authentication using the target image.

[0093] According to this embodiment, the information about the normalized image includes a normalized gradient, which is a local gradient in the normalization process. The correction unit 105 includes a gradient calculation unit 112 that calculates the normalized gradient based on the normalized image, and a gradient calculation unit 113 that calculates the first parameter based on the normalized gradient. Fixes and a parameter correction unit 113 for correcting the parameter.

[0094] As a result, as described above, machine learning of the first neural network and the second neural network can be performed simultaneously using a common loss function, and the first parameters can be modified so that image features suitable for authentication can be extracted, thereby improving the accuracy of authentication using target images.

[0095] According to this embodiment, the parameter correction unit 113 further corrects second parameters, which are parameters used in the second neural network, based on second gradients, which are local gradients in the second neural network.

[0096] This allows the first and second parameters to be simultaneously (i.e., end-to-end) modified using the chain rate of partial derivatives to reduce the loss calculated using a common loss function for the first and second neural networks. Because machine learning for the first and second neural networks can be performed simultaneously using a common loss function, the first and second parameters can be simultaneously modified to extract image features suitable for authentication. This makes it possible to improve the accuracy of authentication using target images.

[0097] According to this embodiment, the correction unit 105 further includes a loss calculation unit 111 that calculates a loss function for calculating a loss based on the extracted image feature amount and ground truth data. The gradient calculation unit 112 calculates a normalized gradient based on the normalized image so as to reduce the loss.

[0098] As a result, as described above, machine learning of the first neural network and the second neural network can be performed simultaneously using a common loss function, and the first parameters can be modified so that image features suitable for authentication can be extracted, thereby improving the accuracy of authentication using target images.

[0099] According to this embodiment, the target image is an eye image obtained by photographing an eye. The first processing is an eye detection processing for detecting the iris in the eye image.

[0100] This allows the parameters (first parameters) used in the eye detection process to be corrected so that image features suitable for iris authentication can be obtained, thereby improving the accuracy of authentication using the target image.

[0101] According to the present embodiment, the first output information includes the position of the detected iris. The normalization unit 103 performs normalization processing using the position of the detected iris to generate a normalized image of the iris included in the eye image.

[0102] This makes it possible to obtain a normalized image suitable for iris authentication, thereby improving the accuracy of authentication using the target image.

[0103] <Variation 1> In the first embodiment, an example has been described in which the normalization process includes a process of cutting out an iris image (cutting process) and a development process, but the normalization process is not limited to this.

[0104] The normalization process includes, for example, at least one of (1) unfolding, (2) cropping, (3) scaling, (4) translation, (5) rotation, (6) resizing, (7) inversion, and (8) shearing. The image subjected to each process may be a target image or a region-of-interest image indicating a predetermined part or location.

[0105] (1) The unfolding process is a process for converting a circular image into a rectangular image, as described in the first embodiment.

[0106] (2) The cutout process is a process of cutting out an image of a region of interest from an image. The cutout process in the first embodiment is an example of a cutout process in which the iris is set as the region of interest, and can be performed using the iris center position and iris diameter.

[0107] (3) Scale conversion processing is processing that converts an image so that the lengths of predetermined portions related to the image have a predetermined relationship. For example, scale conversion processing is processing that converts an image using the pupil center position, pupil radius, iris center position, and iris radius so that the pupil radius and iris radius have a predetermined relationship (for example, a predetermined ratio).

[0108] (4) Translation processing is processing for translating an image. For example, translation processing is processing for translating an image of an iris identified using the iris center position and radius.

[0109] (5) Rotation processing is a process of rotating an image. For example, rotation processing is a process of rotating an eye image so that the outer and inner corners of the eyes in the eye image are horizontal, using position information of the outer and inner corners of the eyes in the target image.

[0110] (6) The resizing process is a process of changing (enlarging or reducing) the size of an image. For example, the resizing process is a process of enlarging or reducing an image of an eye, a pupil, or the like so that the iris radius becomes a predetermined size.

[0111] (7) Inversion is the process of inverting an image. (8) Shearing is the process of shearing an image, which is a movement parallel to a line by an amount proportional to the signed distance from the line.

[0112] (Actions and Effects) By performing the normalization process exemplified in this modification, an image suitable for extracting image features can be obtained. This makes it possible to extract image features suitable for authentication. Therefore, it is possible to improve the accuracy of authentication using the target image.

[0113] <Embodiment 2> The functional configuration, physical configuration, and operation of the information processing system according to this embodiment are generally similar to those of the information processing system 100 according to embodiment 1. In this embodiment, a detailed configuration example of the parameter correction unit 113 will be described. Also, a detailed example of the parameter correction process (step S143) will be described.

[0114] In this embodiment, for the sake of simplicity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0115] 9 is a diagram illustrating an example of the functional configuration of the parameter correction unit 113 according to embodiment 2. The parameter correction unit 113 functionally includes a second update unit 121, an error propagation unit 122, and a first update unit 123.

[0116] Second update unit 121 calculates a second update value based on the gradient of loss function L and the second gradient. Error propagation unit 122 calculates a first loss gradient (∂L / ∂x) based on the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x). First update unit 123 calculates a first update value based on the first loss gradient (∂L / ∂x) and the first gradient.

[0117] FIG. 10 is a flowchart showing a detailed example of the parameter correction process (step S143) according to the second embodiment.

[0118] The second update unit 121 obtains a second update value based on the gradient of the loss function L and the second gradient (step S143a).

[0119] The error propagation unit 122 calculates the first loss gradient (∂L / ∂x) based on the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x) (step S143b). For example, the error propagation unit 122 calculates the first loss gradient (∂L / ∂x) as the product of the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x).

[0120] The first update unit 123 calculates a first update value based on the first loss gradient (∂L / ∂x) and the first gradient (step S143c), and returns to the correction process (step S104) shown in FIG.

[0121] (Actions and Effects) As described above, according to this embodiment, the gradient calculation unit 112 further calculates a first gradient, which is a local gradient in the first neural network, and a second gradient, which is a local gradient in the second neural network, so as to reduce the loss.

[0122] Parameter correction unit 113 includes second update unit 121, error propagation unit 122, and first update unit 123. Second update unit 121 calculates a second update value for updating the second parameter based on the gradient of loss function L and the second gradient. Error propagation unit 122 calculates a first loss gradient (∂L / ∂x), which is the gradient of the loss function to be applied to the first neural network, based on the second loss gradient (∂L / ∂y) and the normalized gradient (∂y / ∂x). First update unit 123 calculates a first update value for updating the first parameter based on the first loss gradient (∂L / ∂x) and the first gradient.

[0123] As a result, similar to the first embodiment, the chain rate of partial derivatives can be used to simultaneously (i.e., end-to-end) modify the first and second parameters so as to reduce the loss calculated using a common loss function for the first and second neural networks. Since machine learning can be performed simultaneously on the first and second neural networks using a common loss function, the first and second parameters can be simultaneously modified so as to extract image features suitable for authentication. This improves the accuracy of authentication using target images.

[0124] <Embodiment 3> In the third embodiment, an example in which an information processing system is configured from a plurality of devices will be described. In this embodiment, for simplicity of explanation, explanations that overlap with other embodiments will be omitted as appropriate.

[0125] 11 is a diagram showing an example of the configuration of an information processing system 300 according to embodiment 3. The information processing system 300 includes an information processing device 101 that is functionally and physically similar to those in embodiment 1, and an authentication device 331.

[0126] The information processing device 101 and the authentication device 331 are connected via a network N that is configured by wired or wireless communication or a combination of these, and can transmit and receive information to and from each other via the network N.

[0127] The authentication device 331 functionally includes the first processing unit 102, normalization unit 103, and second processing unit 104, which are the same as those in the first embodiment, and an authentication unit 332. The authentication unit 332 performs authentication processing using the extracted image feature amount.

[0128] The authentication device 331 may be physically similar to the information processing device 101 according to the first embodiment (see FIG. 6). However, the storage device 1040 of the authentication device 331 may store a program module for realizing the functions of the authentication device 331.

[0129] FIG. 12 is a flowchart illustrating an example of information processing for authentication according to the third embodiment.

[0130] In the information processing for authentication, the first processing unit 102, normalization unit 103, and second processing unit 104 of the authentication device 331 execute steps S101, S102, and S103 similar to those in the first embodiment, respectively.

[0131] The authentication unit 332 performs authentication processing using the image feature amount extracted in step S103 (step S305).

[0132] In detail, for example, the authentication unit 332 compares pre-registered image features with the image features extracted in step S103. The pre-registered image features correspond to the image features extracted in step S103, and are, for example, image features of a person obtained by performing steps S101, S102, and S103 at the time of registration using an image of a person's eyes (image for registration).

[0133] The authentication unit 332 determines whether authentication is successful or not based on the result of comparing the image features and outputs the determination result. For example, if the compared image features match, the authentication unit 332 determines that the person registered in advance and the person photographed to obtain the target image are the same, and that authentication is successful. For example, if the compared image features do not match, the authentication unit 332 determines that the person registered in advance and the person photographed to obtain the target image are different, and that authentication is unsuccessful.

[0134] Here, the matching of image feature amounts may mean that the image feature amounts are a perfect match, or that the similarity of the image feature amounts is within a predetermined range.

[0135] (Actions and Effects) This embodiment also provides the same functions and effects as those of embodiment 1. Furthermore, this embodiment makes it possible to perform highly accurate authentication using a target image.

[0136] <Embodiment 4> In the fourth embodiment, another example in which an information processing system is configured from a plurality of devices will be described. In this embodiment, for simplicity of explanation, explanations that overlap with other embodiments will be omitted as appropriate.

[0137] 13 is a diagram showing an example of the configuration of an information processing system 400 according to the fourth embodiment. The information processing system 400 includes an information processing device 401 and an authentication device 331 that is functionally and physically similar to that of the third embodiment.

[0138] The information processing device 401 has the same functionalities as the correction unit 105 in the first embodiment.

[0139] The information processing device 401 may be physically similar to the information processing device 101 according to the first embodiment (see FIG. 6). However, the storage device 1040 of the information processing device 401 may store a program module for realizing the functions of the information processing device 401.

[0140] The information processing system 400 according to this embodiment executes information processing similar to that of the first embodiment and information processing for authentication similar to that of the third embodiment.

[0141] However, steps S101 to S103 in the information processing are executed by the first processing unit 102, the normalization unit 103, and the second processing unit 104 of the authentication device 331, respectively.

[0142] In step S104 according to this embodiment, the correction unit 105 (for example, the loss calculation unit 111) acquires the result of step S103 from the authentication device 331, for example, via the network N. Then, the correction unit 105 may use the acquired result of step S103 to execute processing similar to the correction processing (step S104) according to the first embodiment.

[0143] (Actions and Effects) This embodiment also provides the same functions and effects as those of embodiment 1. Furthermore, this embodiment makes it possible to perform highly accurate authentication using a target image.

[0144] <Embodiment 5> In the first embodiment, the first process is described as a process for detecting predetermined feature points (e.g., irises and pupils) of an object (e.g., an eye) in an object image. However, the first process is not limited to this.

[0145] The first processing may be, for example, super-resolution processing for generating, based on a target image, a super-resolution image that has a higher resolution than the target image.

[0146] The first processing may be a sharpening processing for generating a clear image, which is an image with a higher sharpness than the target image, based on the target image. An image with a high sharpness may be, for example, an image that is out of focus when photographing, or an image that is too sharp to be photographed. of If the target image contains blurred areas due to reasons such as dirty lenses, this image will enhance the clarity of those areas.

[0147] In this embodiment, a case where the first processing is super-resolution processing will be described as an example. In this embodiment, for simplicity, descriptions that overlap with other embodiments will be omitted as appropriate.

[0148] The first processing according to this embodiment is a super-resolution processing for generating, based on a target image, a super-resolution image that is an image with a higher resolution than the target image. In this embodiment, the target image may be an eye image obtained by photographing an eye, as in the first embodiment. The first output information may include the super-resolution image generated based on the eye image.

[0149] The normalization unit 103 may perform normalization processing using the super-resolution image, which is an image included in the first output information, to generate a normalized image related to the eye image.

[0150] In more detail, for example, the normalization unit 103 may perform normalization processing using the iris position and the super-resolution image to generate a normalized image of the eye image. One example of this normalization processing is processing to cut out an iris image based on the iris position from a super-resolution image obtained by performing super-resolution processing on a target image.

[0151] The normalization unit 103 can acquire the iris position in various ways.

[0152] The normalization unit 103 may obtain the iris position based on, for example, a user input.

[0153] Furthermore, for example, the information processing system (e.g., the information processing device 101) may further include a third processing unit that performs third processing using a third neural network with a target image as input. In this case, the third processing may be eye detection processing for detecting the iris in the eye image. Third output information resulting from the third processing may include the position of the iris. The modifying unit 105 may modify the third parameter based on information related to the normalized image, similar to the first parameter. The third parameter is a parameter used in the third neural network.

[0154] Except for these, the functional configuration and operation of the information processing system according to this embodiment may be the same as, for example, embodiment 1. Furthermore, the physical configuration of the information processing system according to this embodiment may be the same as, for example, embodiment 1.

[0155] (Actions and Effects) According to this embodiment, the first processing is super-resolution processing for generating, based on a target image, a super-resolution image that has a higher resolution than the target image.

[0156] This makes it possible to obtain an image suitable for extracting image features from the target image, thereby extracting image features suitable for authentication, thereby improving the accuracy of authentication using the target image.

[0157] According to this embodiment, the target image is an eye image obtained by photographing an eye. The first output information includes a super-resolution image generated based on the eye image. The normalization unit 103 performs a normalization process using the super-resolution image to generate a normalized image related to the eye image.

[0158] This makes it possible to obtain an image suitable for extracting image features from the target image, thereby extracting image features suitable for iris authentication, thereby improving the accuracy of authentication using the target image.

[0159] According to this embodiment, the first processing is a sharpening processing for generating, based on the target image, a sharp image that is an image with a sharpness higher than that of the target image.

[0160] This makes it possible to obtain an image suitable for extracting image features from the target image, thereby extracting image features suitable for authentication, thereby improving the accuracy of authentication using the target image.

[0161] <Embodiment 6> In the sixth embodiment, an example including a mask process for excluding a predetermined exclusion area from an image will be described. In this embodiment, for simplicity of explanation, explanations that overlap with other embodiments will be omitted as appropriate.

[0162] (Configuration of information processing system 600) 14 is a diagram showing an example of the configuration of an information processing system 600 according to the sixth embodiment. The information processing system 600 includes an information processing device 601. Functionally, the information processing device 601 includes the first processing unit 102, the second processing unit 104, and the correction unit 105 that are similar to those of the first embodiment, and a normalization unit 603 that replaces the normalization unit 103 according to the first embodiment.

[0163] The normalization unit 603 may perform normalization processing using the first output information that is the result of the first processing, similarly to embodiment 1, to generate a normalized image for the target image. The normalization processing includes the same normalization processing as embodiment 1 and mask processing for excluding a predetermined exclusion region from the image.

[0164] 15 is a diagram illustrating an example of the functional configuration of the normalization unit 603 according to embodiment 6. The normalization unit 603 includes a first normalization unit 603a and a second normalization unit 603b.

[0165] The first normalization unit 603a performs a first normalization process. The first normalization process is, for example, a mask process for excluding a predetermined exclusion region from the target image. The predetermined exclusion region is, for example, a region indicating at least one of the eyelids and eyelashes.

[0166] The second normalization unit 603b performs second normalization processing on the target image from which the exclusion region has been removed. The second normalization processing is, for example, the same normalization processing as in the first embodiment.

[0167] The information processing device 601 may be physically similar to the information processing device 101 according to the first embodiment (see FIG. 6).

[0168] (Operation of information processing system 600) 16 is a flowchart showing an example of information processing according to embodiment 6. The first processing unit 102 performs step S101 similar to that of embodiment 1.

[0169] The normalization unit 603 performs normalization processing using the first output information that is the result of the first processing, and generates a normalized image for the target image (step S602).

[0170] FIG. 17 is a flowchart showing an example of the normalization process (step S602) according to the sixth embodiment.

[0171] The first normalization unit 603a performs a first normalization process (step S602a).

[0172] The first normalization process is, for example, as described above, a mask process for excluding a predetermined exclusion region indicating at least one of the eyelids, eyelashes, and the like from the target image.

[0173] The second normalization unit 603b performs the second normalization process on the target image that has been subjected to the first normalization process in step S602a (step S602b).

[0174] The second normalization process is, for example, the same normalization process as in the first embodiment.

[0175] That is, the second normalization unit 603b cuts out an iris image from the target image from which the exclusion region has been removed, based on the iris position included in the first output information. Then, the second normalization unit 603b performs, for example, an unfolding process on the cut-out iris image.

[0176] As described in variant example 1, the second normalization process may be at least one of the following processes: (1) unfolding process, (2) cutting process, (3) scaling process, (4) translation process, (5) rotation process, (6) resizing process, (7) inversion process, and (8) shearing process.

[0177] After executing step S602b, the second normalization unit 603b returns to the information processing shown in FIG.

[0178] See Fig. 16. The second processing unit 104 and the correction unit 105 respectively execute steps S103 and S104 similar to those in the first embodiment. Then, the correction unit 105 ends the information processing.

[0179] (Actions and Effects) According to this embodiment, the normalization process includes a mask process for excluding a predetermined exclusion region from the target image.

[0180] By performing normalization processing including masking, it is possible to obtain an image suitable for extracting image features. This makes it possible to extract image features suitable for authentication. Therefore, it is possible to improve the accuracy of authentication using the target image.

[0181] According to this embodiment, the target image is an eye image obtained by photographing an eye, and the predetermined exclusion area is an area showing at least one of the eyelid and the eyelashes.

[0182] By performing normalization processing including such mask processing, it is possible to obtain an image suitable for extracting image features. This makes it possible to extract image features suitable for iris authentication. Therefore, it is possible to improve the accuracy of authentication using the target image.

[0183] Although the embodiments and modifications of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0184] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps performed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments and variations can be combined as long as the content is not contradictory.

[0185] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0186] 1. A first processing means for performing a first processing using a first neural network with a target image as an input; a normalization unit that performs a normalization process using first output information that is a result of the first process to generate a normalized image related to the target image; a second processing means for performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; and a correction means for correcting a first parameter, which is a parameter used in the first neural network, based on information about the normalized image. Information processing system. 2. The normalization process includes a process of nonlinearly normalizing the target image or an image included in the first output information, 1. The information processing system described in 3. The information about the normalized image includes a normalized gradient, which is a local gradient in the normalization process; The correction means is gradient calculation means for calculating the normalized gradient based on the normalized image; and parameter modifying means for modifying the first parameter based on the normalized gradient. 1. An information processing system according to 1. or 2. 4. The parameter correction means further corrects second parameters, which are parameters used in the second neural network, based on second gradients, which are local gradients in the second neural network. 3. The information processing system described in item 3. 5. The correction means further includes a loss calculation unit that calculates a loss function for calculating a loss based on the extracted image feature amount and the correct answer data; The gradient calculation means calculates the normalized gradient based on the normalized image so as to reduce the loss. 4. The information processing system described in item 4. 6. The gradient calculation means further calculates a first gradient, which is a local gradient in the first neural network, and a second gradient, which is a local gradient in the second neural network, so as to reduce the loss; The parameter modification means second updating means for determining a second update value for updating a second parameter based on the gradient of the loss function and the second gradient; an error propagation means for calculating a first loss gradient, which is a gradient of a loss function to be applied to the first neural network, based on a second loss gradient, which is a gradient of a loss function to be applied to the second neural network, and the normalized gradient; and a first update means for determining a first update value for updating the first parameter based on the first loss gradient and the first gradient. 5. The information processing system described in. 7. The target image is an eye image obtained by photographing an eye; The first processing is an eye detection processing for detecting an iris in the eye image. 1. An information processing system according to any one of 1. to 6. 8. The first output information includes a position of the detected iris; The normalization means performs normalization processing using the detected position of the iris to generate a normalized image of the iris included in the eye image. 7. The information processing system described in. 9. The first processing is a super-resolution processing for generating a super-resolution image, which is an image with a higher resolution than the target image, based on the target image. 6. An information processing system according to any one of 1. to 5. 10. The target image is an eye image obtained by photographing an eye; the first output information includes the super-resolution image generated based on the eye image; The normalization means performs normalization processing using the super-resolution image to generate a normalized image related to the eye image. 9. The information processing system described in. 11. The first processing is a sharpening processing for generating a sharp image, which is an image with a sharpness higher than that of the target image, based on the target image. 6. An information processing system according to any one of 1. to 5. 12. The normalization process includes at least one of (1) an unfolding process that converts a circular image into a rectangular image, (2) a cutting process that cuts out an image of a region of interest from the image, (3) a scaling process that converts the image so that the lengths of predetermined portions related to the image have a predetermined relationship, (4) a translation process that translates the image, (5) a rotation process that rotates the image, (6) a resizing process that changes the size of the image, (7) an inversion process that flips the image, and (8) a shearing process that shears the image. An information processing system according to any one of 1. to 11. 13. The normalization process includes a mask process for excluding a predetermined exclusion region from the target image. An information processing system according to any one of 1. to 12. 14. The target image is an eye image obtained by photographing an eye; The predetermined exclusion area is an area showing at least one of the eyelids and the eyelashes. 13. An information processing system according to the present invention. 15. One or more computers performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; modifying first parameters used in the first neural network based on information about the normalized image; Information processing methods. 16. On one or more computers, performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; modifying first parameters used in the first neural network based on information about the normalized image; A recording medium on which a program for executing the above is recorded. [Explanation of symbols]

[0187] 100,300,400,600 Information Processing Systems 101,401,601 Information processing equipment 102 First Processing Section 103,603 Normalization part 104 Second Processing Section 105 Correction section 111 Loss calculation section 112 Gradient calculation unit 113 Parameter Modification Section 121 2nd update part 122 Error Propagation Unit 123 1st update part 331 Authentication Device 332 Authentication Department 603a 1st normalization part 603b Second normalization section

Claims

1. a first processing means for performing a first processing using a first neural network with a target image as an input; a normalization unit that performs a normalization process using first output information that is a result of the first process to generate a normalized image related to the target image; a second processing means for performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; a correcting means for correcting a first parameter, which is a parameter used in the first neural network, based on information about the normalized image; The information about the normalized image includes a normalized gradient that is a local gradient in the normalization process and is calculated based on the image feature amount so as to reduce loss. Information processing system.

2. The normalization process includes a process of nonlinearly normalizing the target image or the image included in the first output information. The information processing system according to claim 1 .

3. the target image is an eye image obtained by photographing an eye, The first process is an eye detection process for detecting an iris in the eye image.

3. The information processing system according to claim 1 or 2.

4. The first processing is a super-resolution processing for generating a super-resolution image, which is an image having a higher resolution than the target image, based on the target image.

3. The information processing system according to claim 1 or 2.

5. The first processing is a sharpening processing for generating a sharp image, which is an image with a sharpness higher than that of the target image, based on the target image.

3. The information processing system according to claim 1 or 2.

6. The normalization process includes at least one of (1) an unfolding process that converts a circular image into a rectangular image, (2) a cutting process that cuts out an image of a region of interest from the image, (3) a scaling process that converts the image so that the lengths of predetermined portions related to the image have a predetermined relationship, (4) a translation process that translates the image, (5) a rotation process that rotates the image, (6) a resizing process that changes the size of the image, (7) an inversion process that flips the image, and (8) a shearing process that shears the image.

3. The information processing system according to claim 1 or 2.

7. The normalization process includes a mask process for excluding a predetermined exclusion region from the target image.

3. The information processing system according to claim 1 or 2.

8. One or more computers performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; modifying first parameters used in the first neural network based on information about the normalized image; The information about the normalized image includes a normalized gradient that is a local gradient in the normalization process and is calculated based on the image feature amount so as to reduce loss. Information processing methods.

9. On one or more computers, performing a first process using a first neural network with a target image as an input; performing a normalization process using first output information resulting from the first process to generate a normalized image for the target image; performing a second process using a second neural network with the normalized image as an input to extract image features related to the normalized image; modifying first parameters used in the first neural network based on information about the normalized image; A program in which the information about the normalized image is a local gradient in the normalization process, and includes a normalized gradient calculated so as to reduce loss calculated based on the image feature amount.

Images