Information processing systems, information processing methods, and programs

Abstract

An information processing system (100) comprises a quality estimation unit (111) and an image processing unit (113). The quality estimation unit (111) estimates quality information of an object image. The image processing unit (113) processes the object image by using a neural network according to a network policy determined on the basis of the quality information of the object image.

Description

Technical Field

[0005] , , , , , , ,

[0006] ,

[0004] , ,

[0001] This disclosure relates to an information processing system , love an information processing method, and program is related to it.

Background Art

[0002] In iris authentication, generally, when the quality such as the resolution of an image is low, the authentication accuracy may decrease. Therefore, in iris authentication, it is desirable to capture an authentication target using a high-resolution imaging device, but it may be difficult to use a high-resolution imaging device due to installation space, cost, etc. of the imaging device.

[0003] For example, Patent Document 1 discloses that when performing iris authentication, a predetermined number of captured images are used to generate an iris authentication image with a second resolution higher than the first resolution of the captured image. As an example, Patent Document 1 discloses that from eight continuously captured images, 1×1 pixels of the captured image are converted into 2×2 pixels to generate an authentication image with a resolution twice as high.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure aims to improve the technology described in the above-mentioned prior art documents.

Means for Solving the Problems

[0006] According to one aspect of the present invention, quality estimation means for estimating quality information of a target image, <000,0038>The system comprises an image processing means that processes the target image using a neural network that corresponds to a network policy determined based on the quality information of the target image. 、 A learning means that trains the neural network by inputting training target images into the neural network, A training policy determination means that determines a network policy for the training target image based on the quality information and training probability distribution of the training target image. Equipped with 、 The learning means performs training on the neural network by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. An information processing system is provided.

[0008] According to one aspect of the present invention, One or more computers, Estimate the quality information of the target image, The target image is processed using a neural network that corresponds to a network policy determined based on the quality information of the target image. death, By inputting the training target images into the neural network, the neural network is trained. Based on the quality information and training probability distribution of the aforementioned training image, a network policy is determined for the training image. This includes fruit, In training the neural network, the training of the neural network is performed by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. Information processing methods are provided.

[0009] According to one aspect of the present invention, On one or more computers, Estimate the quality information of the target image, The target image is processed using a neural network that corresponds to a network policy determined based on the quality information of the target image. death, By inputting the training target images into the neural network, the neural network is trained. Based on the quality information and training probability distribution of the aforementioned training image, a network policy is determined for the training image. This includes 、 In training the neural network, the training of the neural network is performed by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. Professional Mu It will be provided. [Brief explanation of the drawing]

[0010] [Figure 1] It is a diagram showing an overview of the information processing system according to Embodiment 1. [Figure 2] It is a diagram showing an overview of the information processing apparatus according to Embodiment 1. [Figure 3] It is a flowchart showing an overview of the information processing according to Embodiment 1. [Figure 4] It is a diagram showing a configuration example of the information processing system according to Embodiment 1. [Figure 5] It is a diagram showing a configuration example of the neural network according to Embodiment 1. [Figure 6] It is a diagram showing a physical configuration example of the information processing apparatus according to Embodiment 1. [Figure 7] It is a flowchart showing an example of the information processing according to Embodiment 1. [Figure 8] It is a diagram showing an example of the target image according to Embodiment 1. [Figure 9] It is a diagram showing a functional configuration example of the image processing unit according to Embodiment 2. [Figure 10] It is a diagram showing a configuration example of the neural network according to Embodiment 2. [Figure 11] It is a flowchart showing a detailed example of the image processing according to Embodiment 2. [Figure 12] It is a diagram showing an example of the iris area according to Embodiment 2. [Figure 13] It is a diagram showing an example of the standard image according to Embodiment 2. [Figure 14] It is a diagram showing a configuration example of the neural network according to Embodiment 3. [Figure 15] It is a diagram showing a configuration example of the information processing system according to Embodiment 4. [Figure 16] It is a flowchart showing an example of the learning process according to Embodiment 4. [Figure 17] It is a diagram showing an example of the probability distribution for learning according to Embodiment 4. [Figure 18] It is a diagram showing a configuration example of the information processing system according to Embodiment 5. [Figure 19]This flowchart shows an example of the learning process according to Embodiment 5. [Figure 20] This figure shows an example of a selection probability distribution according to Embodiment 5. [Figure 21] This figure shows an example configuration of the information processing system according to Embodiment 6. [Figure 22] This flowchart shows an example of the learning process according to Embodiment 6. [Modes for carrying out the invention]

[0011] Hereinafter, one embodiment of the present invention will be described with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted where appropriate.

[0012] <Embodiment 1> (overview) Figure 1 is a diagram showing an overview of the information processing system 100 according to Embodiment 1. The information processing system 100 includes a quality estimation unit 111 and an image processing unit 113.

[0013] The quality estimation unit 111 estimates the quality information of the target image. The image processing unit 113 processes the target image using a neural network that corresponds to the network policy determined based on the quality information of the target image.

[0014] This information processing system 100 makes it possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0015] Figure 2 shows an overview of the information processing device 102 according to Embodiment 1. The information processing device 102 includes a quality estimation unit 111 and an image processing unit 113.

[0016] The quality estimation unit 111 estimates the quality information of the target image. The image processing unit 113 processes the target image using a neural network that corresponds to the network policy determined based on the quality information of the target image.

[0017] This information processing device 102 makes it possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0018] Figure 3 is a flowchart showing an overview of the information processing according to Embodiment 1.

[0019] The quality estimation unit 111 estimates the quality information of the target image (step S103).

[0020] The image processing unit 113 processes the target image using a neural network that corresponds to a network policy determined based on the quality information of the target image (step S105).

[0021] This information processing method makes it possible to perform authentication regardless of the quality of the target image while keeping processing time down.

[0022] The following describes a detailed example of the information processing system 100 according to Embodiment 1.

[0023] (detail) The technology described in Patent Document 1 above uses so-called image super-resolution technology to improve resolution. However, image super-resolution has the problem of requiring a long processing time.

[0024] One example of the purpose of this disclosure is, in light of these circumstances, to provide an information processing system, information processing device, information processing method, and recording medium that solve the problem of performing authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0025] (Example of the configuration of the information processing system 100 according to Embodiment 1) Figure 4 shows an example of the configuration of the information processing system 100 according to Embodiment 1. The information processing system 100 is a system for performing processing using an image of an object that has been photographed. In this embodiment, an example is given in which the processing using the image of the object is an authentication process for authenticating the object.

[0026] The information processing system 100 comprises an imaging device 101 and an information processing device 102.

[0027] The imaging device 101 and the information processing device 102 are connected via a network configured by wire, wireless, or a combination thereof, and they may send and receive information from each other via the network.

[0028] The subject in this embodiment is a human being. However, the subject is not limited to a human being; it may also be an animal, such as a dog or a snake.

[0029] Furthermore, the authentication process according to this embodiment performs iris authentication. In this embodiment, an example of iris authentication using an image of the right eye iris will be described. However, the authentication process is not limited to this, and any image-based authentication process is acceptable, such as iris authentication using an image of the left eye iris or both eyes' irises, facial authentication using a facial image showing the entire face, or vein authentication using an image showing veins.

[0030] The imaging device 101 photographs the target and generates an image of the target.

[0031] The target image is an image that includes the target region used for authentication processing. In this embodiment, as described above, iris authentication is performed using an image, so the target region is the iris region in which the iris is reflected. More specifically, in this embodiment, iris authentication is performed using an image of the right eye iris, so the target region is the iris region in which the right eye iris is reflected. Note that iris authentication may also be performed using an image of the left eye iris, in which case the target region is: left It may also be the iris region where the iris of the eye is reflected.

[0032] (Example of the functional configuration of the information processing device 102 according to Embodiment 1) The information processing device 102 according to this embodiment uses the target image generated by the imaging device 101 to authenticate the object that appears in the target image.

[0033] In this embodiment, as will be described later, an example is used in which the information processing device 102 is equipped with an authentication unit 114 that performs authentication processing (authentication processing) for the target. However, the information processing device 102 only needs to perform processing to authenticate the target using the target image, and the authentication processing may be performed by another information processing device, for example, which is not shown. In this case, it is preferable that the other information processing device is equipped with the authentication unit 114.

[0034] As shown in Figure 4, the information processing device 102 functionally comprises, for example, a quality estimation unit 111, a policy determination unit 112, an image processing unit 113, and an authentication unit 114.

[0035] The quality estimation unit 111 estimates the quality information of the target image.

[0036] Quality information is information that indicates the quality of the target image. In this embodiment, quality is the resolution of the target image, for example, the resolution of the target region included in the target image. More specifically, in this embodiment, quality is the resolution of the iris region in which the right eye iris used in the authentication process is reflected. When quality is resolution, the lower the resolution, the lower the quality.

[0037] The quality estimation unit 111 may further estimate the position of the target region in the target image. The position of the target region is information for identifying the position of the target region in the target image. In this embodiment, since the target region is the iris region described above, the position of the target region in this embodiment is the position of the right eye iris in the target image. Hereinafter, the position of the right eye iris in the target image will also be referred to as the "eye position".

[0038] Eye position includes, for example, the position of the pupil center, pupil diameter, and iris diameter. Pupil diameter is the diameter or radius of the pupil. Iris diameter is the diameter or radius of the outer edge of the iris. By using such eye position, the position of the right eye iris in the target image can be determined.

[0039] Furthermore, the quality information indicates quality that is not limited to the resolution of the target image, but may include one or more of the following: the resolution of the target image, the iris diameter of the iris reflected in the target image, the estimated distance between the imaging device 101 and the target, and the presence or degree of blur. This estimated distance may be estimated from information used for focusing on the imaging device 101, or it may be estimated based on information from a distance measuring sensor (not shown) for estimating the distance to the target. In addition, the resolution of the target image is not limited to the target area, but may include, for example, the resolution of a predetermined area included in the target image.

[0040] Furthermore, quality information may be represented, for example, as a vector quantity. When quality information is represented by multiple parameters, using a vector quantity to represent the quality information allows the multiple parameters to be treated as a single variable.

[0041] The policy determination unit 112 determines a network policy based on the quality information estimated by the quality estimation unit 111. This network policy is applied to the neural network used for image processing by the image processing unit 113, which will be described later.

[0042] The policy determination unit 112 may, for example, determine a network policy that satisfies at least one of the following conditions (A) and (B), regardless of the quality information of the target image.

[0043] (A) The processing time using the neural network must be within a predetermined range. (B) The accuracy of authentication using the target image must be within a predetermined range.

[0044] The image processing unit 113 processes the target image using a neural network corresponding to the network policy determined by the policy determination unit 112. In processing the target image according to this embodiment, the image processing unit 113 extracts features from the target image, for example.

[0045] Here, the image processing unit 113 may process the target image using a neural network that performs calculations using more detailed information the lower the quality of the target image (i.e., the quality indicated by the quality information), depending on the network policy. In other words, the policy determination unit 112 determines a network policy that performs calculations using more detailed information the lower the quality of the target image.

[0046] Figure 5 shows an example configuration of the neural network NN1 according to Embodiment 1. The neural network NN1 is used for processing the target image performed by the image processing unit 113. Here, an example in which the neural network NN1 is a convolutional neural network is used for explanation. However, the neural network NN1 is not limited to this, and for example, a vision transformer may be used.

[0047] The neural network NN1, for example, includes multiple layers. The multiple layers included in the neural network NN1 consist of a pre-stage group FNN1 that includes an input layer INL, and a post-stage group RNN that includes a final layer OUTL.

[0048] The number of layers in the preceding group FNN1 and the succeeding group RNN may be changed as appropriate, but for example, the number of layers in the preceding group FNN1 may be greater than the number of layers in the succeeding group RNN. In other words, the preceding group FNN1 may be located on the input side of the layer that is located in the middle of the multiple layers that make up the neural network NN1.

[0049] The preceding group FNN1 includes, for example, an input layer INL, at least one convolutional layer FCL, and an intermediate output layer MOUT. The preceding group FNN1 may also include batch norm layers, activation functions, etc., similar to a typical convolutional neural network (not shown).

[0050] The input layer INL acquires the input image for the neural network NN1 and propagates the input image to subsequent layers, for example. The input image is the target image. The image processing unit 113 may also extract an image contained within the target image (for example, an image of the target region) and use this extracted image as the input image. In other words, the input image can be any image obtained from the target image.

[0051] Each layer, including the convolutional layer FCL, processes the input image, for example, by extracting features from the input image. The intermediate output layer MOUT outputs the processing results from the neural network NN1 as intermediate features. Note that the intermediate output layer MOUT may also be a convolutional layer.

[0052] Here, the network policy determined by the policy determination unit 112 is applied to image processing using the preceding group FNN1. In other words, the policy determination unit 112 determines the network policy to be applied to the preceding group FNN1. Specifically, for example, the policy determination unit 112 determines a network policy that uses more detailed information for calculations the lower the quality of the target image.

[0053] In this case, the policy determination unit 112 may determine a network policy that satisfies at least one of (A) and (B) above. For example, the policy determination unit 112 may pre-store policy information that defines network policies according to the quality of the target image, and determine a network policy according to the quality of the target image by obtaining a network policy according to the quality of the target image from the policy information. In the policy information, for example, it is preferable that a network policy is pre-defined in which the preceding group FNN1 performs calculations using detailed information according to the quality of the target image, so as to satisfy at least one of (A) and (B) above.

[0054] The subsequent group RNN includes, for example, an intermediate input layer MIN, at least one convolutional layer RCL, and a final layer (output layer) OUTL. The subsequent group RNN may also include batch norm layers, activation functions, etc., similar to a typical convolutional neural network (not shown).

[0055] The intermediate input layer MIN acquires intermediate features from the preceding group FNN1. Each layer, including the convolutional layer RCL, further processes the intermediate features, for example, by extracting features from the input image. Note that the intermediate input layer MIN may also be a convolutional layer.

[0056] The final layer OUTL outputs the final processing results of the neural network NN1. The final layer OUTL outputs feature vectors based on the results of further processing in each layer, including, for example, the convolutional layer RCL.

[0057] Thus, the lower the quality of the target image, the more detailed the information used in the calculations, which helps to minimize the decrease in authentication accuracy even with low-quality images. On the other hand, because detailed information is used in the calculations, processing time using neural networks increases for low-quality images compared to high-quality images.

[0058] As described above, for example, the policy determination unit 112 may determine a network policy such that the processing time using the neural network (A) is within a predetermined range, regardless of the quality information of the target image (i.e., the quality indicated by the quality information). This suppresses the increase in processing time (processing time using the neural network) when the target image is of low quality, and makes it possible to keep the processing time using the neural network roughly the same regardless of the quality of the target image.

[0059] Alternatively, for example, the policy determination unit 112 may determine a network policy such that (B) the accuracy of authentication using the target image is within a predetermined range, regardless of the quality information of the target image (i.e., the quality indicated by the quality information). This suppresses the decrease in authentication accuracy when the target image is of low quality, and makes it possible to maintain a similar level of authentication accuracy in the authentication process regardless of the quality of the target image.

[0060] The authentication unit 114 performs authentication processing using the results of the image processing unit 113's processing of the target image. Authentication processing is the process of authenticating the target using the results of the processing of the target image.

[0061] For example, when the authentication unit 114 obtains the feature quantities extracted by the image processing unit 113, it performs authentication based on whether or not the obtained feature quantities are registered in the registered data. Although not shown in the diagram, the authentication unit 114 may store the registered data in advance.

[0062] Up to this point, we have mainly described examples of the functional configuration of the information processing system 100. From here, we will describe examples of the physical configuration of the information processing system 100.

[0063] (Physical configuration of information processing system 100) The information processing system 100 according to this embodiment is physically composed of, for example, a shooting device 101 and an information processing device 102. The shooting device 101 is, for example, a camera such as a near-infrared camera.

[0064] Figure 6 shows an example of the physical configuration of the information processing device 102 according to Embodiment 1. The information processing device 102 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0065] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070 to send and receive data to and from each other. However, the method of connecting the processor 1020 and other components to each other is not limited to bus connection.

[0066] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0067] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0068] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules for realizing the functions of the information processing device 102. The processor 1020 loads each of these program modules into memory 1030 and executes them, thereby realizing the functions corresponding to those program modules.

[0069] The network interface 1050 is an interface for connecting the information processing device 102 to a network.

[0070] The input interface 1060 is an interface for the user to input information, and consists of, for example, a touch panel, a keyboard, a mouse, etc.

[0071] The output interface 1070 is an interface for presenting information to the user and consists of, for example, an LCD panel or an OLED (Electro-Luminescence) panel.

[0072] The functions of the information processing device 102 may also be implemented using multiple physical information processing devices. In this case, each information processing device may be physically configured similarly to the information processing device 102 shown in Figure 6. Furthermore, the multiple information processing devices may be connected to each other via a network configured by wire, wireless, or a combination thereof, and configured to send and receive information from each other via the network.

[0073] We have now described an example configuration of the information processing system 100 according to Embodiment 1. From here, we will describe an example of operation of the information processing system 100 according to Embodiment 1.

[0074] (Example of operation of the information processing system 100 according to Embodiment 1) The information processing system 100 performs information processing. The information processing according to this embodiment is a process that uses an image of the subject taken to authenticate the subject that appears in the image.

[0075] In this embodiment, an example is described using an authentication process that performs authentication of the target, but the information processing only needs to be a process for authenticating the target using the target image, and does not necessarily have to include an authentication process.

[0076] Information processing may be initiated, for example, by receiving a signal from a sensor (not shown). This sensor is, for example, one that detects the presence or entry of a person into the area being photographed by the imaging device 101. However, the method for initiating information processing is not limited to this.

[0077] Figure 7 is a flowchart showing an example of information processing according to Embodiment 1.

[0078] The imaging device 101 photographs the target (step S101). As a result, the imaging device 101 generates an image of the target.

[0079] Figure 8 shows an example of a target image according to Embodiment 1. The target image according to this embodiment is an eye image including both eyes of the target. The eye image is an example of an image including the target region according to this embodiment. As described above, the target region according to this embodiment is the iris region in which the iris of the right eye is reflected.

[0080] The target image may be modified in various ways depending on the type of authentication performed using it. For example, in the case of facial recognition, it may be a facial image including the target's face and background, or an image including the target's upper body or entire body. In this case, the target area is the facial area in which the face is visible. include This is the target area. For example, in the case of vein authentication, the image may include the area where the veins are visible as the target area.

[0081] Furthermore, for example, the target image used in iris recognition is not limited to an eye image, but may also be a face image, an image including the upper body or the entire body of the subject, etc. Moreover, the eye image is not limited to an image including both eyes of the subject, but may also be a single-eye image including one of the predetermined left or right eyes of the subject. (Figure) 8 The eye image provided as an example includes not only the iris but also the pupil, white of the eye, eyelids, eyelashes, etc. However, the eye image only needs to include at least the iris region used for authentication processing, and may not include one or more of the pupil, white of the eye, eyelids, eyelashes, etc.

[0082] Refer to Figure 7 again. The quality estimation unit 111 estimates the position of the target region in the target image (step S102).

[0083] The quality estimation unit 111 according to this embodiment estimates, for example, the eye position including the position of the pupil center of the right eye, the pupil diameter, and the iris diameter from the target image.

[0084] The quality estimation unit 111 may use general techniques such as pattern matching or machine learning models to estimate eye positions. When using a machine learning model, the quality estimation unit 111 may use a trained model that has been trained to estimate eye positions from target images, and use the target image as input to estimate the eye positions in the target image. In training this trained model, supervised learning may be performed using training target images as input and training data that includes the eye positions in the target images.

[0085] The quality estimation unit 111 estimates the quality of the target image (step S103). As a result, the quality estimation unit 111 generates quality information including the quality of the target image.

[0086] The quality estimation unit 111 according to this embodiment identifies the iris region (i.e., the region in which the iris of the target's right eye is reflected) included in the target image, for example, based on the eye position estimated in step S102. The iris region is generally an annular region surrounded by concentric circles with diameters corresponding to the pupil diameter and iris diameter, respectively, centered on the pupil center. As shown in Figure 8, if a part of the iris region is covered by the eyelid or the like, the iris region will have a shape where the part covered by the eyelid or the like is missing from the annular region.

[0087] The quality estimation unit 111 determines the resolution of the identified iris region. Resolution is, for example, the image size. In this case, the quality estimation unit 111 determines the image size of the identified iris region. The image size of the iris region is the number of pixels in the iris region.

[0088] Alternatively, the quality estimation unit 111 may determine the quality of the target image using a learning model. In this case, the learning model should be one that has been trained to estimate the quality of the target image, and should take the target image as input to estimate the quality of the target image. Furthermore, in the training of this learning model, supervised learning should be performed using training target images as input and training data that includes the quality of the target image.

[0089] The iris diameter may be used as the image size. In this case, step S102 will include a process for estimating the quality of the target image. Even if a value other than the iris diameter is used for the quality of the target image, a common learning model may be used in steps S102 and S103. In this case, the learning model should be one that has been trained to estimate eye position and the quality of the target image from the target image, and should take the target image as input to estimate the eye position and the quality of the target image in the target image. Furthermore, in training this learning model, supervised learning should be performed using training target images as input and training data including eye position and the quality of the target image from the target image.

[0090] The policy determination unit 112 determines the network policy based on the quality of the target image estimated in step S103 (step S104).

[0091] In detail, for example, the policy determination unit 112 may determine a network policy such that at least one of (A) and (B) above is satisfied, regardless of the quality information of the target image.

[0092] The image processing unit 113 processes the target image using the neural network NN1 according to the network policy determined in step S104 (step S105).

[0093] In detail, for example, the image processing unit 113 uses a neural network NN1 according to the network policy to extract feature quantities from the iris region estimated in step S102.

[0094] Here, for example, suppose that in step S104, the policy determination unit 112 determines a network policy that satisfies both (A) and (B) regardless of the quality information of the target image. This makes it possible to make both the processing time using the neural network NN1 (i.e., processing for extracting features, for example) and the accuracy of authentication performed in the authentication process roughly the same, regardless of the quality of the target image.

[0095] The authentication unit 114 performs authentication processing using the result of the processing in step S105 (step S106).

[0096] In detail, for example, the authentication unit 114 acquires the feature quantities extracted by the image processing unit 113. The authentication unit 114 compares the acquired feature quantities with the feature quantities obtained from pre-stored registered data, and authenticates the target based on the result of the comparison.

[0097] More specifically, the authentication unit 114 calculates the similarity between the acquired features and the features obtained from the registered data. The authentication unit 114 authenticates the target based on the result of comparing the similarity with a predetermined threshold. For example, the authentication unit 114 determines that the authentication of the target was successful if the similarity is equal to or greater than the threshold. Alternatively, the authentication unit 114 determines that the authentication of the target failed if the similarity is less than the threshold. Note that the authentication method is not limited to this, and a general method may be used.

[0098] (Effects / Actions) As described above, according to this embodiment, the information processing system 100 comprises a quality estimation unit 111 and an image processing unit 113.

[0099] The quality estimation unit 111 estimates the quality information of the target image. The image processing unit 113 processes the target image using a neural network NN1 according to a network policy determined based on the quality information of the target image.

[0100] This allows the processing in the neural network NN1 to be changed according to the quality of the target image. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while keeping the increase in processing time to a minimum.

[0101] According to this embodiment, the image processing unit 113 extracts features from the target image using a neural network NN1 according to the network policy.

[0102] This allows authentication processing to be performed using features. The process for extracting features is carried out using a neural network NN1 according to the network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while keeping the increase in processing time to a minimum.

[0103] According to this embodiment, the target image is an eye image that includes the iris region in which the iris is reflected. The quality information includes the iris diameter.

[0104] This allows the processing in the neural network NN1 to be changed according to the iris diameter of the target image. Therefore, it becomes possible to perform authentication such as iris recognition regardless of the quality of the target image while suppressing the increase in processing time.

[0105] According to this embodiment, the quality information includes the resolution of the target image.

[0106] This allows the processing in the neural network NN1 to be changed according to the resolution of the target image. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing the increase in processing time.

[0107] According to this embodiment, the information processing system 100 further includes a policy determination unit 112 that determines a network policy based on quality information.

[0108] This allows for the determination of a network policy based on the quality of the target image. The processing in the neural network NN1 can then be modified according to this determined network policy. Therefore, authentication can be performed regardless of the quality of the target image while minimizing the increase in processing time.

[0109] According to this embodiment, the policy determination unit 112 determines a network policy such that the processing time using the neural network NN1 falls within a predetermined range, regardless of the resolution of the target image.

[0110] This allows the processing in the neural network NN1 to be modified according to the resolution of the target image, while suppressing the increase in processing time that results from such modifications. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing the increase in processing time.

[0111] According to this embodiment, the neural network NN1 includes a pre-stage group FNN which includes an input layer INL. 1 It consists of a downstream group RNN including the final layer OUTL. The network policy is for the upstream group FNN. 1 This is applied to image processing using this method.

[0112] This allows the processing in the neural network NN1 to be changed according to the resolution of the target image. Furthermore, it can suppress the increase in processing time compared to improving the quality of the input image fed into the neural network NN1 using image super-resolution technology. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing the increase in processing time.

[0113] <Embodiment 2> This embodiment describes detailed examples of the network policy and the processing performed by the image processing unit 113. In this embodiment, in order to simplify the explanation, explanations that overlap with the embodiments described above will be omitted as appropriate.

[0114] The information processing system according to this embodiment is preferably configured in a manner similar to the information processing system 100 according to Embodiment 1 illustrated in Figures 4 and 6. That is, for example, the image processing unit 113 processes the target image using a neural network according to the network policy, similar to Embodiment 1. Also, for example, the neural network constituting the image processing unit 113 performs calculations using more detailed information the lower the quality of the target image, according to the network policy, similar to Embodiment 1.

[0115] The image processing unit 113 according to this embodiment converts the iris region included in the target image into a standardized image of a predetermined shape, and processes the converted standardized image as an input image to the neural network.

[0116] Figure 9 shows an example of the functional configuration of the image processing unit 113 according to Embodiment 2. The image processing unit 113 includes a first processing unit 113a and a second processing unit 113b.

[0117] The first processing unit 113a converts a target region (i.e., the iris region) included in the target image into a standard image of a predetermined shape. Specifically, for example, the first processing unit 113a extracts the iris region from the target image and converts the extracted iris region into a standard image of a predetermined shape. In this way, the first processing unit 113a generates a standard image based on the target image.

[0118] The second processing unit 113b processes the standard image acquired by the first processing unit 113a using a neural network corresponding to the network policy determined by the policy determination unit 112. In processing the standard image according to this embodiment, the second processing unit 113b extracts features from the standard image, for example.

[0119] Figure 10 shows an example configuration of the neural network NN2 according to Embodiment 2. The neural network NN2 is used for processing standard images performed by the second processing unit 113b according to Embodiment 2. In this embodiment as well, the explanation will use an example in which the neural network NN2 is a convolutional neural network.

[0120] The neural network NN2 includes, for example, multiple layers, similar to Embodiment 1. The multiple layers included in the neural network NN2 consist of a pre-stage group FNN2 including an input layer INL, and a post-stage group RNN similar to Embodiment 1.

[0121] The preceding group FNN2 includes an input layer INL, CLN convolutional layers FCL, and an intermediate output layer MOUT.

[0122] The input layer INL acquires the input image, similar to Embodiment 1, and propagates the input image to subsequent layers, for example. The input image according to this embodiment is a standard image generated by the first processing unit 113a.

[0123] Each layer, including CLN convolutional layers FCL, processes the input image, for example, by extracting features from the input image. The intermediate output layer MOUT outputs the processing results from the neural network NN2 as intermediate features. Note that the intermediate output layer MOUT may also be a convolutional layer.

[0124] In this embodiment as well, the network policy determined by the policy determination unit 112 is applied to image processing using the preceding group FNN2.

[0125] In detail, for example, the policy determination unit 112 according to this embodiment determines a network policy such that the number of convolutional layers FCLs (CLN) included in the preceding group FNN2 increases as the resolution of the target image decreases. This network policy is an example of a network policy in which the preceding group FNN2 performs calculations using more detailed information as the quality of the target image decreases.

[0126] The information processing according to this embodiment is preferably configured in general the same way as the information processing according to Embodiment 1 illustrated in Figure 7. That is, for example, in step S105, the image processing unit 113 processes the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0127] Figure 11 is a flowchart showing a detailed example of the image processing (step S105) according to Embodiment 2. In the image processing (step S105), as explained with reference to Figure 7, the image processing unit 113 may process the target image using a neural network according to the network policy determined in step S104.

[0128] The first processing unit 113a extracts the target region from the target image (step S105a).

[0129] More specifically, the target region is the iris region. Therefore, the first processing unit 113a extracts the iris region in a roughly annular shape.

[0130] Figure 12 shows an example of the iris region according to Embodiment 2. In this figure, the iris region is a hatched area and is surrounded by an inner and outer circumference, which are concentric circles centered on the pupil center O. The inner circumference corresponds to the pupil diameter. The outer circumference corresponds to the iris diameter. Such an iris region can be represented, for example, in a polar coordinate system using the pupil center O as the origin, a radial length R, and a predetermined reference angle θ. The area inside the inner circumference corresponds to the pupil, but this area does not necessarily have to be included in the iris region.

[0131] Refer to Figure 11 again. The first processing unit 113a converts the target region extracted in step S105a into a standard image of a predetermined shape (step S105b). In this way, the first processing unit 113a generates a standard image.

[0132] Figure 13 shows an example of a standard image according to Embodiment 2. In this figure, an example is shown where the predetermined shape is a rectangle, but the predetermined shape is not limited to this.

[0133] When converting the annular iris region illustrated in Figure 12 to the rectangular standard image illustrated in Figure 13, for example, the height H of the standard image corresponds to the width R1 of the annular iris region. Also, for example, the width W of the standard image corresponds to the outer and inner sides of the annular iris region.

[0134] In step S105b, the first processing unit 113a may convert the iris region into a standard image of a size corresponding to its quality information (e.g., resolution). That is, the size of the standard image may vary depending on the quality of the target image (e.g., the image size of the iris region). In this case, the first processing unit 113a may convert the target region into a larger standard image the higher the quality of the target image (e.g., the larger the image size of the iris region). Also, if the standard image is rectangular, its aspect ratio may remain constant.

[0135] Any method may be used to convert the ring-shaped iris region into a rectangular standard image.

[0136] For example, the first processing unit 113a may convert the target region to standard images of discretely different sizes depending on the quality of the target image. In this case, for example, the first processing unit 113a may pre-store conversion pattern data indicating the sizes of multiple standard images corresponding to the quality of the target image. Then, the first processing unit 113a may obtain the size of the standard image corresponding to the quality of the target region estimated in step S103 from the conversion pattern data, and convert the extracted target region in step S105a to a standard image of the obtained size. Such conversions may be carried out using appropriate methods such as conversion from polar coordinates to Cartesian coordinates, or conversions to enlarge and reduce the image.

[0137] Alternatively, for example, the first processing unit 113a may convert the target area into standard images of continuously different sizes depending on the quality of the target image. In this case, for example, the first processing unit 113a may convert the iris region extracted in step S105a into a rectangular image using a conversion from a polar coordinate system to a Cartesian coordinate system. If the aspect ratio of this rectangular image differs from that of a predetermined standard image, the first processing unit 113a may convert the rectangular image into a standard image with a predetermined aspect ratio using at least one of an image enlargement or reduction conversion.

[0138] In this case, to convert the aspect ratio of a rectangular image to a predetermined aspect ratio of a standard image, the extent to which the vertical and horizontal directions are converted should be determined according to predetermined conversion conditions.

[0139] For example, if the conversion condition is not to convert in the horizontal direction, the first processing unit 113a may convert the rectangular image into a standard image which is a rectangle with a predetermined aspect ratio by enlarging or reducing it in the vertical direction. Also, for example, if the conversion condition is not to convert in the vertical direction, the first processing unit 113a may convert the rectangular image into a standard image which is a rectangle with a predetermined aspect ratio by enlarging or reducing it in the horizontal direction. Note that the conversion conditions are not limited to these.

[0140] By performing this conversion, the iris region can be transformed into a standardized image of a size corresponding to its quality information (e.g., resolution). Furthermore, the standardized image will have a quality (e.g., resolution) that corresponds to the quality information of the iris region.

[0141] Refer to Figure 11 again. The second processing unit 113b processes the standard image generated in step S105b using the neural network NN2 according to the network policy determined in step S104 (step S105c). Then, the second processing unit 113b returns to the information processing shown in Figure 7.

[0142] In detail, for example, the size of a standard image varies discretely or continuously depending on the quality of the target image, as mentioned above. Examples of network policies are described for each case.

[0143] (Example of a network policy when the sizes of standard images vary discretely) For example, let's assume that the quality of the target image is the resolution of the input image. Also, if the resolution of the input image is a predetermined reference resolution, let's assume that the number of convolutional layers FCL used in the calculations of the preceding group FNN2, CLN, is the reference value RFN1.

[0144] In this case, the policy determination unit 112 determines that if the resolution of the input image becomes 1 / N relative to the reference resolution, the preceding group FNN 2 A network policy may be decided to increase the number of convolutional layers (FCLs) used in the calculation (CLN) to RFN1*N*N. N and RFN1 are both integers greater than or equal to 1. The "*" represents multiplication.

[0145] As a result, the second processing unit 113b processes the input image using a pre-stage group FNN2 that includes several CLN convolutional layers FCL according to the network policy, and outputs intermediate features.

[0146] Note that the network policy is not limited to this, and may be modified as appropriate, for example, to determine how detailed the information used for calculations depends on the quality of the target image.

[0147] (Example of a network policy when the size of standard images varies continuously) For example, let's assume that the quality of the target image is the resolution of the input image, similar to the case where the sizes are discretely different. Also, let's assume that the number of convolutional layers FCLs (CLN) used in the calculations of the preceding group FNN2 is the reference value RFN1, given that the resolution of the input image is a predetermined reference resolution.

[0148] In this case, the policy determination unit 112 determines that the resolution of the input image becomes 1 / α relative to the reference resolution, and the preceding group FNN 2 A network policy may be decided to increase the number of convolutional layers (FCLs) used in the operation, CLN, to [RFN1*α*α]. Here, α is a real number, for example, satisfying 0 < α ≤ 1. Also, the symbol [X] is the Gaussian function (also called the floor function), representing the largest integer not exceeding the real number X in the symbol. That is, [RFN1*α*α] represents the largest integer not exceeding RFN1*α*α.

[0149] As a result, the second processing unit 113b processes the input image using a pre-stage group FNN2 that includes several CLN convolutional layers FCL according to the network policy, and outputs intermediate features. Note that the network policy is not limited to this, and the degree to which detailed information is used for calculations may be changed as appropriate, for example, depending on the quality of the target image, just as when the sizes of standard images are discretely different.

[0150] Furthermore, if the sizes of the standard images vary continuously, the FNN2 group should adjust the shape of the intermediate feature map so that the intermediate features are of a shape (i.e., size) that can be input to the RNN group.

[0151] For example, interpolation can be used as a method to adjust the shape of the intermediate feature map. For instance, suppose a convolution operation is performed using a 3x3 filter with slide S and padding PD, and the shape of the input feature map is (Hin, Win). If the shape of the feature map output by this convolution operation is (Hout, Wout), then the input feature map should be adjusted using interpolation so that it satisfies the relationships Hin = S*(Hout-1) + 3 - 2*PD and Win = S*(out-1) + 3 - 2*PD. Interpolation can be performed using near interpolation. Re Methods such as the Stoneaver method, linear interpolation, bilinear interpolation, and bicubic interpolation are suitable. Such interpolation should ideally be performed before the first convolutional layer FCL (i.e., the convolutional layer FCL closest to the input) in the preceding group FNN2.

[0152] For example, as a method for adjusting the map shape of intermediate features, d is used in the convolutional layer FCL. il The value of the ation parameter can be used. il The ation parameter is used to adjust the interval between values ​​on the map obtained by multiplying the filter used in the convolutional layer FCL.

[0153] Furthermore, these are not the only methods for adjusting the map shape of intermediate features.

[0154] (Effects / Actions) As described above, according to this embodiment, the image processing unit 113 includes a first processing unit 113a and a second processing unit 113b.

[0155] The first processing unit 113a converts the iris region included in the target image into a standardized image of a predetermined shape. The second processing unit 113b processes the standardized image using a neural network NN2 according to the network policy. The first processing unit 113a converts the iris region into a standardized image of a size according to the quality information.

[0156] This allows the shape of the input image to the neural network NN2 to be normalized to a predetermined shape, making processing by the neural network NN2 easier than when the shape of the input image is undefined.

[0157] Furthermore, since an image corresponding to the iris region can be used as the input image, the feature quantities of the iris region can be extracted with high accuracy, thereby improving the accuracy of iris recognition.

[0158] Furthermore, generally, when image resolution is improved using image super-resolution technology, the neural network processes the high-resolution image, which often increases the processing time of the neural network. In this embodiment, since the standard image is sized according to its resolution, the increase in processing time of the neural network NN2 can be suppressed compared to processing high-resolution images uniformly with the neural network NN2.

[0159] Therefore, it becomes possible to perform authentication with high accuracy regardless of the quality of the target image, while keeping the increase in processing time to a minimum.

[0160] According to this embodiment, the neural network NN2 includes multiple convolutional layers FCL and RCL. The policy determination unit 112 determines the network policy such that the number of convolutional layers FCL included in the preceding group FNN2 (CLN) increases as the resolution of the target image decreases.

[0161] This allows for the use of more convolutional layers (FCLs) when the target image quality is low, enabling calculations using more detailed information. Therefore, the neural network NN2 can process input images to enable accurate authentication regardless of the input image quality. Furthermore, this approach minimizes the increase in processing time compared to improving the quality of the input image fed into the neural network NN2 using image super-resolution techniques.

[0162] Therefore, it becomes possible to perform authentication regardless of the quality of the target image while keeping the increase in processing time to a minimum.

[0163] <Embodiment 3> Embodiment 2 describes an example in which the number of convolutional layers (CLN) is increased as the resolution of the target image decreases, thereby enabling calculations to be performed using more detailed information as the quality of the target image decreases. Embodiment 3 describes another example in which calculations are performed using more detailed information as the quality of the target image decreases, by increasing the number of channels in the calculations of the convolutional layer (FCL) as the resolution of the target image decreases. In this embodiment, in order to simplify the explanation, explanations that overlap with the embodiments described above will be omitted as appropriate.

[0164] In this embodiment, the configuration of the neural network used for processing standard images by the second processing unit 113b differs from that of Embodiment 2. Aside from this, the information processing system according to this embodiment may be configured in general the same way as the information processing system according to Embodiment 2.

[0165] Figure 14 shows an example configuration of the neural network NN3 according to Embodiment 3. The neural network NN3 is used for processing standard images performed by the second processing unit 113b according to Embodiment 3. In this embodiment as well, the explanation will use an example in which the neural network NN3 is a convolutional neural network.

[0166] The neural network NN3, similar to Embodiment 1, includes, for example, multiple layers. The multiple layers included in the neural network NN3 include a pre-stage group FNN, which includes an input layer INL. 3 It is composed of a subsequent group RNN similar to that in Embodiment 1.

[0167] The preceding group FNN3 includes an input layer INL similar to that in Embodiment 1, P convolutional layers FCL_1 to FCL_P, and an intermediate output layer MOUT, where P is an integer greater than or equal to 2.

[0168] The convolutional layers FCL_1 to FCL_P have different numbers of channels. The number of channels corresponds, for example, to the number of filters used in each of the convolutional layers FCL_1 to FCL_P. Each layer containing any one of the convolutional layers FCL_1 to FCL_P processes the input image and, for example, extracts features from the input image. In this embodiment, we will explain using an example where there is one convolutional layer FCL with the same number of channels, but the preceding group FNN3 may contain multiple convolutional layers FCL with the same number of channels.

[0169] The intermediate output layer MOUT is part of the neural network NN. 3 The processing results are output as intermediate features.

[0170] In this embodiment, since the number of channels in the convolutional layers FCL_1 to FCL_P are different, the shape of the feature maps output from each of the convolutional layers FCL_1 to FCL_P is different. Regardless of which of the convolutional layers FCL_1 to FCL_P is used, the intermediate output layer MOUT may adjust the size of the intermediate features (the shape of the feature map showing the intermediate features) so that the intermediate features can be input to the subsequent group RNN. For example, this adjustment can be done by using 1×1 convol u It would be good to use words like "tion".

[0171] In this embodiment as well, the network policy determined by the policy determination unit 112 is applied to image processing using the preceding group FNN3.

[0172] In detail, for example, the policy determination unit 112 according to this embodiment determines that the lower the resolution of the target image, the more the preceding group FNN 3 The network policy is determined to use a convolutional layer (FCL) with a large number of channels. This network policy is an example of a network policy where the lower the quality of the target image, the more detailed the information used in the preceding group FNN3 for calculations.

[0173] The information processing according to this embodiment may be configured in general the same way as the information processing according to Embodiment 1 illustrated in Figure 7. Furthermore, the details of the image processing (step S105) according to this embodiment may be configured in general the same way as the image processing (step S105) according to Embodiment 2 illustrated in Figure 11.

[0174] In other words, for example, in step S105c, the second processing unit 113b processes the standard image generated in step S105b using the neural network NN3 according to the network policy determined in step S104.

[0175] For example, suppose the sizes of the standard images are discretely different, and the quality of the target image is the resolution of the input image. Also, if the resolution of the input image is a predetermined reference resolution, suppose the number of channels in the convolutional layers FCL_1 to FCL_P used in the calculations of the preceding group FNN2 is the reference value RFN2. Here, RFN2 is an integer greater than or equal to 1.

[0176] In this case, the policy determination unit 112 determines that if the resolution of the input image becomes 1 / N of the reference resolution, the preceding group FNN 3 A network policy may be decided to increase the number of channels in the convolutional layers FCL_1 to FCL_P used in the calculations to RFN2*N*N.

[0177] As a result, the second processing unit 113b processes the input image using the pre-stage group FNN3, which includes a convolutional layer FCL with a number of channels according to the network policy, from among the convolutional layers FCL_1 to FCL_P, and outputs intermediate features. Note that the network policy is not limited to this, and may be changed as appropriate, for example, to what extent to which detailed information is used for calculations depending on the quality of the target image.

[0178] For example, if the size of the standard image is continuously different, and the resolution of the input image becomes 1 / α relative to the reference resolution, the preceding FNN group 3 A network policy may be decided to increase the number of channels in the convolutional layers FCL_1 to FCL_P used in the calculation to [RFN2*α*α]. Here, the symbol [] is the same Gaussian function as described above. That is, [RFN2*α*α] represents the largest integer not exceeding RFN2*α*α.

[0179] Furthermore, the network policy is not limited to this, and, for example, the level of detail used for calculations depending on the quality of the target image may be changed as appropriate, just as in the case where the sizes of standard images vary discretely.

[0180] Furthermore, when the sizes of the standard images differ continuously, the map shape of the intermediate features in the preceding group FNN3 should be adjusted, similar to Embodiment 2, so that the intermediate features can be input to the subsequent group RNN.

[0181] (Effects / Actions) As described above, according to this embodiment, the neural network NN3 includes multiple convolutional layers FCL_1 to FCL_P and RCL. The policy determination unit 112 determines the network policy such that the number of channels in the calculations performed by the convolutional layers FCL_1 to FCL_P included in the preceding group FNN3 increases as the resolution of the target image decreases.

[0182] Thus, the lower the quality of the target image, the more convolutional layers FCL_1 to FCL_P with a larger number of channels can be used, allowing for calculations using more detailed information. Therefore, the input image can be processed by the neural network NN3 to enable accurate authentication regardless of the quality of the input image. Furthermore, this approach minimizes the increase in processing time compared to improving the quality of the input image fed into the neural network NN3 using image super-resolution techniques.

[0183] Therefore, it becomes possible to perform authentication regardless of the quality of the target image while keeping the increase in processing time to a minimum.

[0184] <Embodiment 4> Embodiment 4 describes examples of training methods for neural networks NN1 to NN3. In this embodiment, the training method for neural network NN1 according to Embodiment 1 is used as an example. In addition, in this embodiment, in order to simplify the explanation, explanations that overlap with the embodiments described above are omitted as appropriate.

[0185] (Example of the configuration of the information processing system 400 according to Embodiment 4) Figure 15 shows an example configuration of the information processing system 400 according to Embodiment 4. The information processing system 400 includes an information processing device 403 in addition to the same imaging device 101 and information processing device 102 as in Embodiment 1.

[0186] The information processing device 403 is a device for training the neural network NN1. The information processing device 403 comprises a learning memory unit 421, a learning unit 422, and a learning policy determination unit 423.

[0187] The learning memory unit 421 is a memory unit for pre-storing learning data. The learning data includes the target image for training and the correct values. For example, when extracting features from a target image using a neural network NN1, the correct values ​​are the features of the target image for training.

[0188] The training image is simply an image used for training. Like the target image, the training image can be any image containing the target region, such as an eye image.

[0189] Furthermore, the learning memory unit 421 may pre-store learning data that includes multiple different learning target images. In this case, it is desirable that the multiple learning target images include learning target images of different qualities.

[0190] The learning unit 422 trains the neural network NN1 by inputting the training target images into the neural network NN1.

[0191] The training policy determination unit 423 determines a network policy for the training target image based on the quality information and training probability distribution of the training target image.

[0192] In detail, for example, the learning policy determination unit 423 includes a first determination unit 423a and a second determination unit 423b.

[0193] The first decision unit 423a modifies the quality information of the training target image according to the training probability distribution using random numbers, and determines the training quality information.

[0194] More specifically, the first decision unit 423a may have the same functions as the quality estimation unit 111. That is, the first decision unit 423a may estimate the position of the target region in the training target image and estimate the quality of the training target image.

[0195] Then, the first decision unit 423a modifies the estimated quality of the training target image according to a training probability distribution using random numbers, and determines the training quality information. The training probability distribution is, for example, a probability distribution in which the highest probability is given to the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases.

[0196] The second decision unit 423b determines a network policy for the training target image according to the training quality information determined by the first decision unit 423a.

[0197] The information processing device 403 may be physically configured similarly to the information processing device 102 (see Figure 6). However, the storage device 1040 of the information processing device 403 stores program modules for realizing the functions of the information processing device 403. In addition, the network interface 1050 of the information processing device 403 is an interface for connecting the information processing device 403 to a network.

[0198] (Example of operation of the information processing system 400 according to Embodiment 4) The information processing system 400 performs information processing. The information processing according to this embodiment includes learning processing in addition to processing, similar to that in Embodiment 1. The learning processing is, for example, processing for training the neural network NN1. For example, when the information processing device 403 receives instructions from the user, it starts the learning processing.

[0199] Figure 16 is a flowchart showing an example of the learning process according to Embodiment 4.

[0200] The learning policy determination unit 423 acquires the target image for learning from the learning storage unit 421 (step S401).

[0201] The learning policy determination unit 423 may acquire learning target images from other information processing devices (not shown) via a network or the like, or it may acquire learning target images via a storage medium.

[0202] The training policy determination unit 423 determines a network policy for the training target images acquired in step S401 based on the quality information of the training target images and the training probability distribution (step S402).

[0203] In detail, for example, the first decision unit 423a modifies the quality information of the training target image according to the training probability distribution and determines the training quality information (step S402a).

[0204] More specifically, for example, the first decision unit 423a estimates the quality of the training target image. The quality of the training target image may be pre-stored in the training storage unit 421 in association with the training target image. In this case, the first decision unit 423a does not need to estimate the quality of the training target image.

[0205] The first decision unit 423a modifies the estimated quality of the training target image according to the training probability distribution and determines the training quality information.

[0206] Figure 17 shows an example of a learning probability distribution according to Embodiment 4. The horizontal axis of the figure represents quality Q, and the vertical axis represents the probability of occurrence P1.

[0207] In the figure, Q1 represents the quality indicated by the quality information of the training target image. The training probability distribution exemplified in the figure is such that quality Q1, indicated by the quality information of the training target image, appears with the highest probability, and the probability of appearance gradually decreases as the degree of difference from this quality increases. Furthermore, the training probability distribution shown in the figure is an example of a symmetrical probability distribution through the quality Q1 of the training target image.

[0208] The first decision unit 423a modifies the quality Q1 of the training target image by generating random numbers according to such a training probability distribution. Q2 in the figure is, for example, an example of training quality obtained based on the quality Q1 of the training target image and random numbers generated according to the training probability distribution. In this case, the first decision unit 423a modifies the quality Q1 of the training target image and determines training quality information including training quality Q2.

[0209] Note that the training probability distribution is not limited to the probability distribution exemplified in Figure 17. For example, generally, the lower the quality of the target image, the more difficult it is to perform accurate authentication using that image. Therefore, the training probability distribution may be one in which a low quality image (lower than the training target image quality Q1) has a higher probability than a high quality image (higher than the training target image quality Q1). Also, for example, the training probability distribution may be such that high quality is higher than quality Q1. low The probability distribution may be one that is more likely to result in a low-quality image. Furthermore, the learning policy determination unit 423 may determine multiple network policies for a single training target image.

[0210] Refer to Figure 16 again. The second decision unit 423b determines a network policy for the training target image according to the training quality information determined in step S402a (step S402b).

[0211] In detail, for example, the second decision unit 423b may determine the network policy for the target images for training based on the training quality information determined in step S402a.

[0212] The learning unit 422 inputs the training target image acquired in step S401 into the neural network NN1, which corresponds to the network policy determined in step S402b. As a result, the learning unit 422 trains the neural network NN1 (step S403) and terminates the training process.

[0213] In the learning unit 422, general machine learning techniques should be used for training. For example, the learning unit 422 should estimate the features of the training image using a neural network NN1 according to the network policy. Then, the learning unit 422 should find a loss function to calculate the error (loss) between the estimated features and the ground truth values ​​of the training image. For the loss, for example, cross-entropy (error), mean squared error, or mean absolute error (MAE) should be used. Based on the loss function, the learning unit 422 should calculate the gradient and update the parameters included in the neural network NN1.

[0214] The learning policy determination unit 423 may acquire multiple training target images in step S401. In this case, the learning policy determination unit 423 and the learning unit 422 may execute steps S402 and S403 for each of the training target images.

[0215] (Effects / Actions) As described above, according to this embodiment, the information processing system 400 further includes a learning unit 422 that performs training on the neural network NN1 by inputting training target images to the neural network NN1.

[0216] This allows us to train the neural network NN1.

[0217] According to this embodiment, the information processing system 400 further includes a learning policy determination unit 423 that determines a network policy for the training target image based on quality information and a learning probability distribution of the training target image. The learning unit 422 learns the neural network NN1 by inputting the training target image to the neural network NN1 according to the determined network policy. The learning probability distribution is a probability distribution in which the probability is highest for the quality indicated by the quality information of the training target image, and gradually decreases as the degree of difference from that quality increases.

[0218] Generally, the quality estimation unit 111 may incorrectly estimate the quality of the target image. According to this embodiment, based on the training target image, it is possible to train the neural network NN1 corresponding to the training quality, which mainly includes the quality of the training target image and the quality of its surroundings. This makes it possible to train the neural network NN1 for image processing (for example, feature extraction using the neural network NN1) that can perform authentication with high accuracy even if the quality of the target image is incorrectly estimated. Therefore, it becomes possible to perform authentication with high accuracy regardless of the quality of the target image.

[0219] According to this embodiment, the learning policy determination unit 423 includes a first determination unit 423a and a second determination unit 423b. The first determination unit 423a modifies the quality information of the target images for training according to a learning probability distribution using random numbers and determines the learning quality information. The second determination unit 423b determines a network policy for the target images for training according to the determined learning quality information.

[0220] This allows the neural network NN1 to be trained based on the target image used for training, primarily by considering the quality of the target image and its surroundings. As a result, as described above, it becomes possible to perform authentication with high accuracy regardless of the quality of the target image.

[0221] <Embodiment 5> Embodiment 5 describes other examples of training methods for neural networks NN1 to NN3. In this embodiment, the training method for neural network NN1 according to Embodiment 1 is described as an example. Also, in this embodiment, in order to simplify the explanation, explanations that overlap with the embodiments described above are omitted as appropriate.

[0222] (Example of the configuration of the information processing system 500 according to Embodiment 5) Figure 18 shows an example configuration of the information processing system 500 according to Embodiment 5. The information processing system 500 includes an imaging device 101 and an information processing device 102 similar to those in Embodiment 1, as well as an information processing device 503.

[0223] The information processing device 503 is a device for training the neural network NN1. The information processing device 503 comprises a learning unit 422, a learning storage unit 521, an image selection unit 524, and a learning policy determination unit 525, which are generally the same as those in Embodiment 4.

[0224] The learning memory unit 521 is a memory unit for pre-storing learning data. In this embodiment, the learning data may be configured to associate multiple learning target images with their respective correct values. The multiple learning target images may include learning target images of different qualities. In this embodiment, the learning data may be further configured to associate the quality of each learning target image.

[0225] The learning unit 422 inputs the training target images selected by the image selection unit 524 (details described later) into the neural network NN1, thereby training the neural network NN1.

[0226] The image selection unit 524 selects training images such that lower-quality training images are selected more frequently. For example, the image selection unit 524 selects training images such that low-resolution training images are selected more frequently than high-resolution training images.

[0227] The learning policy determination unit 525 determines the network policy for the training target images selected by the image selection unit 524 based on the quality information of the training target images selected by the image selection unit 524. The learning policy determination unit 525 may determine the network policy for the training target images selected by the image selection unit 524 using a process similar to the process used by the policy determination unit 112 to determine the network policy.

[0228] The information processing device 503 may be physically configured in the same way as the information processing device 102 (see Figure 6). However, the storage device 1040 of the information processing device 503 stores program modules for realizing the functions of the information processing device 503. Also, the network interface 1050 of the information processing device 503 is an interface for connecting the information processing device 503 to a network.

[0229] (Example of operation of the information processing system 500 according to Embodiment 5) The information processing system 500 performs information processing. The information processing according to this embodiment includes learning processing in addition to processing, similar to that in Embodiment 1. The learning processing is, for example, a process for training the neural network NN1, similar to that in Embodiment 4. For example, when the information processing device 503 receives instructions from the user, it starts the learning processing.

[0230] Figure 19 is a flowchart showing an example of the learning process according to Embodiment 5.

[0231] The image selection unit 524 is a learning memory unit. 5 From the training images stored in 21, training images are selected such that more training images of lower quality are selected. (Step S501).

[0232] In detail, for example, the image selection unit 524 may determine the quality according to a selection probability distribution. Quality is, for example, the resolution of the image.

[0233] Figure 20 shows an example of a selection probability distribution according to Embodiment 5. The horizontal axis of the figure represents quality Q. The vertical axis of the figure represents the probability of occurrence P2. The selection probability distribution is a probability distribution in which lower quality has a higher probability within a predetermined range of quality (in Figure 20, the range of Q3 to Q4). Such a selection probability distribution may be obtained, for example, by multiplying the quality (e.g., resolution) by a predetermined weight.

[0234] Then, the image selection unit 524 is a learning memory unit5 From the training images stored in 21, it is recommended to select training images associated with the desired quality.

[0235] If the training data does not include quality information, the image selection unit 524 may, for example, estimate the location of the target region in the training image and estimate the quality of the training image, similar to the quality estimation unit 111. In this case, the image selection unit 524 may, for example, select a training image for which the same quality as the quality obtained according to the selection probability distribution has been estimated.

[0236] Refer to Figure 19 again. The image selection unit 524 stores the target images for learning selected in step S501, for example, in the learning memory unit. 5 Obtain from 21 (step S502).

[0237] The learning policy determination unit 525 determines the network policy for the learning target images acquired in step S502 based on the quality information of the learning target images acquired in step S502 (step S503).

[0238] In detail, for example, the learning policy determination unit 525 may acquire the quality information obtained in step S501 from the image selection unit 524. The information processing device 503 may also have the same functions as the quality estimation unit 111, and may use these functions to estimate the quality information of the training target image acquired in step S502.

[0239] The learning unit 422 performs training on the neural network NN1, similar to Embodiment 4 (step S403), and then terminates the training process.

[0240] In step S403 of this embodiment, the learning unit 422 performs step S50 3Input the learning target image acquired in step S502 into the neural network NN1 according to the network policy determined by .

[0241] (Function and effect) As described above, according to this embodiment, the information processing system 500 further includes an image selection unit 524 that selects learning target images so that more low-quality learning target images are selected. The learning unit 422 inputs the selected learning target images into the neural network NN1 to perform learning of the neural network NN1.

[0242] Generally, it is often difficult to authenticate accurately using a target image as the quality of the target image is lower. According to this embodiment, since more low-quality learning target images are selected, the neural network NN1 corresponding to lower quality can perform more learning. As a result, learning for image processing (for example, extraction of feature amounts using the neural network NN1) that can perform accurate authentication even when the quality of the target image is low can be performed. Therefore, it becomes possible to perform accurate authentication regardless of the quality of the target image.

[0243] <Embodiment 6> The learning methods described in Embodiments 4 and 5 may be combined. In Embodiment 6, as another example of the learning methods of the neural networks NN1 to NN3, an example of combining the learning methods described in Embodiments 4 and 5 will be described. In this embodiment, the learning method of the neural network NN1 according to Embodiment 1 will be described as an example. Also, in this embodiment, for the sake of simplicity, descriptions overlapping with the above-described embodiments will be omitted as appropriate.

[0244] (Configuration example of the information processing system 600 according to Embodiment 6) FIG. 21 is a diagram showing a configuration example of an information processing system 600 according to Embodiment 6. The information processing system 600 includes an information processing device 603 in addition to the imaging device 101 and the information processing device 102 similar to those in Embodiment 1.

[0245] The information processing device 603 is a device for training a neural network NN1. The information processing device 603 includes a learning unit 422 and a learning policy determination unit 423 that are generally similar to those in Embodiment 4, and a learning storage unit 521 and an image selection unit 524 that are generally similar to those in Embodiment 5.

[0246] The learning policy determination unit 423 according to the present embodiment determines a network policy for the learning target image based on the quality information of the learning target image selected by the image selection unit 524 and the learning probability distribution.

[0247] The learning policy determination unit 423 may include a first determination unit 423a and a second determination unit 423b as described in Embodiment 4. That is, the first determination unit 423a may change the quality information of the learning target image selected by the image selection unit 524 according to the learning probability distribution by using a random number, and determine the learning quality information. The second determination unit 423b may determine a network policy for the learning target image according to the learning quality information determined by the first determination unit 423a.

[0248] Physically, the information processing device 603 may be configured similarly to the information processing device 102 (see FIG. 6). However, the storage device 1040 of the information processing device 603 stores program modules for realizing the functions of the information processing device 603. Also, the network interface 1050 of the information processing device 603 is an interface for connecting the information processing device 603 to a network.

[0249] (Operation example of the information processing system 600 according to Embodiment 6) The information processing system 600 performs information processing. The information processing according to this embodiment includes learning processing in addition to processing, similar to that in Embodiment 1. The learning processing is, similar to that in Embodiment 4, for example, processing for training the neural network NN1. For example, when the information processing device 603 receives instructions from the user, it starts the learning processing.

[0250] Figure 22 is a flowchart showing an example of the learning process according to Embodiment 6.

[0251] The image selection unit 524 performs steps S501 and S502, similar to those in Embodiment 5.

[0252] The learning policy determination unit 423 performs a learning policy determination process (step S402) that is generally the same as in Embodiment 4.

[0253] In step S402 of this embodiment, the target image for learning to be processed is, for example, the target image for learning acquired in step S502. That is, in step S402 of this embodiment, the learning policy determination unit 423 determines a network policy for the target image for learning acquired in step S502.

[0254] The quality information of the training target image used in step S402 according to this embodiment may be the quality information obtained in step S501. In this case, the training policy determination unit 423 may obtain the quality information of the training target image from the image selection unit 524. The information processing device 603 may also have the same functions as the quality estimation unit 111, and may use these functions to estimate the quality information of the training target image obtained in step S502. In this case, the estimated quality information may be used as the quality of the training target image in the processing in step S402.

[0255] The learning unit 422 performs the process of step S403, which is generally the same as in Embodiment 4, and then terminates the learning process.

[0256] In detail, for example, the learning unit 422 may input the training target image acquired in step S502 into the neural network NN1 according to the network policy determined in step S402b. The learning unit 422 then performs training on the neural network NN1.

[0257] (Effects / Actions) As described above, according to this embodiment, the information processing system 600 comprises an image selection unit 524, a learning policy determination unit 423, and a learning unit 422.

[0258] The image selection unit 524 selects training images such that lower-quality training images are selected more frequently.

[0259] The training policy determination unit 423 determines a network policy for the training target image based on the quality information of the training target image and the training probability distribution. The training probability distribution is a probability distribution in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases.

[0260] The learning unit 422 trains the neural network NN1 by inputting the selected training target images to the neural network NN1 according to the determined network policy.

[0261] As a result, as described in Embodiment 4, it is possible to train for image processing (e.g., feature extraction using a neural network NN1) that can perform authentication with high accuracy even when the quality of the target image is incorrectly estimated. Furthermore, as described in Embodiment 5, it is possible to train for image processing (e.g., feature extraction using a neural network NN1) that can perform authentication with high accuracy even when the quality of the target image is low. Therefore, it becomes possible to perform authentication with even higher accuracy regardless of the quality of the target image.

[0262] The embodiments and modifications of the present invention have been described above with reference to the drawings. However, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0263] Also, in the plurality of flowcharts used in the above description, a plurality of steps (processes) are described in order. However, the execution order of the steps executed in each of the embodiments is not limited to the order described. In each of the embodiments, the order of the illustrated steps can be changed within a range that does not substantially affect the content. Also, the above-described embodiments and modifications can be combined within a range where the content does not conflict.

[0264] Some or all of the above embodiments can also be described as follows in the appended claims, but are not limited thereto.

[0265] 1. A quality estimation means for estimating quality information of a target image, and an image processing means for processing the target image using a neural network according to a network policy determined based on the quality information of the target image. An information processing system. 2. The information processing system according to 1., wherein the image processing means extracts feature amounts from the target image using the neural network according to the network policy. The information processing system according to 1. 3. The target image is an eye image including an iris region where the iris is reflected, and the quality information includes an iris diameter. The information processing system according to 1. or 2. 4. The quality information includes the resolution of the target image. The information processing system according to any one of 1. to 3. 5. The information processing system according to any one of 1. to 4. further includes a policy determination means for determining the network policy based on the quality information. The information processing system according to any one of 1. to 4. 6. The policy determination means determines the network policy such that the processing time using the neural network is within a predetermined range, regardless of the quality information of the target image. The information processing system described in 5. 7. The image processing means is: A first processing means that converts the iris region included in the target image into a standard image of a predetermined shape, The system includes a second processing means that processes the standard image using the neural network according to the network policy, The first processing means converts the iris region into a standard image of a size corresponding to the quality information. An information processing system described in any one of items 1 through 6. 8. The neural network is composed of a pre-stage group including an input layer and a post-stage group including a final layer. The aforementioned network policy is applied to image processing using the aforementioned group. An information processing system described in any one of items 1 through 7. 9. The neural network includes multiple convolutional layers, The policy determination means determines the network policy such that the lower the resolution of the target image, the larger the number of convolutional layers included in the preceding group. The information processing system described in 8. 10. The neural network includes multiple convolutional layers, The policy determination means determines the network policy such that the lower the resolution of the target image, the larger the number of channels in the calculations performed in the convolutional layer included in the preceding group. The information processing system described in 8. 11. The system further comprises a learning means for training the neural network by inputting target images for training into the neural network. An information processing system described in any one of items 1 through 10. 12. The system further comprises a training policy determination means for determining a network policy for the training target image based on the quality information and training probability distribution of the training target image, The learning means performs training on the neural network by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. The information processing system described in 11. 13. The learning policy determination means is: A first determination means that modifies the quality information of the target image for training according to the training probability distribution by using the aforementioned random numbers, and determines the training quality information, Includes a second determination means for determining a network policy for the target images for training based on the determined training quality information. The information processing system described in 12. 14. The system further provides image selection means for selecting training images so that lower-quality training images are selected more frequently. The learning means performs training on the neural network by inputting the selected training target images into the neural network. An information processing system described in any one of items 11 through 13. 15. A quality estimation means for estimating the quality information of the target image, The system comprises an image processing means that processes the target image using a neural network that corresponds to a network policy determined based on the quality information of the target image. Information processing device. 16. One or more computers, Estimate the quality information of the target image, This includes processing the target image using a neural network that corresponds to a network policy determined based on the quality information of the target image. Information processing methods. 17. On one or more computers, Estimate the quality information of the target image, A recording medium on which a program is stored that causes a neural network to process the target image using a network policy determined based on the quality information of the target image. [Explanation of Symbols]

[0266] 100, 400, 500, 600 Information Processing Systems 101 Imaging device 102,403,503,603 Information Processing Equipment 111 Quality estimation section 112 Policy Decision-Making Department 113 Image Processing Unit 113a First Processing Unit 113b Second Processing Unit 114 Authentication Department 421,521 Learning memory section 422 Learning Department 423 Learning Policy Determination Unit 423a 1st decision section 423b Second decision section 524 Image Selection Section 525 Learning Policy Decision Unit NN1~NN3 Neural Networks

Claims

1. A quality estimation means for estimating the quality information of the target image, Image processing means that processes the target image using a neural network according to a network policy determined based on the quality information of the target image, A learning means that trains the neural network by inputting training target images into the neural network, The system includes a training policy determination means that determines a network policy for the training target image based on the quality information and training probability distribution of the training target image, The learning means performs training on the neural network by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. Information processing system.

2. The image processing means extracts features from the target image using the neural network according to the network policy. The information processing system according to claim 1.

3. The aforementioned target image is an eye image that includes the iris region in which the iris is reflected. The aforementioned quality information includes the iris diameter. The information processing system according to claim 1 or 2.

4. The quality information includes the resolution of the target image. The information processing system according to claim 1 or 2.

5. The system further comprises a policy determination means for determining the network policy based on the aforementioned quality information, The policy determination means determines the network policy such that the processing time using the neural network falls within a predetermined range, regardless of the quality information of the target image. The information processing system according to claim 1 or 2.

6. The aforementioned image processing means is A first processing means that converts the iris region included in the target image into a standard image of a predetermined shape, The system includes a second processing means that processes the standard image using the neural network according to the network policy, The first processing means converts the iris region into a standard image of a size corresponding to the quality information. The information processing system according to claim 3.

7. The aforementioned neural network consists of a pre-stage group including an input layer and a post-stage group including a final layer. The aforementioned network policy is applied to image processing using the aforementioned group. The information processing system according to claim 1 or 2.

8. The learning policy determination means is A first determination means that modifies the quality information of the target image for training according to the training probability distribution by using the aforementioned random numbers, and determines the training quality information, Includes a second determination means for determining a network policy for the target images to be trained, in accordance with the determined training quality information. The information processing system according to claim 1 or 2.

9. One or more computers, Estimate the quality information of the target image, The target image is processed using a neural network that corresponds to a network policy determined based on the quality information of the target image. By inputting the training target images into the neural network, the neural network is trained. This includes determining a network policy for the training target image based on the quality information and training probability distribution of the training target image, In training the neural network, the training of the neural network is performed by inputting the target images for training into the neural network according to the determined network policy. The aforementioned training probability distribution is one in which the highest probability is given for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases. Information processing methods.

10. On one or more computers, Estimate the quality information of the target image, The target image is processed using a neural network that corresponds to a network policy determined based on the quality information of the target image. By inputting the training target images into the neural network, the neural network is trained. This includes determining a network policy for the training target image based on the quality information and training probability distribution of the training target image, In training the neural network, the training of the neural network is performed by inputting the target images for training into the neural network according to the determined network policy. The program is characterized in that the training probability distribution is highest for the quality indicated by the quality information of the training target image, and the probability gradually decreases as the degree of difference from that quality increases.

Images