Information Processing System, Information Processing Method, and Program

Abstract

An information processing system (100) comprises a quality estimation unit (111) and an image processing unit (113). The quality estimation unit (111) estimates quality information of an image of interest. The image processing unit (113) processes the image of interest by using a neural network corresponding to a network policy which is determined on the basis of quality information of the image of interest.

Description

Information processing system, information processing device, information processing method, and recording medium

[0001] The present disclosure relates to an information processing system, an information processing device, an information processing method, and a recording medium.

[0002] In iris authentication, authentication accuracy generally decreases when the quality of the image, such as resolution, is low. Therefore, in iris authentication, it is desirable to use a high-resolution image capture device to capture an image of the authentication target, but it is sometimes difficult to use a high-resolution image capture device due to installation space, cost, etc.

[0003] For example, Patent Document 1 discloses that when iris authentication is performed, a predetermined number of captured images are used to generate an iris authentication image with a second resolution higher than the first resolution of the captured images. As an example, Patent Document 1 discloses that, from eight consecutive captured images, 1×1 pixels of the captured images are converted to 2×2 pixels to generate an authentication image with double the resolution.

[0004] JP 2009-282925 A

[0005] The present disclosure aims to improve upon the techniques described in the prior art documents mentioned above.

[0006] According to one aspect of the present invention, there is provided an information processing system comprising: a quality estimation means for estimating quality information of a target image; and an image processing means for processing the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0007] According to one aspect of the present invention, there is provided an information processing device comprising: a quality estimation means for estimating quality information of a target image; and an image processing means for processing the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0008] According to one aspect of the present invention, there is provided an information processing method including: one or more computers estimating quality information of a target image; and processing the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0009] According to one aspect of the present invention, a recording medium is provided having recorded thereon a program for causing one or more computers to estimate quality information of a target image and process the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0010] 1 is a diagram illustrating an overview of an information processing system according to embodiment 1. FIG. 2 is a diagram illustrating an overview of an information processing device according to embodiment 1. FIG. 3 is a flowchart illustrating an overview of information processing according to embodiment 1. FIG. 4 is a diagram illustrating an example of the configuration of an information processing system according to embodiment 1. FIG. 5 is a diagram illustrating an example of the configuration of a neural network according to embodiment 1. FIG. 6 is a diagram illustrating an example of the physical configuration of an information processing device according to embodiment 1. FIG. 7 is a flowchart illustrating an example of information processing according to embodiment 1. FIG. 8 is a diagram illustrating an example of a target image according to embodiment 1. FIG. 9 is a diagram illustrating an example of the functional configuration of an image processing unit according to embodiment 2. FIG. 10 is a diagram illustrating an example of the configuration of a neural network according to embodiment 2. FIG. 11 is a flowchart illustrating a detailed example of image processing according to embodiment 2. FIG. 12 is a diagram illustrating an example of an iris region according to embodiment 2. FIG. 13 is a diagram illustrating an example of a standard image according to embodiment 2. FIG. 14 is a diagram illustrating an example of the configuration of a neural network according to embodiment 3. FIG. 15 is a diagram illustrating an example of the configuration of an information processing system according to embodiment 4. FIG. 16 is a flowchart illustrating an example of a learning process according to embodiment 4. FIG. 17 is a diagram illustrating an example of a learning probability distribution according to embodiment 4. FIG. 18 is a diagram illustrating an example of a configuration of an information processing system according to embodiment 5. FIG. 19 is a flowchart illustrating an example of a learning process according to embodiment 5. FIG. 19 is a diagram illustrating an example of a selection probability distribution according to embodiment 5. FIG. 19 is a diagram illustrating an example of the configuration of an information processing system according to embodiment 6. FIG. 19 is a flowchart illustrating an example of a learning process according to embodiment 6. FIG. 19 is a diagram illustrating an example of the configuration of an information processing system according to 10 is a flowchart showing an example of a network policy determination process according to a seventh embodiment. FIG. 11 is a diagram showing an example of a configuration of an information processing system according to an eighth embodiment. FIG. 12 is a diagram showing an example of a learning process according to the eighth embodiment. FIG. 13 is a diagram showing an example of a configuration of a learning policy determination unit according to a ninth embodiment. FIG. 14 is a flowchart showing an example of a first policy determination model learning process according to the ninth embodiment. FIG. 15 is a diagram showing an example of a configuration of an information processing system according to a tenth embodiment. FIG. 16 is a flowchart showing an example of information processing according to the tenth embodiment. FIG. 17 is a flowchart showing an example of a network policy determination process according to the tenth embodiment. FIG. 18 is a diagram showing an example of a configuration of an information processing system according to an eleventh embodiment. FIG. 19 is a diagram showing an example of the functional configuration of a learning policy determination unit according to the eleventh embodiment.13 is a flowchart showing an example of a second policy determination model learning process according to an eleventh embodiment. FIG. 14 is a diagram showing an example of a configuration of an information processing system according to a twelfth embodiment. FIG. 15 is a flowchart showing an example of information processing according to the twelfth embodiment. FIG. 16 is a flowchart showing an example of a network policy determination process according to the twelfth embodiment. FIG. 17 is a diagram showing an example of a configuration of an information processing system according to a thirteenth embodiment. FIG. 18 is a diagram showing an example of a functional configuration of a learning policy determination unit according to the thirteenth embodiment. FIG. 19 is a flowchart showing an example of a third policy determination model learning process according to the thirteenth embodiment.

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0012] 1 is a diagram showing an overview of an information processing system 100 according to embodiment 1. The information processing system 100 includes a quality estimation unit 111 and an image processing unit 113.

[0013] The quality estimation unit 111 estimates quality information of the target image. The image processing unit 113 processes the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0014] According to this information processing system 100, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0015] 2 is a diagram showing an overview of the information processing apparatus 102 according to embodiment 1. The information processing apparatus 102 includes a quality estimation unit 111 and an image processing unit 113.

[0016] The quality estimation unit 111 estimates quality information of the target image. The image processing unit 113 processes the target image using a neural network according to a network policy determined based on the quality information of the target image.

[0017] According to this information processing device 102, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0018] FIG. 3 is a flowchart showing an outline of information processing according to the first embodiment.

[0019] The quality estimation unit 111 estimates quality information of the target image (step S103).

[0020] The image processing unit 113 processes the target image using a neural network according to a network policy determined based on the quality information of the target image (step S105).

[0021] This information processing makes it possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0022] A detailed example of the information processing system 100 according to the first embodiment will be described below.

[0023] (Details) The technology described in the above-mentioned Patent Document 1 uses so-called image super-resolution technology to improve resolution. However, image super-resolution has a problem in that it takes a long time to perform the processing.

[0024] In view of these circumstances, one example of the objective of this disclosure is to provide an information processing system, an information processing device, an information processing method, and a recording medium that solve the problem of performing authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0025] (Configuration example of information processing system 100 according to embodiment 1) Fig. 4 is a diagram showing a configuration example of the information processing system 100 according to embodiment 1. The information processing system 100 is a system for performing processing using a target image obtained by capturing a target. In this embodiment, an example will be described in which the processing using the target image is authentication processing for authenticating the target.

[0026] The information processing system 100 includes an image capturing device 101 and an information processing device 102 .

[0027] The image capturing apparatus 101 and the information processing apparatus 102 are preferably connected via a network configured by wired or wireless means or a combination of these means, and may transmit and receive information to and from each other via the network.

[0028] The target according to this embodiment is a human, but the target is not limited to this and may be an animal such as a dog or a snake.

[0029] In addition, in the authentication process according to this embodiment, iris authentication is performed. In this embodiment, an example will be described in which iris authentication is performed using an image of the right iris. However, the authentication process is not limited to this, and any authentication process using an image may be used, such as iris authentication using an image of the left iris or both irises, face authentication using a face image showing the entire face, or vein authentication using an image showing veins.

[0030] The image capturing device 101 captures an image of a target and generates an image of the target.

[0031] The target image is an image that includes a target area used in the authentication process. In this embodiment, as described above, iris authentication is performed using an image, so the target area is the iris area where the iris is captured. More specifically, in this embodiment, iris authentication is performed using an image of the right eye iris, so the target area is the iris area where the right eye iris is captured. Note that iris authentication may also be performed using an image of the left eye iris, and in that case the target area may be the iris area where the right eye iris is captured.

[0032] (Example of Functional Configuration of Information Processing Apparatus 102 According to First Embodiment) The information processing apparatus 102 according to this embodiment uses the target image generated by the image capturing apparatus 101 to authenticate the target shown in the target image.

[0033] In this embodiment, as will be described later, an example will be described in which the information processing device 102 is provided with an authentication unit 114 that performs processing to authenticate an object (authentication processing), but the information processing device 102 only needs to perform processing to authenticate an object using an object image, and the authentication processing may be performed by, for example, another information processing device (not shown). In this case, it is preferable that the other information processing device is provided with the authentication unit 114.

[0034] As shown in FIG. 4, the information processing apparatus 102 functionally includes, for example, a quality estimation unit 111, a policy determination unit 112, an image processing unit 113, and an authentication unit 114.

[0035] The quality estimation unit 111 estimates quality information of the target image.

[0036] The quality information is information indicating the quality of the target image. The quality according to this embodiment is the resolution of the target image, for example, the resolution of the target area included in the target image. In detail, the quality according to this embodiment is the resolution of the iris area in which the iris of the right eye used in the authentication process is reflected. When the quality is resolution, the smaller the resolution, the lower the quality.

[0037] The quality estimation unit 111 may further estimate the position of the target area in the target image. The target area position is information for identifying the position of the target area in the target image. In this embodiment, the target area is the iris area described above, so the target area position in this embodiment is the position of the iris of the right eye in the target image. Hereinafter, the position of the iris of the right eye in the target image will also be referred to as the "eye position."

[0038] The eye position includes, for example, the position of the pupil center, pupil diameter, and iris diameter. The pupil diameter is the diameter or radius of the pupil. The iris diameter is the diameter or radius of the outer edge of the iris. By using such eye positions, the position of the right eye iris in the target image can be identified.

[0039] The quality indicated by the quality information is not limited to the resolution of the target image, but may be one or more of, for example, the resolution of the target image, the iris diameter of the iris reflected in the target image, the estimated distance between the image capturing device 101 and the target, the presence or absence or degree of blur, etc. This estimated distance may be estimated from information used for focus adjustment in the image capturing device 101, or may be estimated based on information from a distance measuring sensor (not shown) for estimating the distance to the target. Furthermore, the resolution of the target image is not limited to the target region, and may be, for example, the resolution of a predetermined region included in the target image.

[0040] Furthermore, the quality information may be expressed by, for example, a vector quantity. When the quality information is expressed by a plurality of parameters, the plurality of parameters can be treated as a single variable by expressing the quality information using a vector quantity.

[0041] The policy determination unit 112 determines a network policy based on the quality information estimated by the quality estimation unit 111. This network policy is applied to a neural network that an image processing unit 113 (described later) uses for image processing.

[0042] The policy determination unit 112 may determine a network policy that satisfies at least one of the following (A) and (B), regardless of the quality information of the target image.

[0043] (A) The processing time using the neural network must be within a predetermined range, and (B) the accuracy of authentication using the target image must be within a predetermined range.

[0044] The image processing unit 113 processes the target image using a neural network according to the network policy determined by the policy determination unit 112. In processing the target image, the image processing unit 113 according to this embodiment extracts, for example, feature amounts from the target image.

[0045] Here, the image processing unit 113 may process the target image using a neural network that uses more detailed information for calculations, depending on the network policy, for example, as the quality of the target image (i.e., the quality indicated by the quality information) becomes lower. In other words, the policy determination unit 112 determines a network policy that uses more detailed information for calculations as the quality of the target image becomes lower.

[0046] 5 is a diagram showing an example of the configuration of the neural network NN1 according to the first embodiment. The neural network NN1 is used for processing the target image performed by the image processing unit 113. Here, an example will be described in which the neural network NN1 is a convolutional neural network. Note that the neural network NN1 is not limited to this, and a vision transformer, for example, may also be used.

[0047] The neural network NN1 includes, for example, a plurality of layers, which are configured from a front-stage group FNN1 including an input layer INL and a rear-stage group RNN including a final layer OUTL.

[0048] The number of layers included in each of the front group FNN1 and the rear group RNN may be changed as appropriate, but for example, the number of layers included in the front group FNN1 may be greater than the number of layers included in the rear group RNN. In other words, the front group FNN1 may be closer to the input side than the central layer of the multiple layers that make up the neural network NN1.

[0049] The front-stage group FNN1 includes, for example, an input layer INL, at least one convolutional layer FCL, and an intermediate output layer MOUT. Note that the front-stage group FNN1 may include a batch norm layer, an activation function, and the like (not shown), similar to a general convolutional neural network.

[0050] The input layer INL acquires an input image to the neural network NN1 and propagates the input image to, for example, a subsequent layer. The input image is a target image. The image processing unit 113 may extract an image included in the target image (e.g., an image of the target region) and use the extracted image as the input image. In other words, the input image may be any image obtained from the target image.

[0051] Each layer, including the convolutional layer FCL, processes an input image and extracts, for example, features of the input image. The intermediate output layer MOUT outputs the processing results of the neural network NN1 as intermediate features. Note that the intermediate output layer MOUT may be a convolutional layer.

[0052] Here, the network policy determined by the policy determination unit 112 is applied to image processing using the preceding group FNN 1. In other words, the policy determination unit 112 determines the network policy to be applied to the preceding group FNN 1. In detail, for example, the lower the quality of the target image, the more detailed information the policy determination unit 112 determines the network policy to perform calculations using.

[0053] In this case, the policy determination unit 112 may determine a network policy that satisfies at least one of the above-mentioned (A) and (B). For example, the policy determination unit 112 may determine a network policy that corresponds to the quality of the target image by previously storing policy information that defines a network policy according to the quality of the target image and acquiring the network policy according to the quality of the target image from the policy information. In the policy information, a network policy that causes the preceding group FNN1 to perform calculations using detailed information according to the quality of the target image may be previously determined so as to satisfy, for example, at least one of the above-mentioned (A) and (B).

[0054] The latter group RNN includes, for example, an intermediate input layer MIN, at least one convolutional layer RCL, and a final layer (output layer) OUTL. Note that the latter group RNN may include a batch norm layer, an activation function, and the like (not shown), similar to a general convolutional neural network.

[0055] The intermediate input layer MIN acquires intermediate features from the previous group FNN1. Each layer, including the convolution layer RCL, further processes the intermediate features, for example, to extract features of the input image. Note that the intermediate input layer MIN may be a convolution layer.

[0056] The final layer OUTL outputs the final processing result of the neural network NN1. The final layer OUTL outputs, for example, feature quantities based on the results of further processing in each layer including the convolution layer RCL.

[0057] In this way, the lower the quality of the target image, the more detailed information is used for calculations, making it possible to suppress a decrease in the accuracy of authentication performed in the authentication process even with low-quality target images. On the other hand, because calculations are performed using detailed information, the processing time using the neural network increases for low-quality target images compared to high-quality target images.

[0058] As described above, for example, the policy determination unit 112 may determine a network policy (A) so that the processing time using a neural network falls within a predetermined range, regardless of the quality information of the target image (i.e., the quality indicated by the quality information). This prevents an increase in processing time (the processing time using a neural network) when the target image is of low quality, and makes it possible to keep the processing time using a neural network at a similar level regardless of the quality of the target image.

[0059] Furthermore, for example, the policy determination unit 112 may determine a network policy (B) so that the accuracy of authentication using the target image falls within a predetermined range, regardless of the quality information of the target image (i.e., the quality indicated by the quality information). This prevents a decrease in authentication accuracy when the target image is of low quality, and makes it possible to maintain the same level of accuracy in authentication performed in the authentication process regardless of the quality of the target image.

[0060] The authentication unit 114 performs authentication processing using the result of processing the target image by the image processing unit 113. The authentication processing is processing for authenticating the target using the result of processing the target image.

[0061] For example, when the authentication unit 114 acquires the feature extracted by the image processing unit 113, it authenticates the target based on whether or not the acquired feature is registered in enrollment data. Although not shown, the enrollment data may be stored in advance by the authentication unit 114.

[0062] Up to now, the description has been mainly given of an example of the functional configuration of the information processing system 100. From here, an example of the physical configuration of the information processing system 100 will be described.

[0063] (Physical Configuration of Information Processing System 100) The information processing system 100 according to this embodiment is physically configured to include, for example, an imaging device 101 and an information processing device 102. The imaging device 101 is, for example, a camera such as a near-infrared camera.

[0064] 6 is a diagram showing an example of the physical configuration of the information processing apparatus 102 according to embodiment 1. The information processing apparatus 102 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, a network interface 1050, an input interface 1060, and an output interface 1070.

[0065] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, network interface 1050, input interface 1060, and output interface 1070. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0066] The processor 1020 is implemented as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit).

[0067] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0068] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules for realizing the functions of the information processing device 102. The processor 1020 reads each of these program modules into the memory 1030 and executes them to realize the function corresponding to the program module.

[0069] The network interface 1050 is an interface for connecting the information processing device 102 to a network.

[0070] The input interface 1060 is an interface for the user to input information, and is composed of, for example, a touch panel, a keyboard, a mouse, and the like.

[0071] The output interface 1070 is an interface for presenting information to the user, and is configured, for example, by a liquid crystal panel, an organic EL (Electro-Luminescence) panel, or the like.

[0072] The functions of the information processing device 102 may be physically realized using a plurality of information processing devices. In this case, each of the information processing devices may be physically configured similarly to the information processing device 102 shown in Fig. 6. The plurality of information processing devices may be connected to each other via a network configured by wired or wireless means or a combination thereof, and may be configured to be able to transmit and receive information to and from each other via the network.

[0073] So far, an example of the configuration of the information processing system 100 according to the first embodiment has been described. Next, an example of the operation of the information processing system 100 according to the first embodiment will be described.

[0074] (Example of Operation of Information Processing System 100 According to First Embodiment) The information processing system 100 executes information processing. The information processing according to this embodiment is processing for authenticating an object shown in a target image by using a target image obtained by capturing the object.

[0075] In this embodiment, an example will be described in which the information processing includes an authentication process for authenticating the target, but the information processing may be any process for authenticating the target using a target image and does not have to include the authentication process.

[0076] The information processing may be started, for example, by receiving a signal from a sensor (not shown). This sensor is, for example, a sensor that detects the presence or entry of a person into an area photographed by the photographing device 101. Note that the method for starting the information processing is not limited to this.

[0077] FIG. 7 is a flowchart illustrating an example of information processing according to the first embodiment.

[0078] The image capturing device 101 captures an image of a target (step S101), thereby generating a target image showing the target.

[0079] 8 is a diagram showing an example of a target image according to embodiment 1. The target image according to this embodiment is an eye image including both eyes of a subject. The eye image is an example of an image including a target region according to this embodiment. As described above, the target region according to this embodiment is an iris region in which the iris of the right eye is captured.

[0080] The target image may be changed in various ways depending on the type of authentication to be performed using the image. For example, when face authentication is performed, the target image may be a face image including the target's face and background, or an image including the target's upper body or entire body. In this case, the target region is the facial region in which the face is reflected. For example, when vein authentication is performed, the target region may be an image including an area in which veins are reflected.

[0081] Furthermore, for example, the target image used for iris authentication is not limited to an eye image, but may be a face image, an image including the upper body or the entire body of the target, etc. Furthermore, the eye image is not limited to an eye image including both eyes of the target, but may be an image of one eye including a predetermined eye, either the left or right eye of the target, etc. The eye image illustrated in FIG. 8 includes not only the iris but also the pupil, the white of the eye, the eyelid, the eyelashes, etc. However, the eye image only needs to include at least the iris region used in the authentication process, and may not include one or more of the pupil, the white of the eye, the eyelid, the eyelashes, etc.

[0082] Referring again to Fig. 7, the quality estimation unit 111 estimates the position of the target region in the target image (step S102).

[0083] The quality estimation unit 111 according to this embodiment estimates, for example, the eye position including the position of the pupil center, pupil diameter, and iris diameter of the right eye from the target image.

[0084] The quality estimation unit 111 may use a general technique such as pattern matching or a machine learning model to estimate the eye position. When using a machine learning model, the quality estimation unit 111 may use a learning model that has been trained to estimate the eye position from a target image, and estimate the eye position in the target image using the target image as an input. In learning this learning model, supervised learning may be performed using training data including the eye position of the target image as an input.

[0085] The quality estimation unit 111 estimates the quality of the target image (step S103), thereby generating quality information including the quality of the target image.

[0086] The quality estimation unit 111 according to this embodiment identifies the iris region (i.e., the region showing the iris of the right eye of the subject) included in the target image based on, for example, the eye position estimated in step S102. The iris region is roughly an annular region centered on the pupil center and surrounded by concentric circles with diameters corresponding to the pupil diameter and the iris diameter, respectively. Note that, as shown in FIG. 8 , if part of the iris region is covered by an eyelid or the like, the iris region will have a shape in which the portion covered by the eyelid or the like is missing from the annular region.

[0087] The quality estimation unit 111 calculates the resolution of the identified iris region. The resolution is, for example, the image size. In this case, the quality estimation unit 111 calculates the image size of the identified iris region. The image size of the iris region is the number of pixels in the iris region.

[0088] Alternatively, the quality estimation unit 111 may obtain the quality of the target image using a learning model. In this case, the learning model may be a learning model that has been trained to estimate the quality of the target image, and may estimate the quality of the target image using the target image as an input. Furthermore, in the learning of this learning model, supervised learning may be performed using training data including the quality of the target image as an input.

[0089] Note that the iris diameter may be used as the image size. In this case, step S102 includes a process of estimating the quality of the target image. Furthermore, even when a value other than the iris diameter is used as the quality of the target image, a common learning model may be used in steps S102 and S103. In this case, the learning model may be a learning model trained to estimate the eye position and the quality of the target image from the target image, and the target image may be used as an input to estimate the eye position in the target image and the quality of the target image. Furthermore, in the learning of this learning model, supervised learning may be performed using training data including the eye position and the quality of the target image from the target image as an input.

[0090] The policy determination unit 112 determines a network policy based on the quality of the target image estimated in step S103 (step S104).

[0091] In more detail, for example, the policy determination unit 112 may determine the network policy so that at least one of the above (A) and (B) is satisfied, regardless of the quality information of the target image.

[0092] The image processing unit 113 processes the target image using the neural network NN1 according to the network policy determined in step S104 (step S105).

[0093] More specifically, for example, the image processing unit 113 uses a neural network NN1 according to the network policy to extract the feature amount from the iris area estimated in step S102.

[0094] For example, assume that in step S104, the policy determination unit 112 determines a network policy that satisfies both (A) and (B) regardless of the quality information of the target image. As a result, regardless of the quality of the target image, it is possible to make the time required for processing using the neural network NN1 (i.e., the processing for extracting features, for example) and the accuracy of authentication performed in the authentication process comparable.

[0095] The authentication unit 114 performs authentication processing using the result of the processing in step S105 (step S106).

[0096] In more detail, for example, the authentication unit 114 acquires the feature amount extracted by the image processing unit 113. The authentication unit 114 compares the acquired feature amount with the feature amount obtained from registered data stored in advance, and authenticates the target based on the comparison result.

[0097] More specifically, for example, the authentication unit 114 calculates the similarity between the acquired feature amount and the feature amount obtained from the enrollment data. The authentication unit 114 compares the similarity with a predetermined threshold and authenticates the target based on the result. For example, the authentication unit 114 determines that the authentication of the target is successful when the similarity is equal to or greater than the threshold. Alternatively, for example, the authentication unit 114 determines that the authentication of the target is unsuccessful when the similarity is less than the threshold. Note that the authentication method is not limited to this, and a general method may be used.

[0098] (Operations and Effects) As described above, according to this embodiment, the information processing system 100 includes the quality estimation unit 111 and the image processing unit 113 .

[0099] The quality estimation unit 111 estimates quality information of the target image. The image processing unit 113 processes the target image using a neural network NN1 according to a network policy determined based on the quality information of the target image.

[0100] This allows the processing in the neural network NN1 to be changed depending on the quality of the target image, thereby making it possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0101] According to this embodiment, the image processing unit 113 extracts features from a target image using a neural network NN1 according to a network policy.

[0102] This allows authentication processing to be performed using the feature values. The processing for extracting the feature values ​​is performed using a neural network NN1 in accordance with the network policy. Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0103] According to this embodiment, the target image is an eye image including an iris region in which the iris is captured, and the quality information includes the iris diameter.

[0104] This allows the processing in the neural network NN1 to be changed depending on the iris diameter of the target image, thereby making it possible to perform authentication such as iris recognition regardless of the quality of the target image while suppressing increases in processing time.

[0105] According to this embodiment, the quality information includes the resolution of the target image.

[0106] This allows the processing in the neural network NN1 to be changed depending on the resolution of the target image, thereby making it possible to perform authentication regardless of the quality of the target image while suppressing increases in processing time.

[0107] According to this embodiment, the information processing system 100 further includes a policy determination unit 112 that determines a network policy based on the quality information.

[0108] This allows a network policy to be determined according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the determined network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0109] According to this embodiment, the policy determination unit 112 determines a network policy so that the processing time using the neural network NN1 falls within a predetermined range, regardless of the resolution of the target image.

[0110] This allows the processing in the neural network NN1 to be changed according to the resolution of the target image, while suppressing an increase in processing time associated with the change in processing. Therefore, authentication can be performed regardless of the quality of the target image while suppressing an increase in processing time.

[0111] According to this embodiment, the neural network NN1 is composed of a front-stage group FNN2 including an input layer INL and a rear-stage group RNN including a final layer OUTL. The network policy is applied to image processing using the front-stage group FNN2.

[0112] This allows the processing in the neural network NN1 to be changed depending on the resolution of the target image. Furthermore, the increase in processing time can be suppressed compared to when the quality of the input image input to the neural network NN1 is improved using image super-resolution technology, etc. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing the increase in processing time.

[0113] Second Embodiment In this embodiment, a detailed example of a network policy and processing executed by the image processing unit 113 will be described. In this embodiment, for simplicity, descriptions that overlap with the above-described embodiment will be omitted as appropriate.

[0114] The information processing system according to this embodiment may be configured in a similar manner to the information processing system 100 according to the first embodiment illustrated in Figures 4 and 6. That is, for example, the image processing unit 113 processes the target image using a neural network according to the network policy, as in the first embodiment. Also, for example, the neural network constituting the image processing unit 113 performs calculations using more detailed information according to the network policy, as the quality of the target image decreases, as in the first embodiment.

[0115] The image processing unit 113 according to this embodiment converts the iris region included in the target image into a standard image of a predetermined shape, and processes the converted standard image as an input image to the neural network.

[0116] 9 is a diagram showing an example of the functional configuration of the image processing unit 113 according to embodiment 2. The image processing unit 113 includes a first processing unit 113a and a second processing unit 113b.

[0117] The first processing unit 113a converts a target region (i.e., for example, an iris region) included in the target image into a standard image of a predetermined shape. In detail, for example, the first processing unit 113a extracts the iris region from the target image and converts the extracted iris region into a standard image of a predetermined shape. In this way, the first processing unit 113a generates a standard image based on the target image.

[0118] The second processing unit 113b processes the standard image acquired by the first processing unit 113a using a neural network according to the network policy determined by the policy determination unit 112. In processing the standard image, the second processing unit 113b according to this embodiment extracts, for example, feature amounts from the standard image.

[0119] 10 is a diagram showing an example of the configuration of a neural network NN2 according to embodiment 2. The neural network NN2 is used for processing standard images performed by the second processing unit 113b according to embodiment 2. In this embodiment, too, an example will be described in which the neural network NN2 is a convolutional neural network.

[0120] The neural network NN2 includes, for example, multiple layers, similar to embodiment 1. The multiple layers included in the neural network NN2 are configured from a front-stage group FNN2 including an input layer INL and a rear-stage group RNN similar to embodiment 1.

[0121] The front-stage group FNN2 includes an input layer INL, CLN convolutional layers FCL, and an intermediate output layer MOUT.

[0122] The input layer INL acquires an input image and propagates the input image to, for example, subsequent layers, similarly to embodiment 1. The input image according to this embodiment is a standard image generated by the first processing unit 113a.

[0123] Each layer, including the CLN convolutional layers FCL, processes an input image and extracts, for example, features of the input image. The intermediate output layer MOUT outputs the processing results of the neural network NN2 as intermediate features. Note that the intermediate output layer MOUT may be a convolutional layer.

[0124] In this embodiment as well, the network policy determined by the policy determination unit 112 is applied to image processing using the front-stage group FNN2.

[0125] In detail, for example, the policy determination unit 112 according to this embodiment determines a network policy such that the lower the resolution of the target image, the larger the number CLN of convolutional layers FCL included in the front-stage group FNN2. This network policy is an example of a network policy in which the lower the quality of the target image, the more detailed information the front-stage group FNN2 uses to perform calculations.

[0126] The information processing according to this embodiment may be configured substantially similarly to the information processing according to the first embodiment illustrated in Fig. 7. That is, for example, in step S105, the image processing unit 113 processes the target image using a neural network according to a network policy determined based on quality information of the target image.

[0127] 11 is a flowchart showing a detailed example of image processing (step S105) according to embodiment 2. In the image processing (step S105), as described with reference to FIG. 7, the image processing unit 113 may process the target image using a neural network according to the network policy determined in step S104.

[0128] The first processing unit 113a extracts a target region from the target image (step S105a).

[0129] More specifically, for example, the target region is the iris region, and therefore the first processing unit 113a extracts a roughly annular iris region.

[0130] 12 is a diagram showing an example of an iris region according to embodiment 2. In the figure, the iris region is a hatched region surrounded by inner and outer circumferences that are concentric circles centered on the pupil center O. The inner circumference corresponds to the pupil diameter. The outer circumference corresponds to the iris diameter. Such an iris region can be represented, for example, by a polar coordinate system that uses the pupil center O as the origin, a radial length R, and an angle θ with respect to a predetermined reference direction. Note that although the area inside the inner circumference corresponds to the pupil, this area does not need to be included in the iris region.

[0131] 11 again, the first processing unit 113a converts the target region extracted in step S105a into a standard image of a predetermined shape (step S105b), thereby generating the standard image.

[0132] 13 is a diagram showing an example of a standard image according to embodiment 2. In this figure, an example in which the predetermined shape is a rectangle is shown, but the predetermined shape is not limited to this.

[0133] When the annular iris region shown in Fig. 12 is converted into the rectangular standard image shown in Fig. 13, for example, the height H of the standard image corresponds to the width R1 of the annular iris region, and the width W of the standard image corresponds to the outer and inner sides of the annular iris region.

[0134] In step S105b, the first processing unit 113a may convert the iris region into a standard image of a size corresponding to its quality information (e.g., resolution). That is, the size of the standard image may vary depending on the quality of the target image (e.g., the image size of the iris region). In this case, the first processing unit 113a may convert the target region into a standard image of a larger size as the quality of the target image increases (e.g., the image size of the iris region increases). Furthermore, if the standard image is rectangular, its aspect ratio may be constant.

[0135] Any suitable method may be used to convert the annular iris region into a standard rectangular image.

[0136] For example, the first processing unit 113a may convert the target region into a standard image of a discretely different size depending on the quality of the target image. In this case, for example, the first processing unit 113a may store in advance conversion pattern data indicating the sizes of multiple standard images corresponding to the quality of the target image. The first processing unit 113a may then obtain, from the conversion pattern data, the size of the standard image corresponding to the quality of the target region estimated in step S103, and convert the target region extracted in step S105a into a standard image of the obtained size. Such conversion may be performed appropriately using conversion from a polar coordinate system to a Cartesian coordinate system, conversion to enlarge or reduce the image, or the like.

[0137] Alternatively, for example, the first processing unit 113a may convert the target region into standard images of successively different sizes depending on the quality of the target image. In this case, for example, the first processing unit 113a may convert the iris region extracted in step S105a into a rectangular image using a conversion from a polar coordinate system to a Cartesian coordinate system. If the aspect ratio of this rectangular image differs from the aspect ratio of a predetermined standard image, the first processing unit 113a may convert the rectangular image into a standard image of the predetermined aspect ratio using at least one of an image enlargement and reduction conversion.

[0138] At this time, in order to convert the aspect ratio of the rectangular image into a predetermined aspect ratio of a standard image, the extent to which the vertical and horizontal directions are converted may be determined according to predetermined conversion conditions.

[0139] For example, if the conversion condition is that the horizontal direction should not be converted, the first processing unit 113a may convert the rectangular image into a standard image that is a rectangle with a predetermined aspect ratio by enlarging or reducing it vertically. Also, for example, if the conversion condition is that the vertical direction should not be converted, the first processing unit 113a may convert the rectangular image into a standard image that is a rectangle with a predetermined aspect ratio by enlarging or reducing it horizontally. Note that the conversion conditions are not limited to these.

[0140] By converting in this way, the iris region can be converted into a standard image of a size according to its quality information (for example, resolution). Furthermore, the standard image becomes an image of quality (for example, resolution) corresponding to the quality information of the iris region.

[0141] Referring again to Fig. 11, the second processing unit 113b processes the standard image generated in step S105b using the neural network NN2 according to the network policy determined in step S104 (step S105c).Then, the second processing unit 113b may return to the information processing shown in Fig. 7.

[0142] More specifically, for example, the size of the standard image may vary discretely or continuously depending on the quality of the target image, as described above. Examples of network policies will be described for each case.

[0143] (Example of network policy when standard image sizes vary discretely) For example, assume that the quality of the target image is the resolution of the input image. Also, assume that when the resolution of the input image is a predetermined reference resolution, the number CLN of convolution layers FCL used for calculation in the front-stage group FNN2 is a reference value RFN1.

[0144] In this case, when the resolution of the input image becomes 1 / N of the reference resolution, the policy determination unit 112 may determine a network policy that increases the number CLN of convolution layers FCL used in the calculations in the front-stage group FNN to RFN1*N*N, where N and RFN1 are each an integer greater than or equal to 1. Also, "*" represents multiplication.

[0145] As a result, the second processing unit 113b processes the input image using the front-stage group FNN2 including the number of convolutional layers FCL according to the network policy, and outputs intermediate features.

[0146] The network policy is not limited to this, and for example, the level of detailed information to be used in the calculation may be changed as appropriate depending on the quality of the target image.

[0147] (Example of a network policy when standard image sizes vary continuously) For example, as in the case where the sizes vary discretely, the quality of the target image is assumed to be the resolution of the input image. Also, when the resolution of the input image is a predetermined reference resolution, the number CLN of convolution layers FCL used in the calculation in the front-stage group FNN2 is assumed to be a reference value RFN1.

[0148] In this case, when the resolution of the input image becomes 1 / α of the reference resolution, the policy determination unit 112 may determine a network policy that increases the number CLN of convolution layers FCL used in the calculations in the front-stage group FNN to [RFN1*α*α]. Here, α is a real number, and satisfies, for example, 0<α≦1. Furthermore, the symbol [X] is a Gaussian function (also called a floor function) and represents the largest integer that does not exceed the real number X in the symbol. In other words, [RFN1*α*α] represents the largest integer that does not exceed RFN1*α*α.

[0149] As a result, the second processing unit 113b processes the input image and outputs intermediate features using a front-stage group FNN2 including a number CLN of convolutional layers FCL according to the network policy. Note that the network policy is not limited to this, and for example, the level of detailed information to be used in the calculation may be changed as appropriate depending on the quality of the target image, just as in the case where the standard image sizes vary discretely.

[0150] Furthermore, when the size of the standard image varies continuously, the shape of the intermediate feature map may be adjusted in the front group FNN2 so that the intermediate features have a shape (i.e., size) that can be input to the rear group RNN.

[0151] For example, interpolation can be used as a method for adjusting the shape of a map of intermediate features. For example, suppose a convolution operation of a slide S and a padding PD is performed using a 3×3 filter, and the shape of the feature map input thereto is (Hin, Win). If the shape of the feature map output by this convolution operation is (Hout, Wout), the input feature map may be adjusted using interpolation so as to satisfy the relationships Hin = S * (Hout-1) + 3-2 * PD and Win = S * (out-1) + 3-2 * PD. The interpolation may be performed using a nearest neighbor method, linear interpolation, a bilinear method, a bicubic method, or the like. For example, such interpolation may be performed before the first convolution layer FCL (i.e., the convolution layer FCL closest to the input) included in the preceding group FNN2.

[0152] Furthermore, for example, as a method for adjusting the map shape of intermediate features, the value of a dialation parameter used in the convolutional layer FCL can be used. The dialation parameter is a parameter for adjusting the interval between values ​​on a map that is multiplied by a filter used in the convolutional layer FCL.

[0153] Note that the method for adjusting the map shape of intermediate feature amounts is not limited to these.

[0154] (Operations and Effects) As described above, according to this embodiment, the image processing unit 113 includes the first processing unit 113a and the second processing unit 113b.

[0155] The first processing unit 113a converts an iris region included in the target image into a standard image of a predetermined shape. The second processing unit 113b processes the standard image using a neural network NN2 according to the network policy. The first processing unit 113a converts the iris region into a standard image of a size according to the quality information.

[0156] This allows the shape of the input image to the neural network NN2 to be standardized to a predetermined shape, making processing in the neural network NN2 easier than when the shape of the input image is indefinite.

[0157] Furthermore, since an image corresponding to the iris region can be used as the input image, the feature amount of the iris region can be extracted with high precision, and the precision of iris authentication can be improved.

[0158] Furthermore, in general, when the resolution of an image is improved using image super-resolution technology or the like, the high-resolution image is processed by a neural network, which often results in a long processing time in the neural network. In this embodiment, the standard image has a size according to the resolution, so the increase in processing time in the neural network NN2 can be suppressed compared to when high-resolution images are uniformly processed by the neural network NN2.

[0159] Therefore, it is possible to perform authentication with high accuracy regardless of the quality of the target image while suppressing an increase in processing time.

[0160] According to this embodiment, the neural network NN2 includes a plurality of convolution layers FCL and RCL. The policy determination unit 112 determines a network policy such that the number CLN of convolution layers FCL included in the preceding group FNN2 increases as the resolution of the target image decreases.

[0161] As a result, the lower the quality of the target image, the more convolutional layers FCL can be used, allowing calculations to be performed using detailed information. Therefore, the input image can be processed by the neural network NN2 so that authentication can be performed with high accuracy regardless of the quality of the input image. Furthermore, the increase in processing time can be suppressed compared to improving the quality of the input image input to the neural network NN2 using image super-resolution technology or the like.

[0162] Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0163] <Embodiment 3> In embodiment 2, an example was described in which the number CLN of convolutional layers FCL is increased as the resolution of the target image becomes lower, thereby performing calculations using more detailed information as the quality of the target image becomes lower. In embodiment 3, as another example in which calculations are performed using more detailed information as the quality of the target image becomes lower, an example will be described in which the number of channels in calculations in the convolutional layers FCL is increased as the resolution of the target image becomes lower. In this embodiment, to simplify the explanation, explanations that overlap with the above-mentioned embodiments will be omitted as appropriate.

[0164] In this embodiment, the configuration of the neural network used in the standard image processing performed by the second processing unit 113b is different from that of embodiment 2. Except for this point, the information processing system according to this embodiment may be configured generally similarly to the information processing system according to embodiment 2.

[0165] 14 is a diagram showing an example of the configuration of a neural network NN3 according to embodiment 3. The neural network NN3 is used for processing standard images performed by the second processing unit 113b according to embodiment 3. In this embodiment, too, an example will be described in which the neural network NN3 is a convolutional neural network.

[0166] The neural network NN3 includes, for example, multiple layers, similar to embodiment 1. The multiple layers included in the neural network NN3 are configured from a front-stage group FNN2 including an input layer INL and a rear-stage group RNN similar to embodiment 1.

[0167] The front-stage group FNN3 includes an input layer INL similar to that in the first embodiment, P convolution layers FCL_1 to FCL_P, and an intermediate output layer MOUT, where P is an integer equal to or greater than 2.

[0168] The convolutional layers FCL_1 to FCL_P have different numbers of channels. The number of channels corresponds to, for example, the number of filters used in each of the convolutional layers FCL_1 to FCL_P. Each layer including one of the convolutional layers FCL_1 to FCL_P processes an input image and extracts, for example, features of the input image. Note that, although the present embodiment will be described using an example in which there is one convolutional layer FCL with the same number of channels, the preceding group FNN3 may include multiple convolutional layers FCL with the same number of channels.

[0169] The intermediate output layer MOUT outputs the processing results of the neural network NN2 as intermediate features.

[0170] In this embodiment, the convolutional layers FCL_1 to FCL_P have different numbers of channels, and therefore the shapes of the feature maps output from each of the convolutional layers FCL_1 to FCL_P are different. Regardless of which of the convolutional layers FCL_1 to FCL_P is used, the intermediate output layer MOUT may adjust the size of the intermediate features (the shape of the feature map indicating the intermediate features) so that the intermediate features can be input to the subsequent group RNN. For example, 1 × 1 convolution may be used for this adjustment.

[0171] In this embodiment as well, the network policy determined by the policy determination unit 112 is applied to image processing using the front-stage group FNN3.

[0172] In detail, for example, the policy determination unit 112 according to this embodiment determines a network policy such that the lower the resolution of the target image, the greater the number of channels in the preceding group FNN2 to use. This network policy is an example of a network policy in which the lower the quality of the target image, the more detailed information the preceding group FNN3 uses to perform calculations.

[0173] The information processing according to this embodiment may be configured generally similarly to the information processing according to the first embodiment illustrated in Fig. 7. The details of the image processing (step S105) according to this embodiment may be configured generally similarly to the image processing (step S105) according to the second embodiment illustrated in Fig. 11.

[0174] That is, for example, in step S105c, the second processing unit 113b processes the standard image generated in step S105b using the neural network NN3 according to the network policy determined in step S104.

[0175] For example, suppose the size of standard images varies discretely and the quality of the target image is the resolution of the input image. Furthermore, when the resolution of the input image is a predetermined reference resolution, the number of channels in the convolution layers FCL_1 to FCL_P used for calculations in the front-stage group FNN2 is a reference value RFN2, where RFN2 is an integer equal to or greater than 1.

[0176] In this case, when the resolution of the input image becomes 1 / N of the reference resolution, the policy determination unit 112 may determine a network policy that increases the number of channels of the convolution layers FCL_1 to FCL_P used for calculations in the previous group FNN to RFN2*N*N.

[0177] As a result, the second processing unit 113b processes the input image and outputs intermediate features using the front-stage group FNN3, which includes the convolutional layer FCL of the number of channels according to the network policy, among the convolutional layers FCL_1 to FCL_P. Note that the network policy is not limited to this, and for example, the level of detailed information to be used in the calculation may be changed as appropriate depending on the quality of the target image.

[0178] For example, if the size of standard images continuously changes, when the resolution of the input image becomes 1 / α of the reference resolution, a network policy may be determined that increases the number of channels of the convolutional layers FCL_1 to FCL_P used for calculations in the front-stage group FNN to [RFN2*α*α]. Here, the symbol [ ] represents the same Gaussian function as above. That is, [RFN2*α*α] represents the largest integer not exceeding RFN2*α*α.

[0179] Note that the network policy is not limited to this, and for example, the level of detailed information to be used for calculations may be changed as appropriate depending on the quality of the target image, just as in the case where the size of the standard image varies discretely.

[0180] Furthermore, when the size of standard images varies continuously, the map shape of the intermediate features may be adjusted in the front group FNN3 so that the intermediate features can be input to the rear group RNN, as in the second embodiment.

[0181] (Operations and Effects) As described above, according to this embodiment, the neural network NN3 includes multiple convolutional layers FCL_1 to FCL_P and RCL. The policy determination unit 112 determines a network policy such that the lower the resolution of the target image, the greater the number of channels used in the calculations in the convolutional layers FCL_1 to FCL_P included in the preceding group FNN3.

[0182] In this way, the lower the quality of the target image, the greater the number of channels that can be used for convolutional layers FCL_1 to FCL_P, allowing for calculations using detailed information. Therefore, the input image can be processed by the neural network NN3 so that authentication can be performed with high accuracy regardless of the quality of the input image. Furthermore, the increase in processing time can be suppressed compared to improving the quality of the input image input to the neural network NN3 using image super-resolution technology or the like.

[0183] Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0184] <Fourth Embodiment> In the fourth embodiment, an example of a learning method for the neural networks NN1 to NN3 will be described. In this embodiment, the learning method for the neural network NN1 according to the first embodiment will be described as an example. In addition, in this embodiment, for the sake of simplicity, descriptions that overlap with the above-mentioned embodiments will be omitted as appropriate.

[0185] 15 is a diagram showing an example of the configuration of an information processing system 400 according to embodiment 4. The information processing system 400 includes an information processing device 403 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0186] The information processing device 403 is a device for training the neural network NN1, and includes a training storage unit 421, a training unit 422, and a training policy determination unit 423.

[0187] The learning memory unit 421 is a memory unit for storing learning data in advance. The learning data includes learning target images and correct answers. For example, when extracting feature quantities of target images using the neural network NN1, the correct answers are the feature quantities of the learning target images.

[0188] The learning target image is an image for learning. Similar to the target image, the learning target image may be an image that includes a target region, such as an eye image.

[0189] Note that learning data including a plurality of different learning target images may be stored in advance in the learning storage unit 421. In this case, it is desirable that the plurality of learning target images include learning target images of different qualities.

[0190] The learning unit 422 inputs learning target images to the neural network NN1, thereby learning the neural network NN1.

[0191] In detail, for example, the learning unit 422 includes a feature extraction unit 422a, a first loss acquisition unit 422b, and a first parameter update unit 422c.

[0192] The feature extraction unit 422a extracts features of the learning target image using a neural network NN1 according to the network policy.

[0193] The first loss acquiring unit 422b calculates the error (first loss) between the extracted feature amount and the correct value of the learning target image to obtain a first loss function.

[0194] The first parameter update unit 422c calculates a gradient based on the first loss function and updates the parameters included in the neural network NN1.

[0195] The learning policy determination unit 423 determines a network policy for the learning target image based on the quality information of the learning target image and the learning probability distribution.

[0196] In detail, for example, the learning policy determiner 423 includes a first determiner 423a and a second determiner 423b.

[0197] The first determination unit 423a changes the quality information of the learning target image in accordance with the learning probability distribution by using random numbers, and determines the learning quality information.

[0198] More specifically, for example, the first determination unit 423a may have the same function as the quality estimation unit 111. That is, the first determination unit 423a may estimate the position of a target area in a learning target image and estimate the quality of the learning target image.

[0199] The first determination unit 423a then determines the quality information for the training image by changing the estimated quality of the training image according to the training probability distribution using random numbers. The training probability distribution is, for example, a probability distribution in which the quality indicated by the quality information of the training image has the highest probability and the probability gradually decreases as the degree of difference from the quality increases.

[0200] The second determination unit 423b determines a network policy for the learning target image in accordance with the learning quality information determined by the first determination unit 423a.

[0201] The information processing device 403 may be physically configured in the same manner as the information processing device 102 (see FIG. 6 ). However, the storage device 1040 of the information processing device 403 stores program modules for realizing the functions of the information processing device 403. The network interface 1050 of the information processing device 403 is an interface for connecting the information processing device 403 to a network.

[0202] (Example of operation of information processing system 400 according to embodiment 4) The information processing system 400 executes information processing. The information processing according to this embodiment includes a learning process in addition to the same processes as those in embodiment 1. The learning process is, for example, a process for training the neural network NN1. For example, the information processing device 403 starts the learning process when it receives a user instruction.

[0203] FIG. 16 is a flowchart illustrating an example of a learning process according to the fourth embodiment.

[0204] The learning policy determination unit 423 acquires a learning target image from the learning storage unit 421 (step S401).

[0205] The learning policy determination unit 423 may acquire learning target images from another information processing device (not shown) via a network or the like, or may acquire learning target images via a storage medium.

[0206] The learning policy determination unit 423 determines a network policy for the learning image acquired in step S401 based on the quality information of the learning image and the learning probability distribution (step S402).

[0207] In detail, for example, the first determination unit 423a changes the quality information of the learning target image in accordance with the learning probability distribution, and determines the learning quality information (step S402a).

[0208] More specifically, for example, the first determination unit 423a estimates the quality of the learning target image. Note that the quality of the learning target image may be associated with the learning target image and stored in advance in the learning storage unit 421. In this case, the first determination unit 423a does not need to estimate the quality of the learning target image.

[0209] The first determination unit 423a changes the quality estimated for the learning target image in accordance with the learning probability distribution, and determines the learning quality information.

[0210] 17 is a diagram showing an example of a learning probability distribution according to the fourth embodiment. The horizontal axis of the figure represents the quality Q. The vertical axis of the figure represents the occurrence probability P1.

[0211] Q1 in the figure is the quality indicated by the quality information of the training target image. The training probability distribution illustrated in the figure is a probability distribution in which the quality Q1 indicated by the quality information of the training target image appears with the highest probability, and as the degree of difference from that quality increases, the probability of appearance decreases. Furthermore, the training probability distribution illustrated in the figure is an example of a probability distribution that is symmetrical with respect to the quality Q1 of the training target image.

[0212] The first determination unit 423a changes the quality Q1 of the training image by generating random numbers according to this training probability distribution. Q2 in the figure is an example of a training quality obtained based on the quality Q1 of the training image and the random numbers generated according to the training probability distribution. In this case, the first determination unit 423a changes the quality Q1 of the training image and determines training quality information including the training quality Q2.

[0213] The training probability distribution is not limited to the probability distribution exemplified in FIG. 17 . For example, generally, the lower the quality of the target image, the more difficult it is to accurately authenticate using the target image. Therefore, the training probability distribution may be a probability distribution in which a low quality lower than the quality Q1 of the training target image is more likely to be achieved than a high quality higher than the quality Q1 of the training target image. Also, for example, the training probability distribution may be a probability distribution in which a high quality is more likely to be achieved than a low quality lower than the quality Q1. Furthermore, the training policy determination unit 423 may determine multiple network policies for one training target image.

[0214] Referring again to Fig. 16, the second determination unit 423b determines a network policy for the learning target image in accordance with the learning quality information determined in step S402a (step S402b).

[0215] In detail, for example, the second determination unit 423b may determine the network policy corresponding to the learning quality information determined in step S402a as the network policy for the learning target image.

[0216] The learning unit 422 inputs the learning target image acquired in step S401 to the neural network NN1 according to the network policy determined in step S402b, and then the learning unit 422 performs learning of the neural network NN1 (step S403), thereby completing the learning process.

[0217] A general machine learning technique may be used for the learning in the learning unit 422. For example, the learning unit 422 (feature extraction unit 422a) extracts feature quantities of the learning target image using a neural network NN1 according to the network policy (step S403a). As a result, the learning unit 422 may estimate the feature quantities of the learning target image using the neural network NN1 according to the network policy.

[0218] Then, the learning unit 422 (first loss acquisition unit 422b) calculates, for example, the error (first loss) between the feature extracted in step S403a and the correct value of the learning target image (step S403b). The learning unit 422 may then calculate a first loss function for calculating the error (loss) between the estimated feature and the correct value of the learning target image. For example, cross entropy (error), mean squared error, mean absolute error (MAE), or the like may be used as the loss.

[0219] The learning unit 422 (first parameter update unit 422c) may calculate a gradient based on the first loss function to calculate first parameters used to update the parameters included in the neural network NN1. Then, the learning unit 422 (first parameter update unit 422c) may use the first parameters to update the parameters included in the neural network NN1 (step S403c).

[0220] The learning policy determination unit 423 may acquire a plurality of learning target images in step S401. In this case, the learning policy determination unit 423 and the learning unit 422 may execute steps S402 and S403 for each of the learning target images.

[0221] (Operations and Effects) As described above, according to this embodiment, the information processing system 400 further includes the learning unit 422 that inputs learning target images to the neural network NN1 to train the neural network NN1.

[0222] This allows the neural network NN1 to learn.

[0223] According to this embodiment, the information processing system 400 further includes a learning policy determination unit 423 that determines a network policy for a training target image based on the quality information of the training target image and the training probability distribution. The learning unit 422 trains the neural network NN1 by inputting the training target image to the neural network NN1 according to the determined network policy. The training probability distribution is a probability distribution that has the highest probability for the quality indicated by the quality information of the training target image and gradually decreases as the degree of difference from the quality increases.

[0224] Generally, the quality estimation unit 111 may erroneously estimate the quality of a target image. According to this embodiment, based on a training target image, the neural network NN1 can be trained to correspond to a training quality that mainly includes the quality of the training target image and the quality of its surroundings. This allows for training for image processing (e.g., feature extraction using the neural network NN1) that enables accurate authentication even when the quality of the target image is erroneously estimated. Therefore, accurate authentication becomes possible regardless of the quality of the target image.

[0225] According to this embodiment, the training policy determination unit 423 includes a first determination unit 423 a and a second determination unit 423 b. The first determination unit 423 a uses random numbers to change the quality information of the training target image in accordance with the training probability distribution and determines the training quality information. The second determination unit 423 b determines a network policy for the training target image in accordance with the determined training quality information.

[0226] This allows the neural network NN1 to be trained based on the training image, with training quality that mainly includes the quality of the training image and the quality of its surroundings. As a result, as described above, it becomes possible to perform authentication with high accuracy regardless of the quality of the target image.

[0227] Fifth Embodiment In a fifth embodiment, another example of the training method for the neural networks NN1 to NN3 will be described. In this embodiment, the training method for the neural network NN1 according to the first embodiment will be described as an example. In this embodiment, for the sake of simplicity, descriptions that overlap with the above-described embodiments will be omitted as appropriate.

[0228] 18 is a diagram showing an example of the configuration of an information processing system 500 according to embodiment 5. The information processing system 500 includes an information processing device 503 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0229] The information processing device 503 is a device for training the neural network NN1. The information processing device 503 includes a training unit 422, a training storage unit 521, an image selection unit 524, and a training policy determination unit 525, which are generally similar to those in the fourth embodiment.

[0230] The learning memory unit 521 is a memory unit for storing learning data in advance. The learning data according to this embodiment may associate a plurality of learning target images with respective correct answer values. The plurality of learning target images may include learning target images of different qualities. The learning data according to this embodiment may further associate the quality of each learning target image.

[0231] The learning unit 422 inputs learning target images selected by an image selection unit 524 (described in detail later) to the neural network NN1, thereby training the neural network NN1.

[0232] The image selection unit 524 selects learning target images such that lower quality learning target images are selected more frequently. For example, the image selection unit 524 selects learning target images such that lower resolution learning target images are selected more frequently than higher resolution learning target images.

[0233] The learning policy determination unit 525 determines a network policy for the learning target image selected by the image selection unit 524 based on the quality information of the learning target image selected by the image selection unit 524. The learning policy determination unit 525 may determine the network policy for the learning target image selected by the image selection unit 524 using a process similar to the process used by the policy determination unit 112 to determine the network policy.

[0234] The information processing device 503 may be physically configured in the same manner as the information processing device 102 (see FIG. 6 ). However, the storage device 1040 of the information processing device 503 stores program modules for realizing the functions of the information processing device 503. The network interface 1050 of the information processing device 503 is an interface for connecting the information processing device 503 to a network.

[0235] (Example of operation of information processing system 500 according to embodiment 5) The information processing system 500 executes information processing. The information processing according to this embodiment includes a learning process in addition to the same processes as those in embodiment 1. The learning process is, for example, a process for training the neural network NN1, as in embodiment 4. For example, the information processing device 503 starts the learning process when it receives a user instruction.

[0236] FIG. 19 is a flowchart illustrating an example of a learning process according to the fifth embodiment.

[0237] The image selection unit 524 selects learning target images from the learning target images stored in the learning storage unit 521 so that the lower the quality of the learning target images, the more likely they are to be selected (step S501).

[0238] In more detail, the image selection unit 524 may determine the quality in accordance with the selection probability distribution, for example. The quality may be, for example, the resolution of the image.

[0239] 20 is a diagram showing an example of a selection probability distribution according to the fifth embodiment. The horizontal axis of the figure represents quality Q. The vertical axis of the figure represents occurrence probability P2. The selection probability distribution is a probability distribution in which the lower the quality, the higher the probability, within a predetermined quality range (the range from Q3 to Q4 in FIG. 20). Such a selection probability distribution may be obtained, for example, by multiplying quality (e.g., resolution) by a predetermined weight.

[0240] The image selection unit 524 may then select a learning target image associated with the determined quality from among the learning target images stored in the learning storage unit 521 .

[0241] If the learning data does not include quality, the image selection unit 524 may estimate the quality of the learning target image by estimating the position of a target region in the learning target image, similar to, for example, the quality estimation unit 111. In this case, the image selection unit 524 may select, for example, a learning target image whose estimated quality is the same as the quality calculated according to the selection probability distribution.

[0242] Referring again to Fig. 19, the image selection unit 524 acquires the learning target image selected in step S501 from, for example, the learning storage unit 421 (step S502).

[0243] The learning policy determination unit 525 determines a network policy for the learning target image acquired in step S502 based on the quality information of the learning target image acquired in step S502 (step S503).

[0244] In detail, for example, the learning policy determination unit 525 may acquire the quality information determined in step S501 from the image selection unit 524. Note that the information processing device 503 may further include a function similar to that of the quality estimation unit 111, and may use this function to estimate the quality information of the learning target image acquired in step S502.

[0245] The learning unit 422 performs learning of the neural network NN1 (step S403), as in the fourth embodiment, and ends the learning process.

[0246] In step S403 according to this embodiment, the learning unit 422 inputs the learning target image acquired in step S502 to the neural network NN1 according to the network policy determined in step S504. As a result, the learning unit 422 performs learning on the neural network NN1.

[0247] As described above, according to this embodiment, the information processing system 500 further includes an image selection unit 524 that selects training target images so that the lower the quality of the training target images, the more frequently the training target images are selected. The training unit 422 inputs the selected training target images to the neural network NN1, thereby training the neural network NN1.

[0248] Generally, the lower the quality of a target image, the more difficult it is to perform accurate authentication using the target image. According to this embodiment, since more learning target images with lower quality are selected, the neural network NN1 corresponding to lower quality can perform more learning. This makes it possible to perform learning for image processing (e.g., feature extraction using the neural network NN1) that enables accurate authentication even when the quality of the target image is low. Therefore, accurate authentication becomes possible regardless of the quality of the target image.

[0249] Sixth Embodiment The learning methods described in the fourth and fifth embodiments may be combined. In the sixth embodiment, as yet another example of the learning method for the neural networks NN1 to NN3, an example in which the learning methods described in the fourth and fifth embodiments are combined will be described. In this embodiment, the learning method for the neural network NN1 according to the first embodiment will be described as an example. In addition, in this embodiment, for the sake of simplicity, explanations that overlap with the above-mentioned embodiments will be omitted as appropriate.

[0250] 21 is a diagram showing an example of the configuration of an information processing system 600 according to embodiment 6. The information processing system 600 includes an information processing device 603 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0251] The information processing device 603 is a device for training the neural network NN1. The information processing device 603 includes a training unit 422 and a training policy determination unit 423 that are generally similar to those in the fourth embodiment, and a training memory unit 521 and an image selection unit 524 that are generally similar to those in the fifth embodiment.

[0252] The learning policy determination unit 423 according to this embodiment determines a network policy for the learning target image selected by the image selection unit 524 based on the quality information and learning probability distribution of the learning target image.

[0253] The learning policy determination unit 423 may include a first determination unit 423a and a second determination unit 423b, as described in embodiment 4. That is, the first determination unit 423a may use random numbers to change the quality information of the learning target image selected by the image selection unit 524 in accordance with the learning probability distribution, and determine the learning quality information. The second determination unit 423b may determine a network policy for the learning target image in accordance with the learning quality information determined by the first determination unit 423a.

[0254] The information processing device 603 may be physically configured in the same manner as the information processing device 102 (see FIG. 6 ). However, the storage device 1040 of the information processing device 603 stores program modules for realizing the functions of the information processing device 603. The network interface 1050 of the information processing device 603 is an interface for connecting the information processing device 603 to a network.

[0255] (Example of operation of information processing system 600 according to embodiment 6) The information processing system 600 executes information processing. The information processing according to this embodiment includes a learning process in addition to the same processes as those in embodiment 1. The learning process is, for example, a process for training the neural network NN1, as in embodiment 4. For example, the information processing device 603 starts the learning process when it receives a user instruction.

[0256] FIG. 22 is a flowchart illustrating an example of a learning process according to the sixth embodiment.

[0257] The image selection unit 524 executes steps S501 and S502 similar to those in the fifth embodiment.

[0258] The learning policy determination unit 423 executes a learning policy determination process (step S402) that is generally similar to that of the fourth embodiment.

[0259] The learning target image to be processed in step S402 according to this embodiment is, for example, the learning target image acquired in step S502. That is, in step S402 according to this embodiment, the learning policy determination unit 423 determines a network policy for the learning target image acquired in step S502.

[0260] The quality information of the learning target image used in step S402 according to this embodiment may be the quality information calculated in step S501. In this case, the learning policy determination unit 423 may acquire the quality information of the learning target image from the image selection unit 524. Note that the information processing device 603 may further include a function similar to that of the quality estimation unit 111, and may use this function to estimate the quality information of the learning target image acquired in step S502. In this case, the estimated quality information may be used in the processing of step S402 as the quality of the learning target image.

[0261] The learning unit 422 performs the process of step S403, which is generally similar to that of the fourth embodiment, and ends the learning process.

[0262] In detail, for example, the learning unit 422 may input the learning target image acquired in step S502 to the neural network NN1 according to the network policy determined in step S402b, thereby allowing the learning unit 422 to learn the neural network NN1.

[0263] (Operations and Effects) As described above, according to this embodiment, the information processing system 600 includes the image selection unit 524 , the learning policy determination unit 423 , and the learning unit 422 .

[0264] The image selection unit 524 selects learning target images so that the lower the quality of the learning target images, the more likely they are to be selected.

[0265] The learning policy determination unit 423 determines a network policy for the training image based on the quality information of the training image and the training probability distribution. The training probability distribution is a probability distribution that has the highest probability for the quality indicated by the quality information of the training image and gradually decreases as the degree of difference from the quality increases.

[0266] The learning unit 422 inputs the selected learning target image into the neural network NN1 according to the determined network policy, thereby learning the neural network NN1.

[0267] This allows for learning for image processing (e.g., feature extraction using the neural network NN1) that enables accurate authentication even when the quality of the target image is erroneously estimated, as described in embodiment 4. Also, as described in embodiment 5, it allows for learning for image processing (e.g., feature extraction using the neural network NN1) that enables accurate authentication even when the quality of the target image is low. Therefore, it becomes possible to perform authentication with even greater accuracy, regardless of the quality of the target image.

[0268] Seventh Embodiment A network policy may be determined using a policy decision model. In this embodiment, an example will be described in which the policy decision model is a first policy decision model configured using a neural network. In this embodiment, for simplicity, descriptions that overlap with the above-described embodiments will be omitted as appropriate.

[0269] 23 is a diagram showing an example of the configuration of an information processing system 700 according to embodiment 7. The information processing system 700 includes the same imaging device 101 as in embodiment 1, and an information processing device 702 that replaces the information processing device 102 according to embodiment 1.

[0270] The information processing device 702 functionally includes, for example, a quality estimation unit 111, an image processing unit 113, and an authentication unit 114 similar to those of the first embodiment, and a policy determination unit 712 that replaces the policy determination unit 112 of the first embodiment.

[0271] Similar to the policy determination unit 112 according to the first embodiment, the policy determination unit 712 determines a network policy based on the quality information estimated by the quality estimation unit 111. This network policy is applied to the neural network used by the image processing unit 113 for image processing (i.e., image processing of the target image).

[0272] The policy determination unit 712 according to this embodiment determines a network policy according to quality information of a target image using a trained first policy determination model. The first policy determination model is a machine learning model configured using a neural network.

[0273] The policy determination unit 712 functionally includes, for example, a likelihood acquisition unit 712a and an application policy determination unit 712b.

[0274] The likelihood obtaining unit 712a obtains a likelihood for each of a plurality of predetermined network policies according to quality information of the target image using the trained first policy decision model. That is, the first policy decision model according to this embodiment is a machine learning model that outputs the likelihood of each network policy when quality information of the target image is input.

[0275] In detail, for example, the likelihood obtaining unit 712a inputs the quality information estimated by the quality estimating unit 111 into the trained first policy decision model to obtain the likelihood of each network policy.

[0276] The application policy determination unit 712b determines a network policy to be applied to image processing of the target image based on the likelihood of each network policy obtained by the likelihood acquisition unit 712a.

[0277] In more detail, for example, the application policy determination unit 712b may determine the network policy corresponding to the maximum likelihood as the network policy to be applied to image processing of the target image. Also, for example, the application policy determination unit 712b may determine the network policy to be applied to image processing of the target image probabilistically according to the likelihood.

[0278] The method for determining a network policy based on the likelihood of each network policy is not limited to the above.

[0279] The information processing system 700 according to the seventh embodiment may be physically configured in the same manner as the information processing system 100 according to the first embodiment. However, the storage device 1040 of the information processing device 702 stores program modules for realizing the functions of the information processing device 702. Furthermore, the network interface 1050 of the information processing device 702 is an interface for connecting the information processing device 702 to a network.

[0280] (Example of Operation of Information Processing System 700 According to Seventh Embodiment) Fig. 24 is a flowchart showing an example of information processing according to the seventh embodiment. As shown in the figure, the information processing according to this embodiment includes step S704 instead of step S104.

[0281] The policy determination unit 712 determines a network policy based on the quality of the target image estimated in step S103 (step S704).

[0282] FIG. 25 is a flowchart showing an example of the network policy determination process (step S704) according to the seventh embodiment.

[0283] The likelihood obtaining unit 712a obtains the likelihood of each of the plurality of determined network policies using the trained first policy decision model and the quality information of the target image estimated in step S103 (step S704a).

[0284] The application policy determination unit 712b determines the network policy to be applied to the image processing of the target image based on the likelihood of each of the network policies calculated in step S704a (step S704b), and returns to information processing.

[0285] As described above, according to this embodiment, the policy determination unit 712 determines a network policy according to the quality information of the target image by using the trained first policy determination model. The first policy determination model is configured using a neural network.

[0286] This allows a network policy to be determined according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the determined network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0287] Eighth Embodiment In an eighth embodiment, an example of a learning method for the first policy decision model described in the seventh embodiment will be described. In this embodiment, for the sake of simplicity, descriptions that overlap with the above-described embodiments will be omitted as appropriate.

[0288] 26 is a diagram showing an example of the configuration of an information processing system 800 according to embodiment 8. The information processing system 800 includes an information processing device 803 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0289] The information processing device 803 is a device for training the first policy decision model and the neural network NN1. The information processing device 803 includes the same training memory unit 421 and training unit 422 as those in the fourth embodiment, a training target image acquisition unit 824, and a training policy decision unit 825.

[0290] The learning target image acquisition unit 824 determines the quality of the learning target image, and acquires the learning target image based on the learning data stored in the learning storage unit 421 .

[0291] In detail, for example, the learning target image acquisition unit 824 includes a learning quality determination unit 824a and an image acquisition unit 824b.

[0292] The learning quality determination unit 824a randomly determines the quality of the learning target image in accordance with a predetermined learning probability distribution (see, for example, the fourth embodiment).

[0293] The image acquisition unit 824b acquires learning target images according to the quality determined by the learning quality determination unit 824a. For example, the image acquisition unit 824b may acquire images of randomly determined quality from the learning storage unit 421. Alternatively, for example, the image acquisition unit 824b may acquire learning target images by acquiring images from the learning storage unit 421 and lowering the quality of the acquired images according to the randomly determined quality.

[0294] The training policy determination unit 825 performs training of the first policy determination model. For example, the training policy determination unit 825 calculates the likelihood of each of a plurality of network policies. The training policy determination unit 825 determines a network policy to be applied to image processing of the training target image based on the likelihood of each of the network policies. In this case, the training policy determination unit 825 may determine a network policy in a manner similar to that of the applied policy determination unit 712b. The training policy determination unit 825 updates the parameters of the first policy determination model based on the first loss.

[0295] The information processing device 803 may be physically configured in the same manner as the information processing device 102 (see FIG. 6 ), for example. However, the storage device 1040 of the information processing device 803 stores program modules for realizing the functions of the information processing device 803. The network interface 1050 of the information processing device 803 is an interface for connecting the information processing device 803 to a network.

[0296] (Example of operation of information processing system 800 according to embodiment 8) The information processing system 800 executes information processing. The information processing according to this embodiment includes, for example, learning processing. The learning processing is, for example, processing for learning the first policy decision model and the neural network NN1. For example, the information processing device 803 starts the learning processing when it receives a user instruction.

[0297] FIG. 27 is a flowchart illustrating an example of a learning process according to the eighth embodiment.

[0298] The learning target image acquisition unit 824 acquires learning target images of randomly determined quality (step S801).

[0299] In detail, for example, the learning quality determining unit 824a randomly determines the quality of the learning target image in accordance with a predetermined learning probability distribution (step S801a).

[0300] The image acquisition unit 824b acquires learning target images of the quality determined in step S801a, for example, using the learning data stored in the learning storage unit 421 (step S801b).

[0301] The learning policy determination unit 825 uses the first policy determination model to determine a network policy for the learning target image acquired in step S801b (step S802).

[0302] As in the fourth embodiment, the learning unit 422 performs learning of the neural network NN1 in accordance with the network policy determined in step S802 (step S403).

[0303] The learning policy determination unit 825 updates the parameters (second parameters) of the first policy determination model using the first loss calculated in step S403b (step S804), and ends the learning process.

[0304] Although an example in which the first policy decision model and the neural network NN1 are trained simultaneously has been described here, the first policy decision model and the neural network NN1 may be trained separately. For example, when training is performed on only one of the first policy decision model and the neural network NN1, it is preferable to set the parameters of the other of the first policy decision model and the neural network NN1 to fixed values.

[0305] (Operations and Effects) As described above, according to this embodiment, the information processing device 803 includes the learning policy determination unit 825 that performs learning of the first policy determination model.

[0306] This makes it possible to generate a trained first policy decision model, and to use this first policy decision model to decide a network policy according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the decided network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0307] Ninth Embodiment In the ninth embodiment, a detailed example of the training method for the first policy decision model described in the eighth embodiment will be described. In this embodiment, an example will be described in which the first policy decision model is trained separately from the training of the neural network NN1. In addition, in this embodiment, for the sake of simplicity, descriptions that overlap with the above-mentioned embodiments will be omitted as appropriate.

[0308] (Configuration Example of Information Processing System According to Ninth Embodiment) In the information processing system according to this embodiment, the information processing device according to this embodiment includes a learning policy determination unit 925 instead of the learning policy determination unit 825 .

[0309] 28 is a diagram illustrating an example of the configuration of the learning policy determiner 925 according to embodiment 9. The learning policy determiner 925 functionally includes, for example, a first learning likelihood acquirer 925a and a second parameter updater 925b.

[0310] The first learning likelihood acquisition unit 925a inputs the quality of the randomly determined learning target image into the first policy determination model, and obtains a likelihood (vector) for each of a plurality of network policies.

[0311] The second parameter update unit 925b updates the parameters (second parameters) of the first policy decision model based on the first loss and likelihood for each of the plurality of network policies.

[0312] In detail, for example, the second parameter update unit 925b includes a second loss acquisition unit 925b1 and a second parameter determination unit 925b2.

[0313] The second loss calculating unit 925b1 calculates a second loss (total loss) which is the sum of the first losses weighted by the likelihood for each of the plurality of network policies.

[0314] The second parameter determination unit 925b2 updates (the second parameter) of the first policy determination model based on the second loss.

[0315] The information processing apparatus according to this embodiment may be physically configured similarly to the information processing apparatus 802 (see FIG. 6 ). However, the storage device 1040 of the information processing apparatus according to this embodiment stores program modules for realizing the functions of the information processing apparatus according to this embodiment. Furthermore, the network interface 1050 of the information processing apparatus according to this embodiment is an interface for connecting the information processing apparatus according to this embodiment to a network.

[0316] (Example of operation of information processing system 900 according to embodiment 9) The information processing system 900 executes information processing. The information processing according to this embodiment includes, for example, learning processing. The learning processing according to this embodiment includes a first policy decision model learning processing for learning a first policy decision model.

[0317] For example, the learning processes of the fourth and fifth embodiments may be applied to the learning of the neural network NN1 etc. In this embodiment, an example will be described in which the learning of the neural network NN1 is performed separately from the learning of the first policy decision model, but the learning of the first policy decision model and the neural network NN1 may be performed simultaneously.

[0318] 29 is a flowchart illustrating an example of a first policy determination model learning process according to embodiment 9. For example, when the information processing device receives an instruction from a user, it starts the first policy determination model learning process.

[0319] For example, first, the process of step S801 is executed as in the eighth embodiment.

[0320] The first learning likelihood obtaining unit 925a obtains a likelihood (vector) for each of a plurality of network policies (step S901).

[0321] For example, the first learning likelihood acquisition unit 925a inputs the quality of the learning target image randomly determined in step S801 (see step S801a in Figure 27) into the first policy determination model to obtain the likelihood (vector) of each network policy.

[0322] The feature extraction unit 422a extracts features of the learning target image using the neural network NN1 corresponding to each of the plurality of network policies (step S902).

[0323] The first loss acquiring unit 422b acquires the error (first loss) between the feature extracted in step S902 and the correct value of the learning target image for each of the plurality of network policies (step S903).

[0324] The second parameter update unit 925b updates the parameters (second parameters) of the first policy decision model based on the likelihood and the first loss calculated in steps S901 and S902 (step S904).

[0325] In more detail, for example, the second loss calculating unit 925b1 calculates the second loss (total loss) which is the sum of the first losses weighted by the likelihood for each of the plurality of network policies (step S904a), thereby calculating a second loss function for calculating the second loss.

[0326] The second parameter determination unit 925b2 updates (the second parameter) of the first policy determination model based on the second loss calculated in step S904a (step S904b), and ends the first policy determination model learning process.

[0327] For example, the second parameter determination unit 925b2 may calculate a gradient based on the second loss function to determine a second parameter used to update the parameters included in the first policy determination model. Then, the second parameter determination unit 925b2 may update the parameters included in the first policy determination model using the calculated second parameter. The first policy determination model learning process may be repeatedly executed until a predetermined condition is satisfied.

[0328] (Actions and Effects) As described above, according to this embodiment, the information processing device includes the training policy determination unit 925 and the first loss acquisition unit 422b. The training policy determination unit 925 trains a first policy determination model for determining a network policy according to quality information of a target image from among predetermined network policies. The first loss acquisition unit 422b calculates the first loss when processing the training target image using a neural network corresponding to each of a plurality of network policies.

[0329] The first policy decision model is composed of a neural network.

[0330] The training policy determination unit 925 includes a first training likelihood acquisition unit 925a and a second parameter update unit 925b. The first training likelihood acquisition unit 925a inputs quality information of the training target image into the first policy determination model to determine the likelihood of each network policy. The second parameter update unit 925b updates the parameters of the first policy determination model based on the first loss and the likelihood of each network policy.

[0331] The second parameter updater 925b includes a second loss acquirer 925b1 and a second parameter determiner 925b2. The second loss acquirer 925b1 calculates a second loss, which is the sum of first losses weighted by likelihood for each network policy. The second parameter determiner 925b2 updates the parameters of the first policy decision model based on the second losses.

[0332] This makes it possible to generate a trained first policy decision model, and to use this first policy decision model to decide a network policy according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the decided network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0333] Tenth Embodiment In the seventh embodiment, an example was described in which a network policy is determined using a first policy decision model configured using a neural network. The policy decision model for determining a network policy does not necessarily have to use a neural network. In this embodiment, the policy decision model for determining a network policy may be configured from a plurality of probability distributions including learnable parameters. The plurality of probability distributions may correspond to a plurality of predetermined network policies, respectively.

[0334] In this embodiment, a policy decision model made up of a plurality of probability distributions is referred to as a second policy decision model, and the probability distributions making up the second policy decision model are referred to as policy probability distributions.

[0335] In addition, in this embodiment, for the sake of simplicity, descriptions that overlap with the above-described embodiment will be omitted as appropriate.

[0336] 30 is a diagram showing an example of the configuration of an information processing system 1100 according to embodiment 10. The information processing system 1100 includes the same imaging device 101 as in embodiment 1, and an information processing device 1102 that replaces the information processing device 102 according to embodiment 1.

[0337] The information processing device 1102 functionally includes, for example, a quality estimation unit 111, an image processing unit 113, and an authentication unit 114 similar to those of the first embodiment, and a policy determination unit 1112 that replaces the policy determination unit 112 of the first embodiment.

[0338] Similar to the policy determination unit 112 according to the first embodiment, the policy determination unit 1112 determines a network policy based on the quality information estimated by the quality estimation unit 111. This network policy is applied to the neural network used by the image processing unit 113 for image processing (i.e., image processing of the target image).

[0339] The policy determination unit 1112 according to this embodiment uses a trained second policy determination model to determine a network policy according to quality information of a target image. The second policy determination model is, for example, a model for determining a network policy according to the quality information of a target image from among predetermined network policies. The second policy determination model is composed of a plurality of policy probability distributions. Each of the plurality of policy probability distributions corresponds to a respective one of a plurality of predetermined neural networks.

[0340] The policy determination unit 1112 functionally includes, for example, a likelihood acquisition unit 1112a and an application policy determination unit 712b similar to that of the seventh embodiment.

[0341] The likelihood obtaining unit 1112a uses the trained second policy decision model to obtain a likelihood corresponding to the quality information of the target image for each of a plurality of predetermined network policies.

[0342] The second policy decision model according to this embodiment is configured from a plurality of policy probability distributions associated with a plurality of network policies, respectively. The likelihood obtaining unit 1112a according to this embodiment obtains the likelihood of each of the plurality of policy probability distributions.

[0343] The information processing system 1100 according to the tenth embodiment may be physically configured in the same manner as, for example, the information processing system 100 according to the first embodiment. However, the storage device 1040 of the information processing device 1102 stores program modules for realizing the functions of the information processing device 1102. Furthermore, the network interface 1050 of the information processing device 1102 is an interface for connecting the information processing device 1102 to a network.

[0344] (Example of Operation of Information Processing System 1100 According to Tenth Embodiment) Fig. 31 is a flowchart showing an example of information processing according to the tenth embodiment. As shown in the figure, the information processing according to this embodiment includes step S1104 instead of step S104.

[0345] The policy determination unit 1112 determines a network policy based on the quality of the target image estimated in step S103 (step S1104).

[0346] FIG. 32 is a flowchart showing an example of the network policy determination process (step S1104) according to the tenth embodiment.

[0347] The likelihood obtaining unit 1112a obtains the likelihood of each of a plurality of predetermined network policies using the trained second policy decision model and the quality information of the target image estimated in step S103 (step S1104a).

[0348] In detail, for example, the likelihood obtaining unit 1112a uses the quality information of the target image estimated in step S103 to obtain the likelihood of each of the multiple policy probability distributions that constitute the second policy decision model.

[0349] As in the seventh embodiment, the application policy determination unit 712b determines the network policy to be applied to image processing of the target image based on the likelihood of each network policy calculated in step S1104a (step S704b), and returns to information processing.

[0350] That is, for example, the applied policy determination unit 712b may determine the network policy corresponding to the policy probability distribution with the highest likelihood for the quality of the target image as the network policy to be applied to image processing of the target image.

[0351] Furthermore, for example, the applied policy determination unit 712b may determine a network policy to be applied to image processing of a target image probabilistically based on the likelihood of each policy probability distribution for the quality of the target image. In more detail, for example, assume that the likelihood when the quality of the target image is x is 0.2 for policy probability distribution A, 0.5 for policy probability distribution B, and 0.3 for policy probability distribution C. In this case, the applied policy determination unit 712b may select network policies A, B, and C corresponding to policy probability distributions A, B, and C, respectively, with probabilities of 20%, 50%, and 30%, respectively.

[0352] According to the present embodiment, the policy decision unit 1112 decides a network policy according to quality information of a target image by using the trained second policy decision model. The second policy decision model is composed of a plurality of policy probability distributions each associated with a plurality of network policies.

[0353] This allows a network policy to be determined according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the determined network policy. Therefore, it becomes possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0354] Eleventh Embodiment In an eleventh embodiment, an example of a learning method for the second policy decision model described in the tenth embodiment will be described. In this embodiment, for the sake of simplicity, descriptions that overlap with the above-described embodiments will be omitted as appropriate.

[0355] 33 is a diagram showing an example of the configuration of an information processing system 1200 according to embodiment 11. The information processing system 1200 includes an information processing device 1203 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0356] The information processing device 1203 is a device for training the second policy decision model and the neural network NN1. The information processing device 1203 includes a training memory unit 421 and a training unit 422 similar to those in the fourth embodiment, a training target image acquisition unit 824 similar to those in the eighth embodiment, and a training policy decision unit 1225.

[0357] The training policy determiner 1225 trains the second policy determination model. For example, the training policy determiner 1225 calculates the likelihood of each of the plurality of network policies, i.e., the likelihood of each of the plurality of policy probability distributions that constitute the second policy determination model.

[0358] The training policy determination unit 1225 determines a network policy to be applied to image processing of the training target image based on the likelihood of each policy probability distribution. In this case, for example, the training policy determination unit 1125 may determine a network policy in a manner similar to that of the applied policy determination unit 712b. The training policy determination unit 1225 updates a parameter (third parameter) of the second policy determination model based on, for example, the first loss.

[0359] 34 is a diagram illustrating an example of the functional configuration of the learning policy determiner 1225 according to embodiment 11. The learning policy determiner 1225 includes, for example, a second learning likelihood acquirer 1225a and a third parameter updater 1225b.

[0360] The second learning likelihood obtaining unit 1225a obtains a likelihood (vector) for each of a plurality of network policies based on the quality of the randomly determined learning target image and the second policy determination model.

[0361] For example, the second learning likelihood obtaining unit 1225a uses the quality of a randomly determined learning target image to obtain the likelihood of each of the multiple policy probability distributions that make up the second policy determination model.

[0362] The third parameter update unit 1225b updates the parameters of the second policy decision model based on the first loss and the corrected loss.

[0363] The correction loss is a loss for correcting the difference between the parameters of the policy probability distributions corresponding to different network policies so that the difference becomes larger. For example, if the mean of each of the policy probability distributions A and B is μ Ａ and μ Ｂ When Ａ and μ Ｂ The reciprocal of the absolute difference (1 / |μ Ａ -μ Ｂ |). However, the adjusted loss is not limited to this.

[0364] The parameters of the second policy decision model are one or more parameters included in each of the multiple policy probability distributions. Examples of the parameters include the variance and mean when the policy probability distribution is a normal distribution. However, the parameters are not limited to these. Furthermore, the policy probability distribution is not limited to a normal distribution.

[0365] In detail, for example, the third parameter update unit 1225b includes a third loss acquisition unit 1225b1 and a third parameter determination unit 1225b2.

[0366] The third loss acquiring unit 1225b1 calculates the third loss by adding the first loss and the adjusted loss. Note that the third loss based on the first loss and the adjusted loss is not limited to this.

[0367] The third parameter determination unit 1225b2 updates the parameters of each policy probability distribution based on the third loss.

[0368] The information processing device 1203 according to this embodiment may be physically configured in the same manner as the information processing device 802 (see FIG. 6 ). However, the storage device 1040 of the information processing device 1203 stores program modules for realizing the functions of the information processing device 1203. The network interface 1050 of the information processing device 1203 is an interface for connecting the information processing device 1203 to a network.

[0369] (Example of Operation of Information Processing System 1200 According to Eleventh Embodiment) The information processing system 1200 executes information processing. The information processing according to this embodiment includes learning processing. The learning processing is, for example, processing for learning the first policy decision model and the neural network NN1.

[0370] In this embodiment, an example will be described in which the second policy decision model is learned separately from the neural network NN1. The learning processes of the fourth and fifth embodiments, for example, may be applied to the learning of the neural network NN1. Note that the second policy decision model may be applied instead of the first policy decision model described in the eighth embodiment, and the learning of the second policy decision model and the neural network NN1 may be performed simultaneously.

[0371] 35 is a flowchart showing an example of a second policy determination model learning process according to embodiment 11. For example, when the information processing device receives an instruction from a user, it starts the second policy determination model learning process.

[0372] For example, first, the process of step S801 is executed as in the eighth embodiment.

[0373] The second learning likelihood obtaining unit 1225a obtains a likelihood (vector) for each of a plurality of network policies (step S1201).

[0374] For example, the second learning likelihood obtaining unit 1225a obtains the likelihood of each policy probability distribution according to the quality of the learning target image randomly determined in step S801 (see step S801a in FIG. 27 ). The second learning likelihood obtaining unit 1225a may determine a network policy based on the likelihood in a manner similar to that of the applied policy determining unit 712b.

[0375] The feature extraction unit 422a and the first loss acquisition unit 422b execute the processes of steps S902 and S903, respectively, similar to those of embodiment 9. When a network policy is determined in step S901, a feature amount and a first loss are obtained using a neural network in accordance with the network policy.

[0376] The third parameter update unit 1225b updates the parameters (third parameters) of the second policy decision model based on the first loss calculated in step S902 (step S1204).

[0377] In more detail, for example, the third loss calculating unit 1225b1 may calculate the third loss by adding the first loss calculated in step S902 and the corrected loss (step S1204a), thereby obtaining a third loss function for calculating the third loss.

[0378] The third parameter determination unit 1225b2 updates the parameters (third parameters) of each policy probability distribution based on the third loss calculated in step S1204a (step S1204b), and ends the second policy determination model learning process.

[0379] For example, the third parameter determination unit 1225b2 may calculate a gradient based on the third loss function to determine a third parameter used to update the parameters included in the second policy decision model (each policy probability distribution). Then, the third parameter determination unit 1225b2 may use the calculated third parameter to update the parameters included in the second policy decision model (each policy probability distribution). The second policy decision model learning process may be repeatedly executed until a predetermined condition is satisfied.

[0380] (Actions and Effects) As described above, according to this embodiment, the information processing device 1203 includes the training policy determination unit 1225 and the first loss acquisition unit 422b. The training policy determination unit 1225 trains a second policy determination model for determining a network policy according to quality information of a target image from among predetermined network policies. The first loss acquisition unit 422b calculates the first loss when processing the training target image using a neural network according to the network policy determined using the second policy determination model.

[0381] The second policy decision model is composed of a plurality of policy probability distributions associated with a plurality of network policies, each of the plurality of policy probability distributions including one or more parameters.

[0382] The training policy determination unit 1225 includes a second training likelihood acquisition unit 1225a and a third parameter update unit 1225b. The second training likelihood acquisition unit 1225a calculates the likelihood of each policy probability distribution using quality information of the training target image and each of the multiple policy probability distributions. The third parameter update unit 1225b updates the parameters of each policy probability distribution based on the first loss and a correction loss for correcting the differences between the parameters of the policy probability distributions corresponding to different network policies so as to increase the differences.

[0383] The third parameter updater 1225b includes a third loss calculator 1225b1 and a third parameter determiner 1225b2. The third loss calculator 1225b1 calculates a third loss by adding the first loss and the corrected loss. The third parameter determiner 1225b2 updates the parameters of each policy probability distribution based on the third loss.

[0384] This allows a trained second policy decision model to be generated, and this second policy decision model can be used to decide a network policy according to the quality of the target image. Then, the processing in the neural network NN1 can be changed according to the decided network policy. Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0385] Furthermore, because high-quality images are generally easy to process, updating the parameters of each policy probability distribution based only on the first loss may result in convergence to a training probability distribution in which high-quality training target images are sampled with high frequency. In this embodiment, a correction loss is added to update the parameters of each policy probability distribution. This makes it possible to prevent the training probability distribution from becoming a probability distribution in which high-quality training target images are sampled with high frequency. Therefore, it becomes possible to train the second policy decision model using training target images selected with appropriate probabilities from images of various qualities.

[0386] Twelfth Embodiment In a twelfth embodiment, an example of a third policy decision model will be described, in which a network policy is determined using the target image itself as an input instead of the quality of the target image. The third policy decision model is configured using, for example, a neural network. In this embodiment, for simplicity, descriptions that overlap with the above-described embodiments will be omitted as appropriate.

[0387] 36 is a diagram showing an example of the configuration of an information processing system 1300 according to embodiment 12. The information processing system 1300 includes the same image capturing device 101 as in embodiment 1, and an information processing device 1302 that replaces the information processing device 102 according to embodiment 1.

[0388] The information processing apparatus 1302 functionally includes, for example, the image processing unit 113 and authentication unit 114 similar to those of the first embodiment, and a policy determination unit 1312 that replaces the policy determination unit 112 according to the first embodiment.

[0389] The policy determination unit 1312 determines a network policy based on the target image, and this network policy is applied to the neural network used by the image processing unit 113 for image processing (i.e., image processing of the target image).

[0390] The policy determination unit 1312 according to this embodiment determines a network policy according to the target image using a trained third policy determination model. The third policy determination model is, for example, a machine learning model for determining a network policy according to the target image from among a plurality of predetermined network policies. The third policy determination model is configured using a neural network.

[0391] The policy determination unit 1312 functionally includes, for example, a likelihood acquisition unit 1312a and an application policy determination unit 712b similar to that of the seventh embodiment.

[0392] The likelihood obtaining unit 1312a obtains the likelihood of each of a plurality of predetermined network policies according to the target image by using the trained third policy decision model. That is, the third policy decision model according to this embodiment is a machine learning model that outputs the likelihood of each network policy when the target image is input.

[0393] In detail, for example, the likelihood obtaining unit 1312a inputs the target image into the trained third policy decision model and obtains the likelihood of each network policy.

[0394] The information processing system 1300 according to the twelfth embodiment may be physically configured in the same manner as, for example, the information processing system 100 according to the first embodiment. However, the storage device 1040 of the information processing device 1302 stores program modules for realizing the functions of the information processing device 1302. Furthermore, the network interface 1050 of the information processing device 1302 is an interface for connecting the information processing device 1302 to a network.

[0395] (Example of operation of information processing system 1300 according to embodiment 12) Fig. 37 is a flowchart showing an example of information processing according to embodiment 12. As shown in the figure, the information processing according to this embodiment includes step S1304 instead of step S104. Furthermore, the information processing according to this embodiment does not need to include step S103.

[0396] The policy determination unit 1312 determines a network policy based on the target image (step S1304).

[0397] FIG. 38 is a flowchart showing an example of a network policy determination process (step S1304) according to the twelfth embodiment.

[0398] The likelihood obtaining unit 1312a obtains the likelihood of each of a plurality of predetermined network policies using the target image and the trained third policy decision model (step S1304a).

[0399] As in the seventh embodiment, the application policy determination unit 712b determines the network policy to be applied to image processing of the target image based on the likelihood of each network policy calculated in step S1304a (step S704b), and returns to information processing.

[0400] (Operations and Effects) As described above, according to this embodiment, the policy determination unit 1312 determines a network policy according to a target image using the trained third policy determination model. The third policy determination model is configured using a neural network.

[0401] This allows a network policy to be determined according to the target image. Then, the processing in the neural network NN1 can be changed according to the determined network policy. Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0402] <Thirteenth Embodiment> In a thirteenth embodiment, an example of a learning method for the third policy decision model described in the twelfth embodiment will be described. In learning the third policy decision model, a learned policy decision model (such as the first policy decision model or the second policy decision model) is used. In this embodiment, an example will be described in which the first policy decision model is used in learning the third policy decision model.

[0403] In addition, in this embodiment, for the sake of simplicity, descriptions that overlap with the above-described embodiment will be omitted as appropriate.

[0404] 39 is a diagram showing an example of the configuration of an information processing system 1400 according to embodiment 13. The information processing system 1400 includes an information processing device 1403 in addition to the same imaging device 101 and information processing device 102 as those in embodiment 1.

[0405] The information processing device 1403 is a device for training the third policy decision model and the neural network NN1. The information processing device 1403 includes a training memory unit 421 and a training unit 422 similar to those in the fourth embodiment, a training target image acquisition unit 824 similar to those in the eighth embodiment, and a training policy decision unit 1425.

[0406] The training policy determination unit 1425 uses the trained first policy determination model to train the third policy determination model. For example, the training policy determination unit 1425 acquires, for each of a plurality of network policies, a likelihood based on the first policy determination model (likelihood of correct solution) and a likelihood based on the third policy determination model (likelihood during training). Then, the training policy determination unit 1425 trains the third policy determination model so that the likelihood during training approaches the correct likelihood.

[0407] 40 is a diagram illustrating an example of the functional configuration of the learning policy determination unit 1425 according to embodiment 13. The learning policy determination unit 1425 includes, for example, a correct answer likelihood acquisition unit 1425a, a third learning likelihood acquisition unit 1425b, and a fourth parameter update unit 1425c.

[0408] The correct likelihood obtaining unit 1425a inputs the quality of the randomly determined learning target image into the trained first policy determination model, and obtains the likelihood (vector) of each of the multiple network policies as the correct likelihood.

[0409] The third learning likelihood acquisition unit 1425b inputs learning target images according to the randomly determined quality into the third policy determination model, and obtains the likelihood (vector) of each network policy as the likelihood during learning.

[0410] The fourth parameter update unit 1425c updates the parameters (fourth parameters) of the third policy decision model based on the fourth loss so that the likelihood during training approaches the correct likelihood. The fourth loss is, for example, the error between the correct likelihood calculated for multiple network policies and the likelihood during training. For example, cross entropy (error), mean square error, mean absolute error (MAE), etc. may be used as the fourth loss.

[0411] In detail, for example, the fourth parameter update unit 1425c includes a fourth loss acquisition unit 1425c1 and a fourth parameter determination unit 1425c2.

[0412] The fourth loss acquiring unit 1425c1 acquires a fourth loss based on the correct likelihoods acquired for a plurality of network policies and the likelihoods being trained.

[0413] The fourth parameter determination unit 1425c2 updates the parameters of the third policy determination model based on the fourth loss.

[0414] The information processing device 1403 according to this embodiment may be physically configured in the same manner as the information processing device 802 (see FIG. 6 ). However, the storage device 1040 of the information processing device 1403 stores program modules for realizing the functions of the information processing device 1403. The network interface 1050 of the information processing device 1403 is an interface for connecting the information processing device 1403 to a network.

[0415] (Example of Operation of Information Processing System 1400 According to Embodiment 13) The information processing system 1400 executes information processing. The information processing according to this embodiment includes learning processing. The learning processing is, for example, processing for learning the third policy decision model and the neural network NN1.

[0416] In this embodiment, an example will be described in which the third policy decision model is learned separately from the neural network NN1. The learning processes of the fourth and fifth embodiments, for example, may be applied to the learning of the neural network NN1. Note that the third policy decision model may be applied instead of the first policy decision model described in the eighth embodiment, and the learning of the third policy decision model and the neural network NN1 may be executed simultaneously.

[0417] 41 is a flowchart showing an example of a third policy determination model learning process according to embodiment 13. For example, when the information processing device receives an instruction from a user, it starts the third policy determination model learning process.

[0418] For example, first, the process of step S801 is executed as in the eighth embodiment.

[0419] The correct likelihood obtaining unit 1425a inputs the quality of the learning target image randomly determined in step S801a into the trained first policy determination model, and obtains the correct likelihood of each of the multiple network policies (step S1401).

[0420] For example, the correct likelihood obtaining unit 1425a stores a trained first policy determination model in advance, and obtains the correct likelihood (vector) of each of the plurality of network policies by inputting the quality of the randomly determined learning target image into the first policy determination model stored in advance.

[0421] The third learning likelihood obtaining unit 1425b inputs the learning target image obtained in step S801b to the third policy decision model, and obtains the likelihood of each network policy during learning (step S1402).

[0422] For example, the third learning likelihood obtaining unit 1425b inputs the learning target image obtained in step S801b to the third policy decision model, thereby obtaining the likelihood during learning for each of the plurality of network policies.

[0423] The fourth parameter update unit 1425c updates the parameters (fourth parameters) of the third policy decision model so that the likelihood of correctness calculated in steps S1401 and S1402 approaches the likelihood during training (step S1403).

[0424] In more detail, for example, the fourth loss calculating unit 1425c1 calculates the fourth loss based on the correct likelihood calculated for the plurality of network policies and the likelihood during training (step S1403a), thereby calculating a fourth loss function for calculating the fourth loss.

[0425] The fourth parameter determination unit 1425c2 updates the parameters of the third policy determination model based on the fourth loss calculated in step S1403a (step S1403b), and ends the third policy determination model learning process.

[0426] For example, the fourth parameter determination unit 1425c2 may obtain a gradient based on the fourth loss function to obtain a fourth parameter used to update the third policy decision model. Then, the fourth parameter determination unit 1425c2 updates the parameters included in the third policy decision model using the obtained fourth parameter. The third policy decision model learning process may be repeatedly executed until a predetermined condition is satisfied.

[0427] (Operations and Effects) As described above, according to this embodiment, the information processing device 1403 further includes a learning policy determination unit 1425. The learning policy determination unit 1425 performs learning of a third policy determination model for determining a network policy according to a target image from among a plurality of predetermined network policies.

[0428] The learning policy determination unit 1425 includes a correct likelihood acquisition unit 1425a, a third learning likelihood acquisition unit 1425b, and a fourth parameter update unit 1425c.

[0429] The correct likelihood acquisition unit 1425a inputs quality information of the training target image into a trained first policy determination model for determining a network policy corresponding to the quality information of the target image from among a plurality of predetermined network policies, and calculates the likelihood of each network policy as the correct likelihood. The third training likelihood acquisition unit 1425b inputs the training target image into the third policy determination model, and calculates the likelihood of each network policy as the likelihood during training. The fourth parameter update unit 1425c updates the parameters of the third policy determination model based on the fourth loss so that the likelihood during training approaches the correct likelihood.

[0430] The fourth loss is a loss based on the error between the likelihood during training and the correct likelihood.

[0431] This allows a trained third policy decision model to be generated, and this third policy decision model can be used to determine a network policy according to the target image. Then, the processing in the neural network NN1 can be changed according to the determined network policy. Therefore, it is possible to perform authentication regardless of the quality of the target image while suppressing an increase in processing time.

[0432] Although the embodiments and modifications of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various configurations other than those described above can also be adopted.

[0433] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps performed in each embodiment is not limited to the order shown. In each embodiment, the order of steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments and variations can be combined as long as the content is not contradictory.

[0434] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0435] 1. An information processing system comprising: quality estimation means for estimating quality information of a target image; and image processing means for processing the target image using a neural network according to a network policy determined based on the quality information of the target image. 2. The information processing system described in 1., in which the image processing means extracts features from the target image using the neural network according to the network policy. 3. The information processing system described in 1. or 2., in which the target image is an eye image including an iris region in which an iris is captured, and the quality information includes an iris diameter. 4. The information processing system described in any one of 1. to 3., in which the quality information includes a resolution of the target image. 5. The information processing system described in any one of 1. to 4., further comprising policy determination means for determining the network policy based on the quality information. 6. The information processing system described in 5., in which the policy determination means determines the network policy so that the processing time using the neural network is within a predetermined range, regardless of the quality information of the target image. 7. The information processing system described in any one of 1. to 6., wherein the image processing means includes: first processing means that converts the iris region included in the target image into a standard image of a predetermined shape; and second processing means that processes the standard image using the neural network in accordance with the network policy, wherein the first processing means converts the iris region into the standard image of a size in accordance with the quality information. 8. The information processing system described in any one of 1. to 7., wherein the neural network is composed of a front-stage group including an input layer and a back-stage group including a final layer, and the network policy is applied to image processing using the front-stage group. 9. The information processing system described in 8., wherein the neural network includes a plurality of convolution layers, and the policy determination means determines the network policy such that the lower the resolution of the target image, the greater the number of convolution layers included in the front-stage group.10. The information processing system described in 8., wherein the neural network includes a plurality of convolution layers, and wherein the policy determination means determines the network policy so that the lower the resolution of the target image, the greater the number of channels in the calculations in the convolution layers included in the previous group. 11. The information processing system described in any one of 1. to 10., further comprising learning means for training the neural network by inputting a training target image to the neural network. 12. The information processing system described in 11., further comprising learning policy determination means for determining a network policy for the training target image based on quality information of the training target image and a training probability distribution, and wherein the learning means trains the neural network by inputting the training target image to the neural network according to the determined network policy, and the training probability distribution is a probability distribution that has the highest probability for the quality indicated by the quality information of the training target image and gradually becomes lower as the degree of difference from that quality increases. 13. 12. The information processing system according to 12, wherein the learning policy determination means includes: a first determination means that uses the random number to change quality information of the learning target images in accordance with a learning probability distribution and determines the learning quality information; and a second determination means that determines a network policy for the learning target images in accordance with the determined learning quality information. 14. The information processing system according to any one of 11. to 13, further comprising image selection means that selects learning target images such that lower quality learning target images are selected more frequently, and the learning means inputs the selected learning target images to the neural network to train the neural network.15. The information processing system according to 11, further comprising: a training policy determination means for training a first policy determination model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and a first loss acquisition means for determining a first loss when the training target image is processed using a neural network corresponding to each of the plurality of network policies, wherein the first policy determination model is constituted by a neural network, and the training policy determination means includes: a first training likelihood acquisition means for inputting quality information of the training target image to the first policy determination model and determining a likelihood of each network policy, and a second parameter update means for updating parameters of the first policy determination model based on the first loss and the likelihood of each network policy, and the second parameter update means includes: a second loss acquisition means for determining a second loss, which is the sum of first losses weighted by the likelihood for each network policy, and a second parameter update means for updating parameters of the first policy determination model based on the second loss.16. The system further comprises: a training policy determination means for training a second policy determination model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and a first loss acquisition means for determining a first loss when the training target image is processed using a neural network corresponding to the network policy determined using the second policy determination model, wherein the second policy determination model is composed of a plurality of policy probability distributions associated with the plurality of network policies, each of the plurality of policy probability distributions including one or more parameters, and the training policy determination means comprises: a second training likelihood acquisition means for determining the likelihood of each policy probability distribution using the quality information of the training target image and each of the plurality of policy probability distributions; and a third parameter update means for updating the parameters of each policy probability distribution based on the first loss and a modified loss for modifying the parameters of policy probability distributions corresponding to different network policies so that a difference between the parameters is increased, and the third parameter update means comprises: a third loss acquisition means for determining a third loss by adding the first loss and the modified loss; and a third parameter determination means for updating the parameters of each policy probability distribution based on the third loss. 11. The information processing system according to Item 11.17. The information processing system according to 11, further comprising: a training policy determination means for training a third policy determination model for determining the network policy corresponding to the target image from among a plurality of predetermined network policies, wherein the training policy determination means includes: a correct likelihood acquisition means for inputting quality information of the training target image into a trained first policy determination model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies, and determining the likelihood of each network policy as a correct likelihood; a third training likelihood acquisition means for inputting the training target image into the third policy determination model and determining the likelihood of each network policy as a likelihood during training; and a fourth parameter update means for updating parameters of the third policy determination model based on a fourth loss so that the likelihood during training approaches the correct likelihood, wherein the fourth loss is a loss based on an error between the likelihood during training and the correct likelihood. An information processing device comprising: quality estimation means for estimating quality information of a target image; and image processing means for processing the target image using a neural network in accordance with a network policy determined based on the quality information of the target image. 19. An information processing method including one or more computers estimating quality information of a target image, and processing the target image using a neural network in accordance with a network policy determined based on the quality information of the target image. 20. The information processing method described in 19., wherein processing the target image includes extracting features from the target image using the neural network in accordance with the network policy. 21. The information processing method described in 19. or 20., wherein the target image is an eye image including an iris region in which an iris is captured, and the quality information includes an iris diameter. 22. The information processing method described in any one of 19. to 21., wherein the quality information includes a resolution of the target image. 23. The information processing method described in any one of 19. to 22., further including determining the network policy based on the quality information.24. The information processing method according to 23., wherein determining the network policy determines the network policy so that the processing time using the neural network falls within a predetermined range regardless of quality information of the target image. 25. The information processing method according to any one of 19. to 24., wherein processing the target image includes converting the iris region included in the target image into a standard image of a predetermined shape, and processing the standard image using the neural network according to the network policy, and converting into a standard image includes converting the iris region into the standard image of a size according to the quality information. 26. The information processing method according to any one of 19. to 25., wherein the neural network is composed of a front-stage group including an input layer and a back-stage group including a final layer, and the network policy is applied to image processing using the front-stage group. 27. The neural network includes multiple convolution layers, and determining the network policy determines the network policy so that the lower the resolution of the target image, the greater the number of convolution layers included in the front-stage group. 26. 28. The information processing method according to 26., wherein the neural network includes a plurality of convolution layers, and determining the network policy determines the network policy such that the lower the resolution of the target image, the greater the number of channels in the calculations in the convolution layers included in the preceding group. 29. The information processing method according to any one of 19. to 28., further comprising inputting a training target image to the neural network to train the neural network.30. The information processing method described in 29., further comprising determining a network policy for the training target image based on quality information of the training target image and a training probability distribution, wherein training the neural network involves training the neural network by inputting the training target image into the neural network according to the determined network policy, and the training probability distribution is a probability distribution that has the highest probability for the quality indicated by the quality information of the training target image and gradually decreases as the degree of difference from that quality increases. 31. The information processing method described in 30., further comprising determining the network policy for the training target image by using the random number to change the quality information of the training target image in accordance with the training probability distribution and determining the training quality information, and determining the network policy for the training target image according to the determined training quality information. 32. 32. The information processing method according to any one of 29. to 31., further comprising selecting training target images such that more training target images with lower quality are selected, and training the neural network includes inputting the selected training target images into the neural network to train the neural network.33. The information processing method described in 29., further comprising: training a first policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and determining a first loss when the training target image is processed using a neural network corresponding to each of the plurality of network policies, wherein the first policy decision model is composed of a neural network; training the first policy decision model includes inputting quality information of the training target image to the first policy decision model to determine a likelihood of each network policy, and updating parameters of the first policy decision model based on the first loss and the likelihood of each network policy, and updating the parameters of the first policy decision model includes determining a second loss, which is the sum of first losses weighted by the likelihood for each network policy, and updating the parameters of the first policy decision model based on the second loss.34. The information processing method according to 29., further comprising: training a second policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and determining a first loss when the training target image is processed using a neural network corresponding to the network policy determined using the second policy decision model, wherein the second policy decision model is composed of a plurality of policy probability distributions associated with the plurality of network policies, each of the plurality of policy probability distributions including one or more parameters; training the second policy decision model includes determining the likelihood of each policy probability distribution using the quality information of the training target image and each of the plurality of policy probability distributions; updating the parameters of each policy probability distribution based on the first loss and a correction loss for correcting so that a difference between parameters of policy probability distributions corresponding to different network policies becomes larger; and updating the parameters of each policy probability distribution includes: determining a third loss by adding the first loss and the correction loss; and updating the parameters of each policy probability distribution based on the third loss. 29. The information processing method according to 29, further comprising training a third policy decision model for determining the network policy corresponding to the target image from among a plurality of predetermined network policies, wherein training the third policy decision model comprises: inputting quality information of the training target image into a trained first policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies to obtain a likelihood of each network policy as a correct likelihood; inputting the training target image into a third policy decision model to obtain a likelihood of each network policy as a likelihood during training; and updating parameters of the third policy decision model based on a fourth loss so that the likelihood during training approaches the correct likelihood, wherein the fourth loss is a loss based on an error between the likelihood during training and the correct likelihood.36. A recording medium having recorded thereon a program for causing one or more computers to estimate quality information of a target image, and process the target image using a neural network according to a network policy determined based on the quality information of the target image. 37. The recording medium described in 36., having recorded thereon a program for extracting features from the target image using the neural network according to the network policy in processing the target image. 38. The recording medium described in 36. or 37., having recorded thereon a program for causing the target image to be an eye image including an iris region in which an iris is captured, and for the quality information to include iris diameter. 39. The recording medium described in any one of 36. to 38., having recorded thereon a program for causing the quality information to include resolution of the target image. 40. The recording medium described in any one of 36. to 39., having recorded thereon a program for further causing the network policy to be determined based on the quality information. 41. The recording medium described in 40., on which a program is recorded for determining the network policy so that the processing time using the neural network falls within a predetermined range regardless of quality information of the target image. 42. The recording medium described in any one of 36. to 41., on which a program is recorded for processing the target image: converting the iris region included in the target image into a standard image of a predetermined shape, and processing the standard image using the neural network according to the network policy, and converting into the standard image: converting the iris region into the standard image of a size according to the quality information. 43. The recording medium described in any one of 36. to 42., on which a program is recorded for applying the network policy to image processing using the standard group, the neural network being composed of a front-stage group including an input layer and a back-stage group including a final layer.44. The recording medium described in 43., wherein the neural network includes a plurality of convolution layers, and a program for determining the network policy is recorded on the recording medium such that the lower the resolution of the target image, the greater the number of convolution layers included in the preceding group. 45. The recording medium described in 43., wherein the neural network includes a plurality of convolution layers, and a program for determining the network policy is recorded on the recording medium such that the lower the resolution of the target image, the greater the number of channels used in calculations in the convolution layers included in the preceding group. 46. The recording medium described in any one of 36. to 45., wherein a program is recorded on the recording medium such that the neural network is trained by inputting a training target image into the neural network. 47. 46. ​​The recording medium described in 46., on which a program is recorded for further comprising determining a network policy for the training target image based on quality information of the training target image and a training probability distribution, wherein training the neural network involves inputting the training target image into the neural network according to the determined network policy, and the training probability distribution is a probability distribution that has the highest probability for the quality indicated by the quality information of the training target image and gradually decreases as the degree of difference from that quality increases. 48. The recording medium described in 47., on which a program is recorded for determining a network policy for the training target image, wherein determining the training quality information involves changing the quality information of the training target image according to the training probability distribution by using the random number, and determining the training quality information, and determining a network policy for the training target image according to the determined training quality information.49. The recording medium according to any one of 46. to 48., further comprising selecting training target images such that more training target images of lower quality are selected, and wherein training the neural network includes inputting the selected training target images into the neural network, thereby training the neural network. 50. 46. ​​The recording medium described in 46., further comprising: training a first policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and determining a first loss when the training target image is processed using a neural network corresponding to each of the plurality of network policies, wherein the first policy decision model is composed of a neural network; training the first policy decision model includes inputting quality information of the training target image to the first policy decision model to determine a likelihood of each network policy; updating parameters of the first policy decision model based on the first loss and the likelihood of each network policy; and updating the parameters of the first policy decision model includes determining a second loss, which is the sum of first losses weighted by the likelihood for each network policy, and updating the parameters of the first policy decision model based on the second loss.51. The recording medium described in 46., on which a program for: determining a third loss by adding the first loss and the correction loss to modify the parameter of each policy probability distribution; and updating the parameter of each policy probability distribution based on the third loss. 52. The recording medium described in 46., further comprising: training a second policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies; and determining a first loss when the training target image is processed using a neural network corresponding to the network policy determined using the second policy decision model; the second policy decision model being composed of a plurality of policy probability distributions associated with the plurality of network policies, each of the plurality of policy probability distributions including one or more parameters; and training the second policy decision model: determining the likelihood of each policy probability distribution using the quality information of the training target image and each of the plurality of policy probability distributions; updating the parameter of each policy probability distribution based on the first loss and a correction loss for modifying the parameter of each policy probability distribution so that a difference between the parameters of the policy probability distributions corresponding to different network policies becomes larger; and updating the parameter of each policy probability distribution: determining a third loss by adding the first loss and the correction loss; and updating the parameter of each policy probability distribution based on the third loss. 46. ​​The recording medium described in 45, further comprising: training a third policy decision model for determining the network policy corresponding to the target image from among a plurality of predetermined network policies, wherein training the third policy decision model includes inputting quality information of the training target image into a trained first policy decision model for determining the network policy corresponding to quality information of the target image from among a plurality of predetermined network policies to obtain a likelihood of each network policy as a correct likelihood; inputting the training target image into a third policy decision model to obtain a likelihood of each network policy as a likelihood during training; and updating parameters of the third policy decision model based on a fourth loss so that the likelihood during training approaches the correct likelihood, wherein the fourth loss is a loss based on an error between the likelihood during training and the correct likelihood.

[0436] This application claims priority based on PCT / JP2022 / 041157, filed November 4, 2022, the disclosure of which is incorporated herein in its entirety.

[0437] 100, 400, 500, 600 Information processing system 101 Imaging device 102, 403, 503, 603 Information processing device 111 Quality estimation unit 112 Policy determination unit 113 Image processing unit 113a First processing unit 113b Second processing unit 114 Authentication unit 421, 521 Learning memory unit 422 Learning unit 423 Learning policy determination unit 423a First determination unit 423b Second determination unit 524 Image selection unit 525 Learning policy determination unit NN1 to NN3 Neural network

Claims

1. Quality estimation means for estimating the quality information of the target image, and image processing means for processing the target image using a neural network according to a network policy determined based on the quality information of the target image. An information processing system.

2. The image processing means extracts feature amounts from the target image using the neural network according to the network policy. The information processing system according to Claim 1.

3. The target image is an eye image including an iris region in which the iris is reflected, and the quality information includes an iris diameter. The information processing system according to Claim 1 or 2.

4. The information processing system further includes learning means for performing learning of the neural network by inputting a learning target image into the neural network. The information processing system according to Claim 1 or 2.

5. The information processing system further includes learning policy determination means for determining a network policy for the learning target image based on the quality information of the learning target image and a learning probability distribution, wherein the learning means performs learning of the neural network by inputting the learning target image into the neural network according to the determined network policy, and the learning probability distribution is a probability distribution having the highest probability at the quality indicated by the quality information of the learning target image and gradually decreasing probabilities as the degree of difference from the quality increases. The information processing system according to Claim 4.

6. The learning policy determination means includes first determination means for determining learning quality information by changing the quality information of the learning target image according to the learning probability distribution using a random number, and second determination means for determining a network policy for the learning target image according to the determined learning quality information. The information processing system according to Claim 5.

7. Learning policy determination means for performing learning of a first policy determination model for determining the network policy according to the quality information of the target image from among a plurality of predetermined network policies, and first loss acquisition means for obtaining a first loss when processing the learning target image using a neural network according to each of the plurality of network policies. The first policy determination model is composed of a neural network. The learning policy determination means First likelihood acquisition means for obtaining the likelihood of each network policy by inputting the quality information of the learning target image into the first policy determination model; Second parameter update means for updating the parameters of the first policy determination model based on the first loss and the likelihood of each network policy; The second parameter update means includes: Second loss acquisition means for obtaining a second loss which is the sum of the first losses weighted by the likelihood for each network policy; Second parameter update means for updating the parameters of the first policy determination model based on the second loss; The information processing system according to claim 4.

8. The learning policy determination means for further training a third policy determination model for determining the network policy corresponding to the target image from a plurality of predetermined network policies; The learning policy determination means includes: Correct likelihood acquisition means for inputting the quality information of the learning target image into a trained first policy determination model for determining the network policy corresponding to the quality information of the target image from a plurality of predetermined network policies, and obtaining the likelihood of each network policy as the correct likelihood; Third learning likelihood acquisition means for inputting the learning target image into the third policy determination model and obtaining the likelihood of each network policy as the likelihood during learning; Fourth parameter update means for updating the parameters of the third policy determination model based on the fourth loss so that the likelihood during learning approaches the correct likelihood; The fourth loss is a loss based on the error between the likelihood during learning and the correct likelihood. The information processing system according to claim 4.

9. One or more computers perform: Estimating the quality information of the target image; Processing the target image using a neural network corresponding to a network policy determined based on the quality information of the target image. An information processing method.

10. A program for causing one or more computers to: Estimate the quality information of the target image; Process the target image using a neural network corresponding to a network policy determined based on the quality information of the target image.