Information processing device, information processing method, and recording medium

Abstract

This information processing device comprises: an acquisition means for acquiring a plurality of images including a user's face; a normalization means for normalizing each of the plurality of images on the basis of the user's face; an integration means for generating an integrated image by integrating the plurality of normalized images; and an extraction means for extracting a feature quantity related to the user's face from the integrated image. According to this information processing device, it is possible to appropriately extract a feature quantity from a plurality of images.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Known examples of this type of device include one that authenticates a user by matching a facial image. For example, Patent Document 1 discloses a technology that calculates integrated facial feature points based on a plurality of facial feature point candidates detected from a facial image, and performs face authentication based on the calculated integrated facial feature points.

[0003] Japanese Patent Application Laid-Open No. 2021-064424

[0004] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques disclosed in prior art documents.

[0005] One aspect of the information processing device disclosed herein comprises an acquisition means for acquiring a plurality of images including a user's face, a normalization means for normalizing each of the plurality of images based on the user's face, an integration means for integrating the normalized plurality of images to generate an integrated image, and an extraction means for extracting features related to the user's face from the integrated image.

[0006] One aspect of the information processing method disclosed herein involves using at least one computer to acquire a plurality of images including a user's face, normalizing each of the plurality of images based on the user's face, integrating the normalized plurality of images to generate an integrated image, and extracting features related to the user's face from the integrated image.

[0007] One aspect of the recording medium of this disclosure is a computer program recorded on at least one computer that causes the computer to execute an information processing method, which includes acquiring a plurality of images including a user's face, normalizing each of the plurality of images based on the user's face, integrating the normalized plurality of images to generate an integrated image, and extracting features related to the user's face from the integrated image.

[0008] 1 is a block diagram showing the hardware configuration of a first information processing device. FIG. 2 is a block diagram showing the functional configuration of the first information processing device. FIG. 3 is a flowchart showing the flow of operation of the first information processing device. FIG. 4 is a block diagram showing the functional configuration of a second information processing device. FIG. 5 is a flowchart showing the flow of operation of the second information processing device. FIG. 6 is a graph showing an example of changing weights according to quality values ​​in the second information processing device. FIG. 7 is a plan view showing an example of determining weights for each pixel in the second information processing device. FIG. 8 is a plan view showing an example of changing weights according to tracking results in the second information processing device. FIG. 9 is a table showing an example of changing weight ranges according to attribute estimation accuracy in the second information processing device. FIG. 10 is a block diagram showing the functional configuration of a third information processing device. FIG. 11 is a flowchart showing the flow of registration operations by the third information processing device. FIG. 12 is a flowchart showing the flow of authentication operations by the third information processing device.

[0009] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.

[0010] First Embodiment A first information processing apparatus will be described with reference to FIGS. 1 to 3. FIG.

[0011] (Hardware Configuration) First, the hardware configuration of the first information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the hardware configuration of the first information processing apparatus.

[0012] 1, a first information processing device 10 includes a processor 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, and a storage device 14. The information processing device 10 may further include an input device 15 and an output device 16. The processor 11, RAM 12, ROM 13, storage device 14, input device 15, and output device 16 are connected to each other via a data bus 17. The data bus 17 may be an interface other than a data bus (for example, a LAN, a USB, etc.).

[0013] The processor 11 loads a computer program. For example, the processor 11 is configured to load a computer program stored in at least one of the RAM 12, the ROM 13, and the storage device 14. Alternatively, the processor 11 may load a computer program stored in a computer-readable storage medium using a storage medium reading device (not shown). The processor 11 may acquire (i.e., load) the computer program from a device (not shown) located outside the information processing device 10 via a network interface. The processor 11 controls the RAM 12, the storage device 14, the input device 15, and the output device 16 by executing the loaded computer program. In particular, in this embodiment, when the processor 11 executes the loaded computer program, a functional block for extracting features from multiple images is realized within the processor 11. In other words, the processor 11 may function as a controller that executes each control in the information processing device 10.

[0014] The processor 11 may be configured as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or a quantum processor. The processor 11 may be configured as one of these, or may be configured to use multiple processors in parallel.

[0015] The RAM 12 temporarily stores computer programs executed by the processor 11. The RAM 12 temporarily stores data that the processor 11 temporarily uses while it is executing the computer programs. The RAM 12 may be, for example, a dynamic random access memory (D-RAM) or a static random access memory (SRAM). Alternatively, other types of volatile memory may be used instead of the RAM 12.

[0016] The ROM 13 stores computer programs executed by the processor 11. The ROM 13 may also store fixed data. The ROM 13 may be, for example, a programmable read-only memory (PROM) or an erasable read-only memory (EPROM). Alternatively, other types of non-volatile memory may be used instead of the ROM 13.

[0017] The storage device 14 stores data that the information processing device 10 stores for a long period of time. The storage device 14 may operate as a temporary storage device for the processor 11. The storage device may store computer programs executed by the processor 11. The storage device 14 may include, for example, at least one of a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device.

[0018] The input device 15 is a device that receives input instructions from a user of the information processing device 10. The input device 15 may include, for example, at least one of a keyboard, a mouse, and a touch panel. The input device 15 may be configured as part of a smartphone, a tablet terminal, an earphone-type terminal, a watch-type terminal, an HMD (Head Mounted Display) terminal, etc. The input device 15 may be, for example, a device that includes a microphone and is capable of voice input.

[0019] The output device 16 is a device that outputs information related to the information processing device 10 to the outside. For example, the output device 16 may be a display device (e.g., a display or digital signage) that can display information related to the information processing device 10. The output device 16 may also be a speaker or the like that can output information related to the information processing device 10 as audio.

[0020] 1 may be configured to be included in a device external to the first information processing device 10. For example, the first information processing device 10 may be configured to include a processor 11, a RAM 12, and a ROM 13, and the other devices, such as a storage device 14, an input device 15, and an output device 16, may be configured as external devices. That is, the first information processing device 10 may be configured as an information processing system including a plurality of different devices. Furthermore, some of the calculation functions of the first information processing device 10 may be realized by an external server, a cloud, or the like.

[0021] (Functional Configuration) Next, the functional configuration of the first information processing device 10 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the functional configuration of the first information processing device.

[0022] 2, the first information processing device 10 is configured as a device that extracts features related to a user's face from multiple images including the user's face. The first information processing device 10 may be configured as part of an authentication device that performs face authentication using features extracted from facial images, for example. The first information processing device 10 is configured to include an image acquisition unit 110, a normalization unit 120, an image integration unit 130, and a feature extraction unit 140 as processing blocks for realizing its functions. Note that each of the image acquisition unit 110, the normalization unit 120, the image integration unit 130, and the feature extraction unit 140 may be realized, for example, by the above-mentioned processor 11 (see FIG. 1).

[0023] The image acquisition unit 110 is configured to be able to acquire multiple images including the user's face. The image acquisition unit 110 may acquire the multiple images as a video. The image acquisition unit 110 may acquire the multiple images by, for example, photographing the user with a camera. In this case, the image acquisition unit 110 may capture the images in response to the user's photographing operation. Alternatively, the image acquisition unit 110 may automatically capture images when a moving user reaches a predetermined position (for example, within the imaging range of a camera installed at a gate, entrance, etc.). The image acquisition unit 110 may be configured to acquire images from multiple cameras. The multiple images acquired by the image acquisition unit 110 are configured to be output to the normalization unit 120.

[0024] The normalization unit 120 is configured to perform normalization processing on each of the multiple images acquired by the image acquisition unit 110. Specifically, the normalization unit 120 performs normalization processing based on the user's face included in each of the multiple images. For example, the normalization unit 120 performs processing to align the size and position of the user's face included in the multiple images. Note that the specific method of the normalization processing is not particularly limited, and existing technology may be adopted as appropriate. The normalization unit 120 may have a function to detect the user's face from the multiple images. In this case, the normalization unit 120 may detect the user's face from each of the multiple images and perform normalization processing based on the detected face. The multiple images normalized by the normalization unit 120 are configured to be output to the image integration unit 130.

[0025] The image integration unit 130 is configured to integrate multiple images normalized by the normalization unit 120 to generate an integrated image. That is, the image integration unit 130 integrates multiple images to generate a single integrated image. The image integration unit 130 may be configured to, for example, perform a process of averaging multiple images. Alternatively, the image integration unit 130 may be configured to perform a process of calculating a weighted average by setting different weights for each of the multiple images. The configuration for calculating the weighted average will be described in detail in another embodiment described later. The integrated image generated by the image integration unit 130 includes a face (e.g., an averaged face) obtained by integrating the user's faces in each of the multiple images. The integrated image generated by the image integration unit 130 is configured to be output to the feature extraction unit 140.

[0026] The feature extraction unit 140 is configured to extract features related to the user's face from the integrated image generated by the image integration unit 130. The extracted features may be parameters that can be used for face authentication, for example. The feature extraction unit 140 may be configured to extract features using an extraction model configured with a neural network such as a convolutional neural network (CNN). This extraction model may be trained to extract features from normal face images (i.e., non-integrated face images), or may be trained to extract features from integrated images (i.e., a model trained for integrated images). The extraction model for integrated images may be trained using training data including integrated images, for example.

[0027] (Flow of Operation) Next, the flow of operation in the first information processing device 10 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of operation of the first information processing device.

[0028] 3, when the operation of the first information processing device 10 starts, the image acquisition unit 110 first acquires a plurality of images including a user's face (step S101). Then, the normalization unit 120 performs normalization processing on each of the plurality of images acquired by the image acquisition unit 110 (step S102).

[0029] Next, the image integration unit 130 integrates the multiple images normalized by the normalization unit 120 to generate an integrated image (step S103). Then, the feature extraction unit 140 extracts features from the integrated image generated by the image integration unit 130 (step S104).

[0030] (Technical Effects) Next, technical effects obtained by the first information processing device 10 will be described.

[0031] As described with reference to FIGS. 1 to 3 , the first information processing device 10 generates an integrated image by integrating multiple images, and extracts features from the integrated image. This allows for appropriate feature extraction from multiple images. Alternatively, a method for extracting features from multiple images may involve extracting features from each of the multiple images and integrating the extracted features. However, extracting features from each of the multiple images requires performing a feature extraction process for each image (i.e., performing a feature extraction process for each image), which may increase the processing load and processing time. However, the first information processing device 10 only requires performing a feature extraction process for the integrated image obtained by integrating multiple images. Therefore, the first information processing device 10 allows for efficient feature extraction without increasing the processing load and processing time.

[0032] Second Embodiment A second information processing device 10 will be described with reference to Figures 4 to 9. The second information processing device 10 differs in some configurations and operations from the first information processing device 10 described above, but other parts may be similar to the first information processing device 10. Therefore, hereinafter, parts that differ from the first embodiment will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate.

[0033] (Functional Configuration) First, the functional configuration of the second information processing device 10 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the functional configuration of the second information processing device. Note that in Fig. 4, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0034] 4, the second information processing device 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, a normalization unit 120, an image integration unit 130, a feature extraction unit 140, and an integration parameter determination unit 150. That is, the second information processing device 10 is configured to further include, in addition to the configuration of the first information processing device (see FIG. 2), an integration parameter determination unit 150. The integration parameter determination unit 150 may be realized by, for example, the above-mentioned processor 11 (see FIG. 1).

[0035] The integration parameter determination unit 150 is configured to be able to determine an integration parameter for each of a plurality of images. The integration parameter is a weight used when integrating a plurality of images (hereinafter, the integration parameter may be referred to as a "weight" as appropriate). The plurality of images are integrated, for example, at an integration ratio according to the weight. The integration parameter determination unit 150 may determine an integration parameter for each image. For example, the integration parameter determination unit 150 determines an integration parameter for image A, an integration parameter for image B, and an integration parameter for image C.

[0036] The integrated parameter determination unit 150 determines integrated parameters based on the attributes of each of the multiple images. That is, the integrated parameter determination unit 150 determines different weights for each of the multiple images according to their respective attributes. Here, "attributes" is a general term for various information related to an image. Attributes may be estimated, for example, by analyzing the image. In this case, the integrated parameter determination unit 150 may have a function for estimating attributes from the image. An example of an attribute is an image quality value. Here, the image quality value may be, for example, a value indicating the quality of the image itself (e.g., parameters related to image resolution, brightness, etc.), or a value indicating the quality of a specific object included in the image (e.g., parameters related to the facial resemblance, facial angle, facial size, etc.) of a face included in the image. Another example of an attribute is the tracking result of an object included in an image. Here, the tracking result may be, for example, information indicating the tracking result of a specific user included in multiple images captured in chronological order using the user's appearance information. Specific processing examples using attributes will be described in detail later.

[0037] (Operation Flow) Next, the operation flow in the second information processing device 10 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the operation flow of the second information processing device. Note that in Fig. 5, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0038] 5, when the operation of the second information processing device 10 starts, the image acquisition unit 110 first acquires a plurality of images including a user's face (step S101). Then, the normalization unit 120 performs normalization processing on each of the plurality of images acquired by the image acquisition unit 110 (step S102).

[0039] On the other hand, the integration parameter determination unit 150 determines integration parameters based on the attributes of each of the multiple images (step S201). Note that the integration parameter determination unit 150 may determine the integration parameters using the processing results of the normalization unit 120. For example, the integration parameter determination unit 150 may determine the integration parameters using the results of face detection in the normalization unit 120. Alternatively, the integration parameter determination unit 150 may determine the integration parameters using the images normalized by the normalization unit 120.

[0040] Next, the image integration unit 130 integrates the multiple images normalized by the normalization unit 120 to generate an integrated image (step S202). At this time, the image integration unit 130 integrates the multiple images using the integration parameters determined by the integration parameter determination unit 150. Specifically, the image integration unit 130 executes a process of calculating a weighted average of the multiple images using weights indicated by the integration parameters. Thereafter, the feature extraction unit 140 extracts features from the integrated image generated by the image integration unit 130 (step S104).

[0041] (Example of Using Quality Values) Next, an example of determining integration parameters using quality values ​​of images will be described with reference to Fig. 6. Fig. 6 is a graph showing an example of changing weights according to quality values ​​in the second information processing apparatus.

[0042] As shown in Figure 6, the second information processing device 10 may determine integration parameters (weights) based on image quality values. Specifically, as shown in Figure 6(a), the integration parameters may be determined using a linear model. In this case, the image quality values ​​are used directly as weights for integration. Alternatively, as shown in Figure 6(b), the integration parameters may be determined using a sigmoid model. In this case, the weights can be changed significantly, for example, at a threshold value previously specified by the user.

[0043] The integrated parameter may be calculated using an arbitrary function F as shown in the following formula (1).

[0044] In addition, q in formula (1) Ａ and q Ｂare the quality values ​​of images A and B, respectively. Ａ and w Ｂ are the weights of images A and B, respectively.

[0045] By determining integration parameters based on the quality values ​​of the images, an integrated image can be generated taking the quality values ​​of the images into consideration. Specifically, by increasing the weight of images with high quality values ​​and decreasing the weight of images with low quality values, multiple images can be integrated so that the influence of images with high quality values ​​is increased. Note that the quality values ​​of the images may be estimated by analyzing the images. For example, the quality values ​​of the images may be estimated based on the appearance of faces in the images (e.g., facial clarity, facial size, etc.).

[0046] (Determining Weights for Each Pixel) Next, an example of determining an integration parameter for each pixel in an image will be described with reference to Fig. 7. Fig. 7 is a plan view showing an example of determining weights for each pixel in the second information processing apparatus.

[0047] As shown in FIG. 7 , the second information processing device 10 may determine an integration parameter for each pixel constituting an image. Specifically, the integration parameter determination unit 150 may estimate attributes for each pixel and determine a weight for each pixel. The integration parameter determination unit 150 may, for example, estimate a quality value for each pixel and determine a weight for each pixel according to the quality value. In this case, a pixel with a high quality value may be weighted higher, while a pixel with a low quality value may be weighted lower. The integration parameter determination unit 150 may also calculate a score indicating face-likeness for each pixel and determine a weight according to the score. In this case, a pixel with a high score may be weighted higher, while a pixel with a low score may be weighted lower. A configuration using a score indicating face-likeness is effective, for example, when a face is hidden by a mask or when part of the face is not visible due to the direction of the face.

[0048] By estimating attributes for each pixel, it is possible to determine integration parameters for each pixel. Therefore, the degree of influence on the integrated image can be varied depending on the pixel. In this way, it is possible to set more detailed integration conditions than, for example, when determining integration parameters for each image. While the example of estimating attributes on a pixel-by-pixel basis has been given here, attributes may be estimated on other units. For example, attributes may be estimated on a block-by-block basis, each including a plurality of pixels. Alternatively, attributes may be estimated on a part-by-part basis that constitutes an object included in the image (e.g., the eyes, nose, and mouth that constitute a face). The unit for estimating attributes may be set in advance. Alternatively, multiple types of units may be prepared in advance, and an appropriate unit may be selected from these to estimate attributes. The unit for estimating attributes may be selected by the user, or the device may analyze the image and automatically select the unit. For example, if it is determined that determining integration parameters on a block-by-block basis is appropriate as a result of analyzing the image, attributes may be estimated on a block-by-block basis. If it is determined that determining integration parameters on a part-by-part basis is appropriate, attributes may be estimated on a part-by-part basis.

[0049] (Example of Using Tracking Results) Next, an example of determining integrated parameters using tracking results of a user will be described with reference to Fig. 8. Fig. 8 is a plan view showing an example of changing weights according to tracking results in the second information processing apparatus.

[0050] As shown in FIG. 8 , the second information processing device 10 may determine integration parameters based on the results of tracking users included in images. Specifically, the integration parameter determination unit 150 executes a process of tracking a specific user in multiple images acquired in time series. The specific method of the tracking process is not particularly limited, and, for example, appearance information of users may be stored and users matching the appearance information may be tracked. The integration parameter determination unit 150 determines integration parameters for each user included in the images. For example, the integration parameter determination unit 150 may determine integration parameters such that a weight for a user currently being tracked is higher than weights for other users. More specifically, the integration parameter determination unit 150 assigns a weight of "1" to a user currently being tracked. Furthermore, the integration parameter determination unit 150 assigns a weight of "0" to a user not currently being tracked. In this case, only the faces of users currently being tracked are subject to integration, and the faces of users not currently being tracked are excluded.

[0051] By determining the integration parameters based on the tracking results, it is possible to appropriately integrate images even when multiple users are included in the image. Specifically, it is possible to prevent the generation of an inappropriate integrated image (i.e., an integrated image in which the faces of different users are mixed) by integrating the faces of different users.

[0052] (Adjustment According to Estimation Accuracy) Next, an example of adjusting the integrated parameter according to the accuracy of attribute estimation will be described with reference to Fig. 9. Fig. 9 is a table showing an example of changing the weight range according to the estimation accuracy of the attribute in the second information processing device.

[0053] As shown in FIG. 9 , the second information processing device 10 may adjust the integration parameter based on the attribute estimation accuracy. The adjustment of the integration parameter may, for example, limit the range of the integration parameter. In this case, the range of the integration parameter may be adjusted to be narrower as the estimation accuracy becomes poorer. For example, when the attribute estimation accuracy is high, the weight is determined to be in the range of "0.0 to 1.0." When the attribute estimation accuracy is normal, the weight is determined to be in the range of "0.4 to 1.0." When the attribute estimation accuracy is poor, the weight is determined to be in the range of "0.7 to 1.0." Note that the attribute estimation accuracy may be set in advance depending on the type of means for estimating the attribute (e.g., an attribute estimator), etc.

[0054] By adjusting the integration parameters based on the accuracy of attribute estimation, it is possible to prevent inappropriate integration parameters from being determined due to poor attribute estimation accuracy. For example, when the accuracy of attribute estimation is poor, the range of the integration parameters is significantly limited. In this way, the range of weights used for integration is narrowed, thereby preventing inappropriate integration from being performed based on erroneously estimated attributes.

[0055] The above-described examples may be combined. That is, the configuration for determining an integrated parameter based on a quality value (see FIG. 6 ), the configuration for determining an integrated parameter for each pixel (see FIG. 7 ), the configuration for determining an integrated parameter based on a tracking result (see FIG. 8 ), and the configuration for adjusting an integrated parameter based on the accuracy of attribute estimation (see FIG. 9 ) may be implemented in an appropriate combination. For example, a quality value may be calculated for each image, and a quality value may be calculated for each pixel included in the image, and the integrated parameter may be determined by multiplying these quality values ​​together. Alternatively, the integrated parameter may be determined based on the tracking result, and then the determined value may be adjusted based on the accuracy of attribute estimation.

[0056] (Technical Effects) Next, technical effects obtained by the second information processing device 10 will be described.

[0057] As described with reference to Figures 4 to 9, in the second information processing device 10, integration parameters are determined based on the attributes of the images, and multiple images are integrated based on the integration parameters. In this way, multiple images can be integrated more appropriately. For example, when integration is performed without using integration parameters (e.g., when simply averaging), the influence of each image on the integrated image is uniform. On the other hand, when integration is performed using integration parameters, the influence on the integrated image varies depending on the attributes of each image, allowing for a more appropriate generation of the integrated image. As a result, it is possible to extract appropriate features from the integrated image.

[0058] <Third Embodiment> A third information processing device 10 will be described with reference to Figures 10 to 12. The third information processing device 10 differs in some configurations and operations from the first and second information processing devices 10 described above, but other parts may be similar to the first and second information processing devices 10. Therefore, the following will describe in detail the parts that differ from the embodiments already described, and will omit explanations of other overlapping parts as appropriate.

[0059] (Functional Configuration) First, the functional configuration of the third information processing device 10 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the functional configuration of the third information processing device. Note that in Fig. 10, the same elements as those described in Fig. 2 are denoted by the same reference numerals.

[0060] 10 , the third information processing apparatus 10 is configured to include, as processing blocks for realizing its functions, an image acquisition unit 110, a normalization unit 120, an image integration unit 130, a feature extraction unit 140, a feature extraction unit 160, a feature integration unit 170, a registration unit 180, and an authentication unit 190. That is, the third information processing apparatus 10 is configured to further include, in addition to the configuration of the first information processing apparatus (see FIG. 2 ), the feature extraction unit 160, the feature integration unit 170, the registration unit 180, and the authentication unit 190. Each of the feature extraction unit 160, the feature integration unit 170, the registration unit 180, and the authentication unit 190 may be realized, for example, by the above-described processor 11 (see FIG. 1 ).

[0061] The feature extraction unit 160 is configured to be able to extract features separately for each of the multiple images acquired by the image acquisition unit 110. That is, while the feature extraction unit 130, which has already been described, extracts one feature from an integrated image, the feature extraction unit 160 extracts multiple feature values ​​from multiple images. The feature extraction unit 160 may be configured to extract features using an extraction model formed by a neural network. The feature extraction unit 160 may also be configured to extract features using an extraction model common to the feature extraction unit 130. The features extracted by the feature extraction unit 160 are configured to be output to the feature integration unit 170.

[0062] The feature integration unit 170 is configured to integrate multiple feature quantities extracted by the feature extraction unit 160. That is, the feature integration unit 170 is configured to integrate multiple feature quantities extracted from multiple images to generate a single integrated feature quantity. The feature integration unit 170 may be configured to, for example, perform a process of averaging multiple feature quantities. Alternatively, the feature integration unit 170 may be configured to perform a process of calculating a weighted average by setting different weights for each of the multiple feature quantities. The integrated feature quantity generated by the feature integration unit 170 is configured to be output to the registration unit 180.

[0063] The registration unit 180 is configured to be able to register (store) the integrated feature generated by the feature integration unit 170 as a registered feature corresponding to a registered user. The registration unit 180 may store a plurality of registered features corresponding to a plurality of registered users. The registered feature registered in the registration unit 180 can be read out by the authentication unit 190 as appropriate.

[0064] The authentication unit 190 is configured to be able to perform face authentication. Specifically, the authentication unit 190 compares features extracted from an authenticated user who is the authentication target with registered features registered in the registration unit 180. The features of the authenticated user are features extracted from an integrated image by the feature extraction unit 140. On the other hand, the registered features are integrated features integrated by the feature integration unit 170. The authentication unit 190 may determine that the authenticated user is a registered user when the degree of match between the features extracted from the authenticated user and the registered features exceeds a predetermined value.

[0065] The authentication unit 190 may have a function to output the result of facial authentication. For example, the authentication unit 190 may be configured to be able to display the result of facial authentication on a display or the like. The authentication unit 190 may also have a function to execute various processes depending on the result of facial authentication. For example, the authentication unit 190 may execute a process to allow the authenticated user to pass through the gate if facial authentication is successful. The authentication unit 190 may also execute a process to prohibit the authenticated user from passing through the gate or to output an alarm if facial authentication fails.

[0066] (Registration Operation) Next, the flow of the registration operation (i.e., the operation when registering the feature amount of a registered user) by the third information processing device 10 will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the registration operation by the third information processing device.

[0067] 11 , when the registration operation is started in the third information processing device 10, the image acquisition unit 110 first acquires a plurality of images including the face of the registered user (step S301). Then, the feature extraction unit 160 extracts a feature from each of the plurality of images acquired by the image acquisition unit 110 (step S302).

[0068] Next, the feature integration unit 170 integrates the multiple feature amounts extracted by the feature extraction unit 160 to generate an integrated feature amount (step S303). Then, the registration unit 180 registers the integrated feature amount generated by the feature integration unit 170 as a registered feature amount corresponding to the registered user (step S304).

[0069] (Authentication Operation) Next, the flow of authentication operation (i.e., operation when authenticating a user) by the third information processing device 10 will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of authentication operation by the third information processing device. Note that in Fig. 12, the same processes as those described in Fig. 3 are denoted by the same reference numerals.

[0070] 12, when the authentication operation is started in the third information processing device 10, the image acquisition unit 110 first acquires a plurality of images including the face of the authenticated user (step S101). Then, the normalization unit 120 performs normalization processing on each of the plurality of images acquired by the image acquisition unit 110 (step S102).

[0071] Next, the image integration unit 130 integrates the multiple images normalized by the normalization unit 120 to generate an integrated image (step S103). Then, the feature extraction unit 140 extracts features from the integrated image generated by the image integration unit 130 (step S104).

[0072] Next, the authentication unit 190 reads out the registered features registered in the registration unit 180 (step S351). Then, the feature of the authenticated user extracted from the integrated image is compared with the registered feature read out from the registration unit 180 (step S352). Thereafter, the authentication unit 190 outputs the authentication result of the authenticated user (step S353).

[0073] (Technical Effects) Next, technical effects obtained by the third information processing apparatus 10 will be described.

[0074] As described with reference to Figures 10 to 12, the third information processing device 10 executes a process for registering features of a registered user and a process for authenticating the user. In particular, when authenticating a user, multiple images including the authenticated user to be authenticated are acquired, and features are extracted using an integrated image obtained by integrating the multiple images. This substantially reduces the number of times the feature extraction process is performed, thereby enabling appropriate feature extraction while suppressing increases in processing load and processing time. When authenticating a user, time constraints are often imposed, such as requiring the authentication results to be output in real time. Therefore, the above-described technical effects are significantly achieved.

[0075] On the other hand, when registering the features of a registered user, multiple images including the registered user are acquired, and integrated features that integrate the features extracted from each of the multiple images are registered. When registering a user, there is often more time than when authenticating a user. Therefore, even if a process of extracting features for each of multiple images is executed, the possibility of problems occurring is low. By switching between the operations for authentication and registration in this way, it is possible to extract features appropriately depending on the situation.

[0076] The scope of each embodiment also includes a processing method in which a program that operates the configuration of each embodiment to realize the functions of the above-described embodiments is recorded on a recording medium, the program recorded on the recording medium is read as code, and the program is executed on a computer. In other words, a computer-readable recording medium is also included in the scope of each embodiment. Furthermore, each embodiment includes not only a recording medium on which the above-described program is recorded, but also the program itself.

[0077] Examples of recording media that can be used include floppy disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, magnetic tapes, non-volatile memory cards, and ROMs. Furthermore, the scope of each embodiment is not limited to programs that execute processes by themselves, but also includes programs that execute processes by operating on an OS in conjunction with other software or expansion board functions. Furthermore, the program itself may be stored on a server, and part or all of the program may be downloadable from the server to a user terminal. The program may be provided to the user in, for example, a SaaS (Software as a Service) format.

[0078] <Supplementary Notes> The above-described embodiment may be further described as in the following supplementary notes, but is not limited to the following.

[0079] (Supplementary Note 1) The information processing device described in Supplementary Note 1 is an information processing device that includes an acquisition means that acquires a plurality of images including a user's face, a normalization means that normalizes each of the plurality of images based on the user's face, an integration means that integrates the normalized plurality of images to generate an integrated image, and an extraction means that extracts features related to the user's face from the integrated image.

[0080] (Supplementary Note 2) The information processing device described in Supplementary Note 2 is the information processing device described in Supplementary Note 1, further comprising a determination means for determining an integration parameter, which is a weight when integrating the plurality of facial images, based on attributes of the plurality of images, and the integration means integrates the plurality of images based on the integration parameter.

[0081] (Supplementary Note 3) The information processing device according to Supplementary Note 3 is the information processing device according to Supplementary Note 2, wherein the determining means determines the integration parameter using a quality value indicating quality of the plurality of images as the attribute.

[0082] (Supplementary Note 4) The information processing device according to Supplementary Note 4 is the information processing device according to Supplementary Note 2 or 3, wherein the determining means determines the integrated parameter for each pixel based on the attribute on a pixel-by-pixel basis.

[0083] (Supplementary Note 5) The information processing device described in Supplementary Note 5 is the information processing device described in any one of Supplementary Notes 2 to 4, wherein the acquisition means acquires the plurality of images in chronological order, and the determination means determines the integrated parameters using tracking results of the user's face in the plurality of images as the attributes.

[0084] (Supplementary Note 6) The information processing device described in Supplementary Note 6 is the information processing device described in any one of Supplements 2 to 5, wherein the determination means adjusts the integrated parameter based on estimation accuracy when estimating the attribute. (Supplementary Note 7) The information processing device described in Supplementary Note 7 is the information processing device described in any one of Supplements 1 to 6, further including: registration means for registering facial features of a registered user as registered features; and authentication means for performing authentication processing of the user by comparing features extracted from the integrated image with the registered features.

[0085] (Appendix 8) The information processing device described in Appendix 8 is the information processing device described in Appendix 8, wherein the registration means extracts facial features of the registered user from each of a plurality of images including the face of the registered user, and registers an integrated version of the extracted features as the registered features.

[0086] (Appendix 9) The information processing method described in Appendix 9 is an information processing method that, by at least one computer, acquires a plurality of images including a user's face, normalizes each of the plurality of images based on the user's face, integrates the normalized plurality of images to generate an integrated image, and extracts features related to the user's face from the integrated image.

[0087] (Appendix 10) The recording medium described in Appendix 10 is a recording medium having recorded thereon a computer program that causes at least one computer to execute an information processing method, which includes acquiring a plurality of images including a user's face, normalizing each of the plurality of images based on the user's face, integrating the normalized plurality of images to generate an integrated image, and extracting features related to the user's face from the integrated image.

[0088] (Supplementary Note 11) The computer program described in Supplementary Note 11 is a computer program that causes at least one computer to execute an information processing method, which includes acquiring a plurality of images including a user's face, normalizing each of the plurality of images based on the user's face, integrating the normalized plurality of images to generate an integrated image, and extracting features related to the user's face from the integrated image.

[0089] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0090] REFERENCE SIGNS LIST 10 Information processing device 11 Processor 12 RAM 13 ROM 14 Storage device 15 Input device 16 Output device 110 Image acquisition unit 120 Normalization unit 130 Image integration unit 140 Feature extraction unit 150 Integration parameter determination unit 160 Feature extraction unit 170 Feature integration unit 180 Registration unit 190 Authentication unit

Claims

1. An acquisition unit that acquires a plurality of images including a user's face, a normalization unit that normalizes each of the plurality of images based on the user's face, an integration unit that integrates the normalized plurality of images to generate an integrated image, and an extraction unit that extracts feature amounts related to the user's face from the integrated image. An information processing apparatus comprising:

2. The information processing apparatus according to claim 1, further comprising a determination unit that determines an integration parameter, which is a weight when integrating the plurality of face images, based on attributes of the plurality of images, wherein the integration unit integrates the plurality of images based on the integration parameter.

3. The information processing apparatus according to claim 2, wherein the determination unit determines the integration parameter using, as the attribute, a quality value indicating the quality of the plurality of images.

4. The information processing apparatus according to claim 2 or 3, wherein the determination unit determines the integration parameter for each pixel based on the attribute in pixel units.

5. The information processing apparatus according to claim 2, wherein the acquisition unit acquires the plurality of images in a time series, and the determination unit determines the integration parameter using, as the attribute, a tracking result of the user's face in the plurality of images.

6. The information processing apparatus according to claim 2 or 3, wherein the determination unit adjusts the integration parameter based on an estimation accuracy when estimating the attribute.

7. The information processing apparatus according to claim 1 or 2, further comprising a registration unit that registers a feature amount of a registered user's face as a registered feature amount, and an authentication unit that performs an authentication process of the user by collating the feature amount extracted from the integrated image with the registered feature amount.

8. The information processing apparatus according to claim 7, wherein the registration unit extracts a feature amount of the registered user's face from each of a plurality of images including the registered user's face, and registers, as the registered feature amount, an integrated result of the extracted plurality of feature amounts.

9. An information processing method, comprising: obtaining a plurality of images including a user's face by at least one computer; normalizing each of the plurality of images based on the user's face; generating an integrated image by integrating the normalized plurality of images; and extracting feature amounts related to the user's face from the integrated image.

10. A recording medium having recorded thereon a computer program for causing at least one computer to execute an information processing method, the method comprising: obtaining a plurality of images including a user's face; normalizing each of the plurality of images based on the user's face; generating an integrated image by integrating the normalized plurality of images; and extracting feature amounts related to the user's face from the integrated image.

Images