Information processing device, information processing method, and computer program

Abstract

An information processing device 1 comprises: a calculation unit 11 that calculates the degree of similarity between images sequentially captured; an acquisition unit 12 that acquires a representative image from among a plurality of images having the degree of similarity equal to or more than a prescribed value; an extraction unit 13 that extracts representative feature amounts from the representative image; and a comparison unit 14 that performs comparison, by using the representative feature amounts, with a registered image which is registered in advance.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Patent Literature 1 describes a technique for determining image similarity by dividing a target image into multiple regions based on the position of the target image relative to a reference point, dividing a reference image into multiple regions in a similar manner, and determining the similarity of the distribution of feature points in the target image and the reference image based on feature points in the target image, feature points in the reference image, and the divided regions. Patent Literature 2 describes a technique for generating perturbed face images by perturbing, i.e., rotating, an input face image by a predetermined angle in the roll angle direction, calculating perturbation similarities with registered face images for each angle, selecting the angle with the highest perturbation similarity as a perturbation central angle, and calculating an integrated similarity from perturbation similarities calculated at angles including the perturbation central angle, thereby eliminating the influence of tilted glasses and performing authentication. It also describes that it is preferable not to use perturbation similarities calculated at angles near the perturbation central angle in calculating the integrated similarity, but to further refer to how the perturbation similarity changes with angle changes.

[0005] Patent Literature 3 describes a technology for acquiring one or more images, detecting a target object image from each of the images, extracting one or more local regions from the target object image, calculating, for each image, a feature amount for recognizing the target object from each of the one or more local regions, calculating a similarity between the feature amounts calculated for each image for each local region, and if there are feature amounts whose similarity is equal to or greater than a threshold, registering one of the feature amounts for each local region as dictionary data for the target object, and recognizing the target object from the image by matching it with the dictionary data. Patent Literature 4 describes a technology for classifying and storing a plurality of first feature information and a plurality of first image data of the same subject into a plurality of groups, comparing second feature information extracted from second image data of the same subject acquired by an imaging means with the plurality of first feature information, calculating a similarity between the second feature information and each of the plurality of first feature information, and displaying the second image data in association with the group with the highest similarity among the plurality of groups.Patent Literature 5 describes a technology in which a HDD stores multiple image data including human faces, a CPU detects facial areas included in the image data, and stores a face table consisting of a list of facial images in the detected facial areas and their features on the HDD. The CPU calculates similarity between the facial images stored in the face table based on their features, and groups the multiple facial images stored in the face table into groups of the same person based on the similarity. Non-Patent Literature 1 describes a technology in which a representative feature is created by taking a weighted average of facial features for each video frame, and then matching the representative feature with the facial feature of a registered image. The technology described in Non-Patent Literature 1 estimates feature weights so that images that are "unsuitable for authentication," such as those with a large face orientation or a large amount of occlusion, are given low weights, and images that are "suitable for authentication," such as images that are facing forward or have high image quality, are given high weights, and representative feature values ​​suitable for authentication are created from the video.

[0003] JP 2018-0978525 A JP 2017-182459 A JP 2012-238121 A JP 2014-186580 A JP 2014-044606 A

[0004] Yang, J., Ren, P., Zhang, D., Chen, D., Wen, F., Li, H., & Hua, G., Neural aggregation network for video face recognition. In CVPR 2017.

[0005] An object of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the techniques described in prior art documents.

[0006] One aspect of the information processing device includes a calculation means for calculating the similarity between images captured sequentially, an acquisition means for acquiring a representative image from a plurality of images whose similarity is equal to or greater than a predetermined value, an extraction means for extracting a representative feature from the representative image, and a comparison means for comparing the representative feature with a registered image that has been registered in advance.

[0007] One aspect of the information processing method calculates the similarity between images captured sequentially, obtains a representative image from a plurality of images whose similarity is equal to or greater than a predetermined value, extracts representative features from the representative image, and uses the representative features to compare the representative image with a registered image that has been registered in advance.

[0008] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute an information processing method that calculates the similarity between images captured sequentially, obtains a representative image from a plurality of the images whose similarity is equal to or greater than a predetermined value, extracts representative features from the representative image, and uses the representative features to compare the representative image with a registered image that has been registered in advance.

[0009] FIG. 1 is a block diagram showing the configuration of an information processing device in a first embodiment. FIG. 2 is a block diagram showing the configuration of an information processing device in a second embodiment. FIG. 3 is a conceptual diagram showing the flow of information processing operations of the information processing device in the second embodiment. FIG. 4 is a flowchart showing the flow of information processing operations of the information processing device in the second embodiment. FIG. 5 is a flowchart showing the flow of information processing operations of the information processing device in the second embodiment. FIG. 6 is a block diagram showing the configuration of an information processing device in a third embodiment. FIG. 7 is a block diagram showing the configuration of an information processing device in a fourth embodiment. FIG. 8 is a block diagram showing the configuration of an information processing device in a fifth embodiment.

[0010] Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings. [1: First Embodiment]

[0011] A first embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the first embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 1 to which the first embodiment of the information processing device, the information processing method, and the recording medium is applied. [1-1: Configuration of Information Processing Device 1]

[0012] 1 is a block diagram showing the configuration of an information processing device 1 according to the first embodiment. As shown in FIG. 1, the information processing device 1 includes a calculation unit 11, an acquisition unit 12, an extraction unit 13, and a matching unit 14.

[0013] The calculation unit 11 calculates the similarity between images captured sequentially. The acquisition unit 12 acquires a representative image from a plurality of images having a similarity equal to or greater than a predetermined level. The extraction unit 13 extracts a representative feature from the representative image. The matching unit 14 uses the representative feature to match the representative image with a registered image that has been registered in advance. [1-2: Technical Effects of the Information Processing Device 1]

[0014] The information processing device 1 in the first embodiment extracts a representative feature from a representative image acquired from a plurality of images, thereby reducing the processing load of the feature extraction process. Furthermore, the representative feature is used to compare the image with a registered image that has been registered in advance, thereby reducing the processing load of the comparison process. [2: Second embodiment]

[0015] Next, a second embodiment of the information processing device, information processing method, and recording medium will be described. In the following, the second embodiment of the information processing device, information processing method, and recording medium will be described using an information processing device 2 to which the second embodiment of the information processing device, information processing method, and recording medium is applied. The information processing device 2 in this embodiment may be applied to an entry / exit management mechanism, a mechanism for searching for criminals, etc. [2-1: Configuration of the information processing device 2]

[0016] 2 is a block diagram showing the configuration of an information processing device 2 in the second embodiment. As shown in FIG. 2, the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing the operations to be performed by the information processing device 2 are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) to be performed by the information processing device 2.

[0018] FIG. 2 shows an example of logical functional blocks implemented within the arithmetic device 21 to perform information processing operations. As shown in FIG. 2, the arithmetic device 21 includes a calculation unit 211, which is a specific example of a "calculation means" described in the appendix, an acquisition unit 212, which is a specific example of an "acquisition means" described in the appendix, an extraction unit 213, which is a specific example of an "extraction means" described in the appendix, a matching unit 214, which is a specific example of a "matching means" described in the appendix, and an input acceptance unit 215. However, the input acceptance unit 215 does not necessarily have to be implemented within the arithmetic device 21. Details of the operations of the calculation unit 211, the acquisition unit 212, the extraction unit 213, the matching unit 214, and the input acceptance unit 215 will be described later with reference to FIGS. 3 and 4.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0020] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or USB (Universal Serial Bus).

[0021] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-2: Information Processing Operation Performed by the Information Processing Device 2]

[0023] The information processing operation performed by the information processing device 2 will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a conceptual diagram of the information processing operation performed by the information processing device 2. Fig. 4 is a flowchart showing the flow of the information processing operation performed by the information processing device 2.

[0024] 3 illustrates video frames I_0, I_1, I_2, I_3, I_4, I_5, and I_6 constituting a video as sequentially captured images. In this embodiment, a video may refer to a set of consecutive video frames. In the example illustrated in FIG. 3, video frames I_0, I_1, I_2, I_3, I_4, I_5, and I_6 are input to the information processing device 2 consecutively in this order.

[0025] As shown in Figure 3, facial images in video frames are often similar. In particular, when the subject does not move much, similar facial images are often included consecutively across multiple video frames.

[0026] In face matching using video (hereinafter sometimes referred to as "video face recognition"), matching is performed for each video frame, and the success or failure of face matching is often determined by finally integrating the matching results. Since features extracted from similar facial images often have similar amounts of information, if feature extraction and feature matching are performed for each video frame, the same features will be integrated in the final integration of the matching results, and the extraction and matching operations for each video frame may become unnecessary processes. This is particularly noticeable when the frame rate of the video is high.

[0027] Next, the flow of information processing operations performed by the information processing device 2 will be described with reference to the flowchart shown in Fig. 4. As shown in Fig. 4, the input receiving unit 215 receives input of images that are captured sequentially (step S20). The images that are captured sequentially may be video frames that constitute a video. The input receiving unit 215 may also receive video frames that constitute a video that are input sequentially. In the second embodiment, the images that are captured sequentially may be facial images.

[0028] The calculation unit 211 calculates the similarity between the images captured sequentially (step S21). The calculation unit 211 may calculate the similarity between a reference face image used as a reference and the accepted face image.

[0029] 3, the facial image in the first input video frame I_0 may be used as the reference facial image. In the example shown in Fig. 3, the similarity S(I_0, I_1) between the facial image in the video frame I_1 input after the video frame I_0 and the reference facial image (the facial image in video frame I_0) is calculated.

[0030] Alternatively, the calculation unit 211 may calculate the similarity between face images that are adjacent in the order of reception by the input reception unit 215. That is, the calculation unit 211 may calculate the similarity between the face image received in the (N-1)th position and the face image received in the Nth position, and may calculate the similarity between the face image received in the Nth position and the face image received in the (N+1)th position. The calculation unit 211 may calculate the similarity between face images between video frames.

[0031] The calculation unit 211 determines whether the similarity between the facial images is less than a predetermined threshold (step S22). If the similarity between the facial images is not less than the predetermined threshold (step S22: No), the process returns to step S20. The predetermined threshold for similarity may be set appropriately depending on the implementation status of the authentication operation, etc. For example, when the calculation unit 211 calculates the similarity between facial images that are adjacent in the order of reception by the input reception unit 215, the predetermined threshold may be set higher than when the calculation unit 211 calculates the similarity with a reference facial image.

[0032] In the example shown in FIG. 3, if the similarity S(I_0, I_1) between the facial images is equal to or greater than a predetermined threshold T (S(I_0, I_1)≧T), the calculation unit 211 may group the facial image in video frame I_1 and the facial image in video frame I_0. In this embodiment, the calculation unit 211 may group video frames with similar facial images among consecutively input video frames. In other words, the calculation unit 211 groups the facial images in chronological order.

[0033] If the similarity between the face images is less than the threshold value (step S22: Yes), the acquisition unit 212 acquires a representative face image from the plurality of face images having a similarity equal to or greater than a predetermined value (step S23).

[0034] In the example shown in FIG. 3 , the calculation unit 211 calculates the similarity S(I_0, I_2) between a face image in video frame I_2, which is input immediately after video frame I_1, and a reference face image (a face image in video frame I_0). If the similarity S(I_0, I_2) between the face images is less than a predetermined threshold T (S(I_0, I_2)<T), the calculation unit 211 does not group the face image in video frame I_2 with the face image in video frame I_0. Here, the calculation unit 211 may determine that the only video frame to be grouped as a video frame containing a face image similar to the face image in video frame I_0 is video frame I_1. The acquisition unit 212 acquires a representative image from the face images in the grouped video frames. Hereinafter, the grouped video frames may be referred to as an image group.

[0035] Acquisition of a representative face image from a plurality of face images by the acquisition unit 212 may be said to be integration of the representative face image from the plurality of face images. The acquisition unit 212 may integrate face images having a certain degree of similarity or higher into a single face image. The representative image may be a typical face image including features shared by face images included in the image group. In the second embodiment, the acquisition unit 212 may acquire one representative face image.

[0036] If the similarity between the face images is less than the threshold (step S22: Yes), the calculation unit 211 may update the reference face image to the accepted face image. That is, in the example shown in FIG. 3, the reference face image may be updated to the face image in video frame I_2.

[0037] The extraction unit 213 extracts representative facial feature values ​​from the representative facial images (step S24). The extraction unit 213 may extract representative facial feature values ​​from one representative facial image. The extraction unit 213 may perform the extraction operation only once for one image group.

[0038] In the second embodiment, the processing load of the calculation unit 211 for calculating the similarity is smaller than the processing load of the extraction unit 213 for extracting the representative feature. In other words, the processing load of the calculation unit 211 for sequentially calculating the similarity for the sequentially captured images accepted by the input accepting unit 215 is smaller than the processing load of the extraction unit 213 for extracting feature amounts from all of the sequentially captured images accepted by the input accepting unit 215.

[0039] The matching unit 214 uses the representative facial feature amount to match the representative facial image with a registered facial image that has been registered in advance (step S25). The matching unit 214 may match the representative facial feature amount extracted from one representative facial image with registered facial feature amounts extracted from registered facial images that have been registered in advance. At least one of the registered facial image and the registered facial feature amount may be stored in the storage device 22. Successful matching by the matching unit 214 may be understood to correspond to successful matching between the registered facial image and a facial image included in the image group from which the representative facial image from which the representative facial feature amount was extracted was acquired.

[0040] 3, the calculation unit 211 calculates the similarity S(I_2, I_3) between the face image in video frame I_3 input after video frame I_2 and the reference face image (the face image in video frame I_2). If the similarity S(I_2, I_3) between the face images is equal to or greater than a predetermined threshold T (S(I_2, I_3)≧T), the calculation unit 211 groups the face image in video frame I_3 and the face image in video frame I_2.

[0041] The calculation unit 211 calculates the similarity S(I_2, I_4) between the face image in video frame I_4, which is input after video frame I_3, and the reference face image (the face image in video frame I_2). If the similarity S(I_2, I_4) between the face images is equal to or greater than a predetermined threshold T (S(I_2, I_4)≧T), the calculation unit 211 groups the face image in video frame I_4 and the face image in video frame I_2.

[0042] The calculation unit 211 calculates the similarity S(I_2, I_5) between the face image in video frame I_5, which is input immediately after video frame I_4, and the reference face image (the face image in video frame I_2). If the similarity S(I_2, I_5) between the face images is less than a predetermined threshold T (S(I_2, I_5)<T), the calculation unit 211 does not group the face image in video frame I_5 and the face image in video frame I_2. Here, the calculation unit 211 may determine that the video frames to be grouped as video frames containing a face image similar to the face image in video frame I_2 are video frames I_3 and I_4. The acquisition unit 212 acquires a representative image from the face images in the grouped video frames. Furthermore, the extraction unit 213 may extract features from the representative image and perform a matching operation using the extracted features. Furthermore, the calculation unit 211 may update the reference face image to the face image in video frame I_5.

[0043] 4, the case where the similarity calculation process and the image matching process are performed sequentially has been described, but the similarity calculation process and the image matching process may be performed in parallel. That is, as shown in FIG. 5A, regardless of the determination result of step S22′, the input receiving unit 215 may receive input of the next video frame, and the calculation unit 211 may calculate the similarity.

[0044] As shown in FIG. 5A, if the similarity between the facial images is not less than the threshold (step S22′: No), the target facial image may be included in the image group (step S26). Furthermore, it may be determined whether the video has ended (step S27). If the video has ended, the matching unit 214 may determine whether matching of the target person included in the video has been successful (step S28). In step S28, the matching unit 214 may determine whether matching of the target person appearing in the facial image has been successful based on the results of matching between the multiple representative facial features and the registered facial features. Alternatively, in step S28, the matching unit 214 may integrate the representative facial features using a method such as Neural Aggregation Networks (NAN) described in the aforementioned Non-Patent Document 1 to calculate the similarity between the video and the registered images, and determine whether matching of the target person included in the video has been successful.

[0045] In this embodiment, a case where face matching is performed has been described, but the information processing device 2 may be applied to matching using human body parts other than the face, or matching objects other than humans. The information processing operation in this embodiment may be applied to matching operations of moving images in human detection, object detection, etc. [2-3: Technical Effects of the Information Processing Device 2]

[0046] In entrance / exit management and personal authentication using surveillance cameras, a person captured on camera is compared to a person registered in a database to see if they match. In real-time authentication, facial images in video frames that make up a video are generally input.

[0047] Extracting facial features from facial images for each video frame that makes up a video requires a lot of processing time. As the video frame rate increases, the processing time required for feature extraction increases. In face recognition, the feature extraction process is often the speed limiting factor, and increasing the video frame rate causes an increase in processing time in video face recognition. Unlike face matching using still images, video face matching often requires an increase in processing time.

[0048] Video facial recognition is widely implemented in society for purposes such as access control, and the need for video facial recognition is expected to continue to grow. There is a demand for faster, more accurate video facial recognition.

[0049] The information processing device 2 in the second embodiment extracts a representative feature from a representative image acquired from a plurality of images, thereby reducing the number of times the feature extraction process is performed, thereby shortening the processing time for matching using moving images. The information processing device 2 calculates similarity with a smaller processing load than extracting a representative feature, thereby shortening the processing time for matching using moving images. [3: Third Embodiment]

[0050] Next, a third embodiment of the information processing device, the information processing method, and the recording medium will be described. Hereinafter, the third embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 3 to which the third embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0051] 6 is a block diagram showing the configuration of an information processing device 3 in the third embodiment. The information processing device 3 in the third embodiment differs from the information processing device 2 in the second embodiment in the operation of the calculation unit 311. [3-1: Information Processing Operation Performed by Information Processing Device 3] The calculation unit 311 in the third embodiment calculates the similarity using at least one of feature points included in an image, the luminance of the image, and the frequency spectrum of the image. The calculation unit 311 may calculate the similarity using the distribution of feature points included in the image, the distribution of luminance of the image, and the frequency spectrum of the image.

[0052] Even if there is no movement, the feature amount often changes depending on the lighting, changes in the environment, and changes in the background. The calculation unit 311 in the third embodiment may calculate the similarity using brightness and frequency spectrum. The calculation unit 311 can group images with similar feature amounts into the same image group.

[0053] A change in the facial orientation often changes the positions of feature points. The calculation unit 311 in the third embodiment may calculate the similarity using the distribution of facial feature points included in the facial images and perform grouping. The calculation unit 311 may group facial images with the same facial orientation into the same image group. Furthermore, the calculation unit 311 may detect that a facial image belongs to a different image group when the facial orientation changes. [3-2: Technical Effects of Information Processing Device 3]

[0054] The information processing device 3 according to the third embodiment can appropriately group face images under various circumstances. [4: Fourth Embodiment]

[0055] Next, a fourth embodiment of the information processing device, the information processing method, and the recording medium will be described. Hereinafter, the fourth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 4 to which the fourth embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0056] FIG. 7 is a block diagram showing the configuration of an information processing device 3 in the fourth embodiment. The information processing device 4 in the fourth embodiment differs from the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment in the operation of the acquisition unit 412. [4-1: Information Processing Operation Performed by Information Processing Device 4] In the fourth embodiment, the acquisition unit 412 selects one image from a plurality of images having a similarity equal to or greater than a predetermined level to acquire a representative image. The acquisition unit 412 may select one face image by, for example, random sampling. Alternatively, the acquisition unit 412 may select, for example, the above-mentioned reference face image as the one face image. Alternatively, the acquisition unit 412 may select, for example, a face image acquired in the middle stage as the one face image. [4-2: Technical Effects of Information Processing Device 4]

[0057] The information processing device 4 in the fourth embodiment selects one image to obtain a representative image, which is advantageous for authentication because there is no deviation in the positions of feature points such as the eyes, nose, and mouth in the face image. Also, selecting one representative image reduces the processing load compared to, for example, combining multiple images. [5: Fifth Embodiment]

[0058] Next, a fifth embodiment of the information processing device, the information processing method, and the recording medium will be described. Hereinafter, the fifth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 5 to which the fifth embodiment of the information processing device, the information processing method, and the recording medium is applied.

[0059] FIG. 8 is a block diagram showing the configuration of the information processing device 5 according to the fifth embodiment. The information processing device 5 according to the fifth embodiment differs from the information processing device 2 according to the second embodiment to the information processing device 4 according to the fourth embodiment in the operation of the acquisition unit 512. [5-1: Information Processing Operation Performed by the Information Processing Device 5] In the fifth embodiment, the acquisition unit 512 generates one representative image from a plurality of images having a similarity equal to or greater than a predetermined level. The acquisition unit 512 may generate an average image as one representative image by, for example, performing averaging processing for each pixel. The acquisition unit 512 may generate one representative image by using all images included in a group of images having a similarity equal to or greater than a predetermined level. Alternatively, the acquisition unit 512 may generate one representative image by using two or more images included in a group of images having a similarity equal to or greater than a predetermined level. In this case, the acquisition unit 512 may select two or more images by, for example, random sampling. Alternatively, the acquisition unit 512 may select two or more images acquired at a predetermined timing, for example. [5-2: Technical Effects of the Information Processing Device 5]

[0060] The information processing device 5 in the fifth embodiment generates one representative image from multiple images with a similarity greater than or equal to a predetermined level, thereby enabling a representative image that includes the characteristics of each of the multiple images. Generating one representative image from multiple images with a similarity greater than or equal to a predetermined level aggregates all of the information from the image group, thereby increasing the amount of information available for authentication, which is advantageous for authentication. For example, compared to selecting one representative image from an image group containing multiple images with a similarity greater than or equal to a predetermined level, the amount of information available for authentication is increased.

[0061] In the fourth embodiment, a case where one representative image is selected from an image group including a plurality of images having a similarity equal to or greater than a predetermined level is described, and in the fifth embodiment, a case where one representative image is generated from an image group including a plurality of images having a similarity equal to or greater than a predetermined level is described. For example, it may be determined for each video whether to select one representative image as described in the fourth embodiment or to generate one representative image as described in the fifth embodiment.

[0062] Alternatively, the multiple representative images acquired from the same video may include both the selected representative image described in the fourth embodiment and the generated representative image described in the fifth embodiment. In this case, for example, depending on the deviation in the positions of faces within face images included in an image group having a predetermined similarity or higher, the method of selecting one representative image described in the fourth embodiment and the method of generating one representative image described in the fifth embodiment may be used interchangeably. For example, if the deviation in the positions of faces within each face image included in the image group is relatively large, the method of selecting one representative image from the image group described in the fourth embodiment may be employed. Alternatively, if the deviation in the positions of faces within each face image included in the image group is small and the positions of faces within each face image are close, the method of generating one representative image from the image group described in the fifth embodiment may be employed. The deviation in the positions of faces within each face image included in the image group may be calculated based on the positions of facial feature points, such as the eyes, nose, and mouth, included in each face image included in the image group. Furthermore, depending on the number of images included in an image group having a similarity equal to or greater than a predetermined level, one representative image may be selected as described in the fourth embodiment, or one representative image may be generated as described in the fifth embodiment. [6: Supplementary Note]

[0063] The following supplementary notes are further disclosed with respect to the above-described embodiment. [Supplementary Note 1] An information processing device comprising: a calculation unit that calculates a similarity between sequentially captured images; an acquisition unit that acquires a representative image from a plurality of the images having a similarity equal to or greater than a predetermined level; an extraction unit that extracts a representative feature from the representative image; and a matching unit that uses the representative feature to match the representative image with a registered image that has been registered in advance. [Supplementary Note 2] The information processing device according to claim 1, wherein the images are facial images; the acquisition unit acquires one representative facial image; the extraction unit extracts a representative feature from the one representative facial image; and the matching unit uses the representative feature extracted from the one representative facial image to match the representative facial image with a registered facial image that has been registered in advance. [Supplementary Note 3] The information processing device according to claim 1 or 2, wherein a processing load of the calculation unit in calculating the similarity is smaller than a processing load of the extraction unit in extracting the representative feature. [Supplementary Note 4] The information processing device according to claim 1 or 2, wherein the calculation unit calculates the similarity using at least one of feature points included in the image, the luminance of the image, and the frequency spectrum of the image. [Supplementary Note 5] The information processing device according to claim 1 or 2, wherein the acquisition means selects one image from the plurality of images whose similarity is equal to or greater than a predetermined level to acquire the representative image. [Supplementary Note 6] The information processing device according to claim 1 or 2, wherein the acquisition means generates one representative image from the plurality of images whose similarity is equal to or greater than a predetermined level. [Supplementary Note 7] An information processing method comprising: calculating a similarity between images captured sequentially; acquiring a representative image from the plurality of images whose similarity is equal to or greater than a predetermined level; extracting a representative feature from the representative image; and using the representative feature to compare the representative image with a registered image that has been registered in advance. [Supplementary Note 8] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method comprising: calculating a similarity between images captured sequentially; acquiring a representative image from the plurality of images whose similarity is equal to or greater than a predetermined level; extracting a representative feature from the representative image; and using the representative feature to compare the representative image with a registered image that has been registered in advance.

[0064] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0065] 1, 2, 3, 4, 5 Information processing device 11, 211, 311 Calculation unit 12, 212, 412, 512 Acquisition unit 13, 213 Extraction unit 14, 214 Collation unit 215 Input reception unit

Claims

1. a calculation means for calculating a similarity between images captured sequentially; an acquisition means for acquiring a representative image from the plurality of images having a similarity equal to or greater than a predetermined value; extraction means for extracting a representative feature amount from the representative image; a matching means for matching the representative feature amount with a registered image that has been registered in advance; An information processing device comprising:

2. the image is a facial image, the acquiring means acquires one representative face image, the extraction means extracts a representative feature amount from the one representative face image; The matching means uses the representative feature extracted from the one representative face image to match it with a registered face image that has been registered in advance. The information processing device according to claim 1 .

3. The processing load of the calculation means for calculating the similarity is smaller than the processing load of the extraction means for extracting the representative feature amount.

3. The information processing device according to claim 1 or 2.

4. The calculation means calculates the similarity using at least one of feature points included in the image, the luminance of the image, and the frequency spectrum of the image.

3. The information processing device according to claim 1 or 2.

5. The acquiring means selects one image from the plurality of images having a similarity equal to or greater than a predetermined value to acquire the representative image.

3. The information processing device according to claim 1 or 2.

6. The acquisition means generates one representative image from the plurality of images having a similarity equal to or greater than a predetermined value.

3. The information processing device according to claim 1 or 2.

7. Calculate the similarity between the images captured sequentially, A representative image is acquired from the plurality of images having a similarity equal to or greater than a predetermined value; extracting a representative feature amount from the representative image; The representative feature amount is used to compare with a pre-registered image. A computer-implemented information processing method.

8. On the computer, Calculate the similarity between the images captured sequentially, A representative image is acquired from the plurality of images having a similarity equal to or greater than a predetermined value; extracting a representative feature amount from the representative image; The representative feature amount is used to compare with a pre-registered image. A computer program for executing an information processing method.