Information processing device, method for controlling information processing device, program, and information processing system

Abstract

To correct recording for a wrong person due to erroneous authentication.SOLUTION: Provided is an information processing device including: face detection means for detecting a face area of a person to be authenticated from an image; feature quantity extraction means for extracting a face feature quantity of the face area; calculation means for collating the face feature quantity extracted by the feature quantity extraction means and a registered face feature quantity to calculate a collation score; recording information management means for storing, as recording information, at least one collation score associated with a recording time; and determination means for identifying the person to be authenticated based on the collation score calculated by the calculation means and the recording information. The determination means corrects recording information at a second time which is a time earlier than the first time based on the recording information at the first time about the identified person.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a control method for an information processing device, a program, and an information processing system. [Background technology]

[0002] In an access control system that uses facial recognition, the face of a person passing through a gate is compared with registered faces on a one-to-many basis, and the face with the highest matching score exceeding a threshold is recognized as the person in question and clocked in. Methods for improving the accuracy of clocking in have been studied.

[0003] Patent Document 1 proposes a method for improving the accuracy of time stamping by excluding faces that have been successfully authenticated from the targets of matching. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-68878 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method disclosed in Patent Document 1 has a problem in that if an authentication error occurs, an incorrect time stamp is made, and if the person in question comes later, they are unable to stamp correctly.

[0006] Therefore, an object of the present invention is to make it possible to correct time stamps made by the wrong person due to erroneous authentication. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, an information processing device as one aspect of the present invention comprises a face detection means for detecting a facial area of ​​a person to be authenticated from an image, a feature extraction means for extracting facial features of the facial area, a calculation means for calculating a matching score by comparing the facial features extracted by the feature extraction means with registered facial features, a timestamp information management means for saving at least one matching score linked to a timestamp as timestamp information, and a determination means for identifying the person to be authenticated based on the matching score calculated by the calculation means and the timestamp information, and the determination means modifies the timestamp information at a second time that is earlier than the first time, based on the timestamp information of the identified person at a first time. [Effects of the Invention]

[0008] According to the present invention, it is possible to correct time stamps made by the wrong person due to erroneous authentication. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a system configuration diagram according to an embodiment. [Figure 2] FIG. 1 is a hardware configuration diagram of an information processing device according to an embodiment. [Figure 3] 1 is a block diagram showing a functional configuration of an information processing device according to an embodiment; [Figure 4] 10 is an example of a table of a face information management unit in the embodiment. [Figure 5] 10 is an example of a table of a time-stamp information management unit in the embodiment. [Figure 6] 10 is a flowchart illustrating a process performed by an information processing device in each embodiment. [Figure 7] 10 is a flowchart illustrating a process performed by an information processing device in each embodiment. [Figure 8] 10 is a diagram illustrating an example of a change in a table of a time-stamp information management unit according to an embodiment. [Figure 9] 10 is a diagram illustrating an example of a change in a table of a time-stamp information management unit according to an embodiment. [Figure 10]10 is a diagram illustrating an example of a change in a table of a time-stamp information management unit according to an embodiment. [Figure 11] 10 is a diagram illustrating an example of a change in a table of a time-stamp information management unit according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as defined in the claims. Although the embodiments describe multiple features, not all of these features are necessarily essential to the invention, and multiple features may be combined in any desired manner. Furthermore, in the accompanying drawings, identical or similar components are designated by reference numerals, and redundant descriptions will be omitted.

[0011] <Embodiment 1> 1 is a diagram showing the system configuration of this embodiment. The system (information processing system) of this embodiment has an imaging device 10, an information processing device 11, a display device 12, and a gate device 13. As will be described in detail later, the information processing device of this embodiment also functions as an entrance / exit management device that manages the entry and exit of people.

[0012] The imaging device 10 and the information processing device 11 are connected to each other so as to be able to communicate with each other via a network 14. The information processing device 11 and the display device 12 are connected to each other so as to be able to communicate with each other via a network 15. The information processing device 11 and the gate device 13 are connected to each other so as to be able to communicate with each other via a network 16.

[0013] Networks 14, 15, and 16 are realized by a plurality of routers, switches, cables, etc. that comply with communication standards such as Ethernet (registered trademark). Note that networks 14, 15, and 16 may also be realized by the Internet, a wireless LAN, a wide area network (WAN), etc.

[0014] The imaging device (camera) 10 includes an imaging lens and an imaging sensor (solid-state imaging element) such as a CCD or CMOS, and captures image data and video data through the imaging lens. The imaging device 10 in this embodiment can be placed at a position and angle of view that allows it to capture (photograph) at least the face of a person. The imaging device 10 may be, for example, a network camera or a module camera, and the form is not particularly specified. The imaging device 10 transmits the captured image to the information processing device 11 via a network 14.

[0015] The information processing device 11 is a device that authenticates a person captured in an image acquired from the imaging device 10 according to a predetermined matching rule and stamps the person's time in accordance with a time-stamping rule, which will be described later. The predetermined matching rule is, for example, whether the likelihood of the captured face exceeds a threshold. The information processing device 11 also creates a feedback image explaining the authentication result and whether or not the person can pass through the gate, and transmits this to the display device 12 via the network 15. The information processing device 11 also transmits instructions for opening or closing the gate according to the authentication result to the gate device 13 via the network 16.

[0016] The display device 12 is a device that displays a feedback image based on the authentication result of the information processing device 11, and is, for example, a monitor or display. The gate device 13 is an opening / closing device having an opening / closing gate, and also serves as opening / closing means that opens and closes the gate in response to an instruction (control signal) from the information processing device 11. Although not shown, the information processing device 11 may have an input device such as a keyboard or a mouse. Furthermore, the input device may be connected to the information processing device 11 via a network. In this case, the input device also becomes part of the system configuration in this embodiment.

[0017] In this embodiment, the imaging device 10, the information processing device 11, and the display device 12 will be described as being separate terminals, such as an independent camera terminal for the imaging device, a PC terminal for the information processing device, and a display device for the display device, but this is not limiting, and for example, the imaging device 10, the information processing device 11, and the display device 12 may be contained in a single terminal such as a tablet.

[0018] 2 is a diagram showing the hardware configuration of the information processing device 11. The information processing device 11 has a CPU 110, a ROM 111, a RAM 112, a hard disk drive (HDD) 113, and a communication unit 114. The communication unit 114 is connected to the network .

[0019] The CPU (processor) 110 is a central processing unit that controls the entire information processing device 11. The CPU 110 reads out a control program recorded in the ROM 111 and executes various processes. The ROM 111 is a non-volatile memory that stores (memorizes), for example, control programs that do not require modification and various parameters. The RAM 112 is a volatile memory that is used as a temporary storage area such as a main memory or a work area. The HDD 113 is a storage device used to store, for example, image files. The HDD 113 may be an optical disk such as a flexible disk (FD) or a compact disk (CD) that is detachable from the information processing device 11, a magnetic or optical card, an IC card, a memory card, or the like. The communication unit 114 is a circuit that performs communication via the network 14.

[0020] As described above, the hardware configuration of the information processing device 11 has the same hardware components as those installed in a personal computer (PC). Therefore, various functions realized by the information processing device 11 can be implemented as software running on the PC. The information processing device 11 can realize each function shown in Fig. 3 and each process in the flowcharts of Figs. 6 and 7, which will be described later, by the CPU 110 executing a program.

[0021] Next, an example of the functional configuration of the information processing device 11 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the functional configuration of the information processing device according to this embodiment.

[0022] The information processing device 11 has, as functional units, an image acquisition unit 200, a face detection unit 201, a feature extraction unit 202, a matching score calculation unit 203, a face information management unit 204, an authentication determination unit 205, a timestamp information management unit 206, and a gate opening / closing notification unit 207.

[0023] The image acquisition unit 200 acquires images (image data) or videos from the imaging device 10. Then, the image acquisition unit 200 sequentially transmits the acquired images to the face detection unit 201. Specifically, the image acquisition unit 200 acquires the captured images transmitted to the information processing device 11 via the network 14.

[0024] The face detection unit (face detection means) 201 performs face detection processing on the image sent from the image acquisition unit 200, and detects an area in the image that includes a face as a face area. After detecting the face area, the face detection unit 201 sends the detected information to the feature extraction unit 202. In this embodiment, a person's face is detected by detecting edges of the eyes, mouth, etc. from the image. However, this is not a limitation, and the detection means is not limited thereto; for example, the face may be detected after detecting the whole body or head.

[0025] The feature extraction unit (feature extraction means) 202 performs feature extraction processing to extract features based on a predetermined calculation from the information on the face region extracted and transmitted by the face detection unit 201. In this embodiment, the feature extraction unit 202 performs processing to extract characteristic parts of a person's face by detecting edges of the eyes, mouth, etc. from the image. After extracting the features, the feature extraction unit 202 transmits the extracted features to the matching score calculation unit 203.

[0026] The matching score calculation unit 203 performs one-to-N matching using the feature amounts of one face (the face of the person whose feature amounts have been extracted) acquired from the feature amount extraction unit 202 and the feature amounts of N faces (registered feature amounts) acquired from the face information management unit 204. As a result, N matching scores are calculated. The matching score calculation unit 203 transmits the calculated N matching scores and face information corresponding to the matching scores to the authentication assessment unit 205.

[0027] The face information management unit (face information management means) 204 stores, as face information, information including the name, person ID, and face image of a person who is permitted to pass through the gate, and face feature amounts extracted from the face image. In this embodiment, the person ID, face image, and face feature amounts are stored as shown in Fig. 4. Fig. 4 is an example of a table managed by the face information management unit 204 in embodiment 1.

[0028] The authentication determination unit (determination means) 205 performs identity authentication using the N matching scores obtained from the matching score calculation unit 203 and past timestamp information obtained from the timestamp information management unit 206. That is, the authentication determination unit 205 identifies the identity of the person to be authenticated (authentication target) based on the N matching scores and past timestamp information. Note that the timestamp information may include, for example, at least one matching score linked to the timestamp (date and time), a person ID, and passing facial features. If authentication is successful, the authentication determination unit 205 sends the N matching scores and facial information to the timestamp information management unit 206. The facial information includes information such as the name, person ID, and facial image of the person permitted to pass through the gate, as well as facial features extracted from the facial image. The authentication determination unit 205 also notifies the gate open / close notification unit 207 of authentication success or authentication failure. The identity authentication logic of the authentication determination unit 205 in this embodiment is to determine that a person is the person whose matching score exceeds a threshold, is the highest, and is higher than the matching score of the saved timestamp information.

[0029] The timestamp information management unit (timestamp information management means) 206 stores the N matching scores and face information obtained from the authentication assessment unit 205 together with the current time. In this embodiment, as shown in Fig. 5, the timestamp information stored includes the timestamp date and time, the feature amount of the face that passed, the person ID of the authentication result, the person IDs ranked 1st to Nth, and the matching score. Fig. 5 is an example of a table managed by the timestamp information management unit 206 in the first embodiment.

[0030] The timestamp information management unit 206 may also perform a reset process to delete all or part of the saved timestamp information at any time. This prevents erroneous changes to timestamp information. For example, if timestamp information from the previous day or earlier contains timestamp information of a person attempting to pass through and the saved verification score is higher than the current verification score, the person currently attempting to pass through may be determined to be a different person and saved. By resetting timestamp information from the previous day or earlier, the person currently attempting to pass through may be determined to be the same person and the timestamp information may be saved correctly. Furthermore, by deleting timestamp information of a person who entered and exited once on the same day, it is possible to prevent either person from being determined to be a different person if that person re-enters. In other embodiments, instead of deleting the timestamp information, timestamp information from a certain date and time may be excluded from comparison or modification. Furthermore, the timestamp information to be compared or modified may be limited by the number of items, number of days, period, date and time, and the like. In other words, the timestamp information management unit 206 may send the timestamp information to the authentication assessment unit 205 according to at least one condition of the number of times, number of days, period, date and time, and the authentication assessment unit 205 may perform identity authentication based on the timestamp information.

[0031] The time-stamp information management unit 206 may also store entry or exit information in addition to the time-stamp information. In this case, the time-stamp information management unit 206 may send the entry or exit information to the authentication determination unit 205, and the authentication determination unit 205 may perform identity authentication (determine whether the person is the person in question) among those who have already entered or those who have already exited. The stored time-stamp information may also be sent to the display device 12 so that it can be viewed. It may also be sent to an external system other than the system in this embodiment, such as an attendance management system (not shown). In other words, the time-stamp information management unit 206 may perform processing to send all or part of the stored time-stamp information to an external system at any time.

[0032] The gate opening / closing notification unit (notification means) 207 notifies the gate device 13 of an instruction to open or close the gate in accordance with an instruction from the authentication assessment unit 205. Specifically, when the gate opening / closing notification unit 207 receives a notification of successful authentication from the authentication assessment unit 205, it sends an instruction to the gate device 13 to open the gate (to open the gate) (first notification). On the other hand, when it receives a notification of unsuccessful authentication from the authentication assessment unit 205, it sends an instruction not to open the gate (to keep it closed) to the gate device 13 (second notification). The gate opening / closing notification unit 207 also functions as notification means when sending the first notification or the second notification to the gate device 13.

[0033] Next, the processing flow performed by the information processing device 11 according to this embodiment will be described with reference to Fig. 6 and Fig. 7. Fig. 6 and Fig. 7 are flowcharts showing the processing performed by the information processing device 11 in each embodiment. Details of the processing other than that in embodiment 1 (the processing in embodiments 2 to 4) will be described later, and below, an overall outline of the processing performed by the information processing device 11 in each embodiment will be described with reference to Fig. 6 and Fig. 7.

[0034] In the following, the processing flow shown below is started when the user presses a start button or the like (not shown) on the information processing device 11 using an input device or the like (not shown). Each of the following processes is realized by the CPU 110 executing a program stored in the ROM 111 or the like. Furthermore, by adding an S to the beginning of each process (step), the process (step) is omitted from the description.

[0035] In S301, the image acquisition unit 200 acquires an image (image capture data) from the imaging device 10 and transmits the acquired image to the face detection unit 201. In S302, the face detection unit 201 performs face detection processing on the image transmitted from the image acquisition unit in S301, and detects a face area (an area in the image that includes a face).

[0036] In S303, the authentication assessment unit 205 determines whether a face area was found in the image in the processing in S302. If the result of the determination is that a face area was found in the image in the processing in S302, the process proceeds to S304. In this case, the face detection unit 201 transmits information about the detected face area to the feature extraction unit 202. On the other hand, if a face area was not found in the image in the processing in S302, the process proceeds to S315. In this case, it is determined that there is a person behaving suspiciously so as to make their face difficult to detect, for example.

[0037] In S304, the feature extraction unit 202 extracts facial features from the face area information transmitted in S303 from the face detection unit 201. After extracting the features, the feature extraction unit 202 transmits the extracted features to the matching score calculation unit 203.

[0038] In S305, the authentication assessment unit 205 determines whether or not facial features were extracted from the facial area information in the processing in S304. If the determination result shows that facial features were extracted from the facial area information in the processing in S304, the process proceeds to S306. On the other hand, if facial features were not extracted from the facial area information in the processing in S304, the process proceeds to S315. In this case, it is determined that there is a person who is behaving suspiciously, for example, by hiding part of their face.

[0039] In S306, the matching score calculation unit 203 performs one-to-N matching using the feature amount acquired from the face information management unit 204. As a result, N matching scores are calculated. The matching score calculation unit 203 transmits the calculated N matching scores and face information corresponding to the matching scores to the authentication assessment unit 205.

[0040] In S307, the authentication determination unit 205 performs identity authentication using the N matching scores obtained from the matching score calculation unit 203 and past timestamp information obtained from the timestamp information management unit 206. During identity authentication, it is determined whether the matching score of the person with the highest matching score exceeds a threshold. If the result of the determination is that the matching score of the person with the highest matching score exceeds the threshold, the process proceeds to S308. In this case, it is also determined that identity authentication has been successful (the person has been identified as the person). On the other hand, if the matching score of the person with the highest matching score does not exceed the threshold, the process proceeds to S315. In this case, it is also determined that there is a suspicious person who is not registered, for example.

[0041] In S308, the authentication assessment unit 205 determines whether or not there is past timestamp information in the timestamp information management unit 206. That is, it determines whether or not there is past timestamp information in the timestamp information management unit 206 for the person whose identity was successfully authenticated in S307 (the person about to pass through the gate). If the result of the determination is that there is past timestamp information in the timestamp information management unit 206 for the person whose identity was successfully authenticated in S307, the process transitions to S310. On the other hand, if there is no past timestamp information in the timestamp information management unit 206 for the person whose identity was successfully authenticated in S307, the process transitions to S309.

[0042] In S309, the authentication assessment unit 205 sends the authentication result (information indicating successful authentication) and N matching scores from S307 to the timestamp information management unit 206, and saves the timestamp information. The authentication assessment unit 205 also controls the gate opening / closing notification unit 207 to send a gate opening notification (first notification) to the gate device 13. After receiving the notification, the gate of the gate device 13 is opened. The timestamp information management unit 206 saves timestamp information for one person, as shown in FIG. 8(A). Then, the processing ends. In this way, when a person with no previous timestamp information approaches the gate device 13 and attempts to pass through the gate, the timestamp information for that person is saved in S309. When the processing in S309 ends, the system returns to the state before this processing flow was executed, and it becomes possible to execute this flow again.

[0043] In S310, the authentication assessment unit 205 compares the matching score of the person attempting to pass through the gate with the matching score of the past timestamp information stored in the timestamp information management unit 206. Then, it is determined whether the matching score of the person attempting to pass through the gate is higher than the matching score of the past timestamp information. If the result of the determination is that the matching score of the person attempting to pass through the gate is higher than the matching score of the past timestamp information, the process proceeds to S311. On the other hand, if the matching score of the person attempting to pass through the gate is lower than the matching score of the past timestamp information, the process proceeds to S314.

[0044] In S311, the timestamp information management unit 206 saves the current timestamp information, that is, the timestamp information of the person (identified person) who has been successfully authenticated at the current time (current time).Then, the process proceeds to S312.

[0045] In S312, the authentication assessment unit 205 determines whether the past timestamp information contains information on one or more people who exceeded the threshold. If the result of the determination is that the past timestamp information contains information on one or more people who exceeded the threshold, the process returns to S308 and performs the same process. On the other hand, if the past timestamp information does not contain information on people who exceeded the threshold, the process transitions to S313. In this case, it is also determined that an unregistered person (a person not previously registered) has already passed through the gate.

[0046] In S313, the authentication assessment unit 205 deletes the target authentication result person ID. Furthermore, measures such as issuing an alarm (issuing an alert) or contacting a security guard may be taken, or these may be combined. Note that when the processing in S313 ends, the state returns to the state before this processing flow was executed, and it becomes possible to execute this flow again.

[0047] In S314, the authentication assessment unit 205 determines whether there are any other faces that exceed one or more thresholds. If the result of the determination is that there are any other faces that exceed one or more thresholds, the process returns to S308 and performs the same process. On the other hand, if there are no other faces that exceed one or more thresholds, the process proceeds to S315.

[0048] In S315, the authentication assessment unit 205 notifies the gate open / close notification unit 207 of authentication failure, and the gate open / close notification unit 207 sends an instruction (second notification) to the gate device 13 not to open the gate (to keep the gate closed). As a result, the gate device 13 closes the gate. Note that since the gate is closed at this point, the gate open / close notification unit 207 sends an instruction to the gate device 13 to keep the gate closed. In addition to closing the gate, other methods may be used, such as issuing an alarm (issuing an alert) or contacting a security guard, or a combination of these. Note that when the processing in S315 ends, the system returns to the state before this processing flow was executed, and it becomes possible to execute this flow again.

[0049] The above is the processing flow performed by the information processing device 11 according to the first embodiment. Below, a detailed description will be given of a case where a person with past timestamp information (a person registered in the table managed by the face information management unit 204) attempts to pass through a gate, with reference to a specific example shown in Fig. 8. Fig. 8 shows an example of changes to the table managed by the timestamp information management unit 206 in the first embodiment. Note that the following description will focus on differences from the processing flows shown in Figs. 6 and 7, and will omit detailed descriptions of similar processes.

[0050] Here, it is assumed that the timestamp information of Person_P has been saved in advance in the timestamp information management unit 206 as shown in Fig. 8(A). The threshold for the matching score is 60 points. It is also assumed that the processes of S301 to S307 have been performed and the process has progressed to S308. Note that the threshold for the matching score is an example and is not limited to this.

[0051] In S308, the authentication assessment unit 205 determines whether or not there is past timestamp information in the timestamp information management unit 206. In this embodiment, since it is determined that there is timestamp information for the person who was successfully authenticated in S307, it is determined that there is past timestamp information, and the process proceeds to S310. Referring to the example in FIG. 8, it is assumed that the person attempting to pass through the gate is Person_P, and that the person has been authenticated with a matching score of 80 points. The authentication assessment unit 205 checks with the timestamp information management unit 206 and confirms that the timestamp information for this person is stored in the first line of FIG. 8(A). Then, the process proceeds to S310.

[0052] In S310, the authentication assessment unit 205 compares the matching score shown in FIG. 8(A), which is a table recording past timestamp information, with the matching score of the person attempting to pass through the gate. Then, it determines whether the matching score of the person attempting to pass through the gate is higher than the matching score in the past timestamp information. If the result of the determination is that the matching score of the person attempting to pass through the gate is higher than the matching score in the past timestamp information, the process proceeds to S311. On the other hand, if the matching score of the person attempting to pass through the gate is lower than the matching score in the past timestamp information, the process proceeds to S314. Here, in this example, Person_P in the first row of FIG. 8(A) has a matching score of 70 points (matching score in the past timestamp information), while the person attempting to pass through has a matching score of 80 points. Therefore, the authentication assessment unit 205 determines that the matching score of the person attempting to pass through the gate is higher than the matching score in the past timestamp information. Therefore, in this case, the process proceeds to S311.

[0053] In S311, the timestamp information management unit 206 saves the current timestamp information as shown in the second line of Fig. 8(B), and then the process proceeds to S312.

[0054] In S312, the authentication assessment unit 205 determines whether the past timestamp information contains information on one or more people who exceeded the threshold. If the determination result shows that the past timestamp information contains information on one or more people who exceeded the threshold, the process returns to S308 and performs the same process. On the other hand, if the past timestamp information does not contain information on people who exceeded the threshold, the process proceeds to S313. In this example, the threshold for the matching score is 60 points, so the timestamp information management unit 206 determines whether there is a person ranked second or lower in the first row of Figure 8(B) whose matching score exceeds the threshold of 60 points. In this case, Person_Q has 65 points, which exceeds the threshold, so the process proceeds to S308.

[0055] When the process proceeds to S308 via S312 in this way, the authentication assessment unit 205 checks with the timestamp information management unit 206 in S308 to confirm that there is no timestamp information for Person_Q in the past timestamp information, and then proceeds to S309. When the process proceeds to S309 via S312 and S308, the authentication assessment unit 205 corrects the timestamp information for Person_Q at the time, as shown in the first line of Fig. 8(C) in S309. In this way, the authentication assessment unit 205 corrects (overwrites) the timestamp information at a second time that is earlier than the first time (e.g., the first line of Fig. 8(B)), based on the timestamp information at a first time of the identified person (e.g., the second line of Fig. 8(B)).

[0056] Furthermore, if multiple stamps occur at a third time different from the first time and the second time due to correction of the stamp information at the second time, the authentication assessment unit 205 corrects the stamp information at the third time.

[0057] Furthermore, CPU 110 may be configured to display at least one piece of information included in the stamp information on the screen of display device 12. Furthermore, CPU 110 may be configured to display at least one piece of information among the captured image, personal attribute information including facial feature amounts, stamp information, matching score, and correction history information of the stamp information on the screen of display device 12. In such a case, CPU 110 also functions as a display control means.

[0058] As described above, according to the information processing device 11 of this embodiment, if a person who arrives later has a higher matching score than the person who passed through the gate before, the later person is stamped as the person in question, and the earlier person is overwritten with the stamped information as the person with the next highest matching score in one-to-many matching. This makes it possible to correct a time stamp made by the wrong person due to erroneous authentication. This also makes it possible to save an image optimal for face recognition as a registered image.

[0059] <Embodiment 2> Next, a system having an information processing device 11 according to a second embodiment will be described. In the second embodiment, a case where a person who passed by in the past is found to be a non-registered person (a person who is not registered) due to correction of past timestamp information will be described with a specific example shown in FIG. 9. Therefore, the second embodiment assumes a case where a person who has past timestamp information and is registered in the face information management unit 204 has arrived. Note that the system configuration in the second embodiment is the same as that of the first embodiment, and therefore a description thereof will be omitted. That is, the information processing device 11 according to the second embodiment has the same hardware configuration and functional configuration as the information processing device 11 according to the first embodiment.

[0060] FIG. 9 shows an example of changes to a table managed by the embodied information management unit in the second embodiment. In the second embodiment, it is assumed that embodied information has been saved in advance as shown in FIG. 9(A). In the second embodiment, the specific example shown in FIG. 9 will be described with reference to the flowcharts in FIGS. 6 and 7, so that the same processing content as in the first embodiment will not be described again, and only the parts related to the specific example shown in FIG. 9 will be described in detail. In the following, it is assumed that the processing proceeds to S310 after the processing of S301 to S308 is performed, and then the processing proceeds from S310 to S311. The threshold for the matching score is 60 points, as in the first embodiment.

[0061] In S311, the timestamp information management unit 206 saves the current timestamp information as shown in the second line of Fig. 9(B), and then the process proceeds to S312.

[0062] In S312, the authentication assessment unit 205 determines whether the past timestamp information contains information on one or more people who exceeded the threshold. If the determination result shows that the past timestamp information contains information on one or more people who exceeded the threshold, the process returns to S308 and performs the same process. On the other hand, if the past timestamp information does not contain information on people who exceeded the threshold, the process proceeds to S313. In this embodiment, the threshold for the matching score is set to 60 points, so the authentication assessment unit 205 determines whether there is a person whose matching score exceeds the threshold of 60 points in the second place or lower in the first row of Figure 9(B). In Figure 9(B), the matching score of the second-place person (Person_Q) is 45 points, and it is determined that there is no information on a person who exceeded the threshold, and the process proceeds to S313.

[0063] In S313, the authentication assessment unit 205 deletes the target authentication result person ID. That is, the authentication assessment unit 205 determines that an unregistered person has already passed through, and instructs the timestamp information management unit 206 to delete the authentication result person ID in the first row as shown in FIG. 9(C), and then ends the process. Note that, as in the first embodiment, methods such as issuing an alarm (issuing an alert) or contacting a security guard may be used, or these may be combined. Note that, when the process in S313 ends, the system returns to the state before this processing flow was executed, and it becomes possible to execute this flow again.

[0064] <Embodiment 3> Next, a system having an information processing device 11 according to a third embodiment will be described. In the third embodiment, a case where the matching score of past timestamp information is higher than the matching score of the person to be authenticated will be described with a specific example shown in FIG. 10. Therefore, the third embodiment assumes a case where a person who has past timestamp information and is registered in the face information management unit 204 arrives. Note that the system configuration in the third embodiment is similar to that of the first embodiment, and therefore a description thereof will be omitted. That is, the information processing device 11 according to the third embodiment has the same hardware configuration and functional configuration as the information processing device 11 according to the first embodiment.

[0065] Fig. 10 shows an example of changes to the table managed by the embossing information management unit 206 in the third embodiment. In the third embodiment, it is assumed that embossing information is stored in advance as shown in Fig. 10(A). In the third embodiment, the specific example shown in Fig. 10 will be described with reference to the flowcharts in Figs. 6 and 7, so that the same processing content as in the first embodiment will not be described again, and only the parts related to the specific example shown in Fig. 10 will be described in detail. In the following, it is assumed that the processing of S301 to S307 has been performed, and the process has progressed to S308.

[0066] In S308, the authentication assessment unit 205 determines whether or not there is past timestamp information in the timestamp information management unit 206. In this embodiment, since it is determined that there is timestamp information for the person who was successfully authenticated in S307, it is determined that there is past timestamp information, and the process proceeds to S310. Here, referring to the example in FIG. 10, Person_P, who has the one-to-N matching results (Person_P 80 points, Person_R 75 points, ...) and has the highest matching score, has already been saved in the first row of (A) in FIG. 10. Therefore, the authentication assessment unit 205 determines that there is past timestamp information for the person, and the process proceeds to S310.

[0067] In S310, the authentication assessment unit 205 compares the matching score of the person attempting to pass through the gate with the matching score shown in FIG. 10(A), which is a table recording past timestamp information, and determines whether the matching score is higher than the past timestamp information. In this embodiment, Person_P's matching score was 80 points, but the authentication assessment unit 205 checked the table in FIG. 10 and found that the matching score of the past timestamp information was 90 points. Therefore, the authentication assessment unit 205 determines that the matching score of the person attempting to pass through the gate is not higher than the past matching score. In this case, the process proceeds to S314.

[0068] In S314, the authentication assessment unit 205 determines whether there are any other faces that exceed one or more thresholds. If the determination result shows that there are any other faces that exceed one or more thresholds, the process returns to S308 and performs the same process. On the other hand, if there are no other faces that exceed one or more thresholds, the process transitions to S315. Here, in the case of this embodiment, the authentication assessment unit 205 investigates whether there are any other people who exceed the threshold of 60 points. As a result of the investigation, Person_R is found as a person who exceeds the threshold of 60 points, so the authentication assessment unit 205 determines that there are other people who exceed the threshold of 60 points. Therefore, in this case, the process transitions to S308.

[0069] In S308, the authentication assessment unit 205 checks whether there is a record of Person_R in the past timestamp information. In this embodiment, it determines that there is no record of Person_R in the past timestamp information, and the process proceeds to S309.

[0070] In S309, the timestamp information management unit 206 determines that Person_R has passed through, saves the timestamp information as shown in the second line of Fig. 10(B), and ends the process. Note that when the process in S309 ends, the state returns to the state before this process flow was executed, and this flow can be executed again.

[0071] <Embodiment 4> Next, a system having an information processing device 11 of embodiment 4 will be described. In embodiment 4, a case where the past timestamp information has a higher matching score but there is no person with the next highest score will be described with a specific example shown in FIG. 11. Note that the system configuration in embodiment 4 is the same as the system configuration in embodiment 1, so a description thereof will be omitted. That is, the information processing device 11 in embodiment 4 has the same hardware configuration and functional configuration as the information processing device 11 in embodiment 1. In addition, only the differences from embodiment 3 will be described below, and unless otherwise noted, it is assumed to be the same as embodiment 3.

[0072] Fig. 11 shows an example of changes to the table managed by the embossing information management unit 206 in the fourth embodiment. In the fourth embodiment, it is assumed that embossing information has been saved in advance as shown in Fig. 11(A). In the fourth embodiment, the specific example shown in Fig. 11 will be described with reference to the flowcharts in Figs. 6 and 7, so that the same processing content as in the first embodiment will not be described again, and only the parts related to the specific example shown in Fig. 11 will be described in detail. In the following, it is assumed that the processing of S301 to S308 has been performed, and the process has progressed to S310.

[0073] In S310, the authentication assessment unit 205 compares the matching score in Figure 11(A), which is a table recording past timestamp information, with the matching score of the person attempting to pass through the gate, and determines whether the matching score is higher than the past timestamp information. In this embodiment, the authentication assessment unit 205 checks with the timestamp information management unit 206 and determines that Person_P's matching score of 80 points is not higher than the 90 points in the past timestamp information, and therefore the matching score is not higher than the past timestamp information. Therefore, in this case, the process proceeds to S314.

[0074] In S314, the authentication assessment unit 205 determines whether there are any other faces that exceed one or more thresholds. If the determination result shows that there are any other faces that exceed one or more thresholds, the process returns to S308 and performs the same process. On the other hand, if there are no other faces that exceed one or more thresholds, the process transitions to S315. Here, in the case of this embodiment, the authentication assessment unit 205 determines whether there are any other people who exceed the threshold of 60 points. As a result of the determination, there are no other people who exceed the threshold of 60 points, so the process transitions to S315.

[0075] In S315, the authentication assessment unit 205 notifies the gate open / close notification unit 207 of authentication failure, and the gate open / close notification unit 207 sends an instruction to the gate device 13 not to open the gate (to keep the gate closed). As a result, the gate device 13 closes the gate. Note that since the gate is closed at this point, the gate open / close notification unit 207 sends an instruction to the gate device 13 to keep the gate closed. In addition to closing the gate, other methods may be used, such as issuing an alarm (issuing an alert) or contacting a security guard, or a combination of these. Here, in the case of this embodiment, the authentication assessment unit 205 determines that the person attempting to pass is an unregistered person and ends the processing. Note that when the processing in S315 ends, the state returns to the state before this processing flow was executed, and it becomes possible to execute this flow again.

[0076] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims.

[0077] Furthermore, the functions of the above-described embodiments are not only realized by the computer reading and executing the program, but also include the operating system (OS) running on the computer performing some or all of the actual processing based on the instructions of the program.

[0078] Furthermore, a program that realizes part or all of the control in the above-described embodiment and the functions of the above-described embodiment may be supplied to the information processing device 11 or the system via a network or various storage media. Then, a computer (or CPU 110, etc.) in the information processing device 11 or the system may read and execute the program. In this case, the program and the storage medium storing the program constitute the present invention.

[0079] Furthermore, the functions of the above-described embodiments may be realized by the following method: Program code read from a recording medium is written to memory provided on a function expansion card inserted into a computer or a function expansion unit connected to the computer. Then, based on the instructions of the program code, a CPU 110 or the like provided on the function expansion card or function expansion unit performs some or all of the actual processing. Furthermore, when the present invention is applied to the above-described recording medium, the recording medium stores program code corresponding to each of the flowcharts described above.

[0080] The disclosure of this embodiment includes the following configuration, method, program, and system.

[0081] (Configuration 1) a face detection means for detecting a face area of ​​a person to be authenticated from an image; a feature extraction means for extracting facial feature amounts from the face region; a calculation means for calculating a matching score by matching the facial feature extracted by the feature extraction means with a registered facial feature; a time-stamp information management means for storing at least one matching score associated with a time stamp as time-stamp information; a determination means for identifying the person to be authenticated based on the matching score calculated by the calculation means and the timestamp information, the determination means corrects the timestamp information at a second time, which is a time earlier than the first time, based on the timestamp information at a first time of the identified person; 1. An information processing device comprising:

[0082] (Configuration 2) The device further includes a notification means for sending a first notification or a second notification to a predetermined opening / closing means, 2. The information processing apparatus according to configuration 1, wherein the determining means determines whether or not the timestamp information of the person specified by the determining means already exists in the timestamp information managing means.

[0083] (Configuration 3) The information processing device described in configuration 2 is characterized in that, if there is no time-stamp information of the person identified by the determination means, the determination means stores the time-stamp information of the person identified by the determination means and controls the notification means to send the first notification to the opening / closing means.

[0084] (Configuration 4) the determination means, when there is timestamp information of the person identified by the determination means, determines whether the verification score of the person identified by the determination means is higher than a past verification score included in the timestamp information; The information processing device according to configuration 2 or 3, characterized in that, if the result of the judgment is that the matching score of the person identified by the judgment means is higher than the past matching score, the judgment means stores the timestamp information of the person identified by the judgment means at the current time.

[0085] (Configuration 5) When the matching score of the person identified by the determining means is lower than past matching scores, the determining means determines whether one or more past matching scores exceed a threshold; The information processing device according to configuration 4, characterized in that if any of the past matching scores exceeds the threshold, the determination means corrects the timestamp information at the second time based on the past matching score that exceeded the threshold.

[0086] (Configuration 6) The information processing device according to configuration 2, wherein the determination means controls the notification means to send the second notification to the opening / closing means when the face area cannot be detected, or when the face feature amount cannot be extracted, or when the matching score of the person with the highest matching score calculated by the calculation means does not exceed a threshold value.

[0087] (Configuration 7) The information processing device described in any one of configurations 1 to 6, characterized in that the determination means determines that the person to be authenticated is the person in question if the matching score of the person with the highest matching score calculated by the calculation means exceeds a threshold.

[0088] (Configuration 8) The information processing device described in any one of configurations 1 to 7, characterized in that the judgment means corrects the time-stamp information at the third time when multiple time-stamps occur at a third time that is different from the first and second times due to correction of the time-stamp information at the second time.

[0089] (Configuration 9) The information processing device described in any one of configurations 1 to 8, characterized in that the determination means determines that the identified person at the first time is a person who has not been registered in advance, in accordance with the correction of the time-stamp information at the second time.

[0090] (Configuration 10) The information processing device described in any one of configurations 1 to 9, wherein the determination means determines whether one or more past matching scores included in the timestamp information exceed a threshold, and if none of the past matching scores exceed the threshold, deletes the person ID of the authentication result from the timestamp information at the second time.

[0091] (Configuration 11) The information processing device according to any one of configurations 2 to 4, wherein the determination means controls the notification means to send the second notification to the opening / closing means when the matching score of the person identified by the determination means is lower than the past matching score included in the time-stamp information and the past matching score included in the time-stamp information does not exceed a threshold.

[0092] (Configuration 12) 12. The information processing device according to any one of configurations 1 to 11, wherein the timestamp information management means transmits the timestamp information to the determination means according to at least one condition of the number of records, the number of days, and the time.

[0093] (Configuration 13) 13. The information processing apparatus according to any one of configurations 1 to 12, wherein the timestamp information management means deletes all or part of the timestamp information at any timing, or transmits the information to an external device.

[0094] (Configuration 14) 14. The information processing device according to any one of configurations 1 to 13, further comprising a display control means for displaying on a screen of a display device at least one piece of information selected from the image, personal attribute information including the facial feature amount, the timestamp information, the matching score, and correction history information of the timestamp information.

[0095] (Configuration 15) 15. The information processing device according to any one of configurations 1 to 14, wherein the determination means identifies the person to be authenticated based on time stamp information that includes information related to entry or exit.

[0096] (Configuration 16) the opening / closing means is a gate device having an opening / closing gate, 5. The information processing device according to any one of configurations 2 to 4, wherein the first notification is an instruction to open a gate, and the second notification is an instruction to close a gate.

[0097] (Configuration 17) A control method for an information processing device, comprising: a face detection step of detecting a face area of ​​a person to be authenticated from an image; a feature extraction step of extracting facial feature amounts from the face region; a calculation step of calculating a matching score by matching the facial feature extracted in the feature extraction step with a registered facial feature; a determination step of identifying the person to be authenticated based on time-stamp information including the matching score calculated in the calculation step and at least one matching score linked to a time stamp, In the determining step, the timestamp information at a second time that is an earlier time than the first time is corrected based on the timestamp information at a first time of the identified person. 2. A method for controlling an information processing apparatus comprising:

[0098] (Configuration 18) A program for causing a computer to execute a control method for an information processing device, the program including: a face detection step of detecting a face area of ​​a person to be authenticated from an image; a feature extraction step of extracting facial feature amounts from the face region; a calculation step of calculating a matching score by matching the facial feature extracted in the feature extraction step with a registered facial feature; a determination step of identifying the person to be authenticated based on time-stamp information including the comparison score calculated in the calculation step and at least one comparison score linked to the time stamp; In the determining step, the timestamp information at a second time that is an earlier time than the first time is corrected based on the timestamp information at a first time of the identified person. A program characterized by:

[0099] (Configuration 19) An imaging device; and an information processing device according to any one of configurations 1 to 16. An information processing system comprising: [Explanation of symbols]

[0100] 10 Imaging equipment 11 Information processing equipment 12 Display device 13 Gate device 200 Image acquisition unit 201 Face detection unit 202 Feature Extraction Unit 203 Matching score calculation unit 204 Face Information Management Department 205 Authentication Judgment Unit 206 Time-Stamp Information Management Department 207 Gate Opening / Closing Notification Unit

Claims

1. a face detection means for detecting a face area of ​​a person to be authenticated from an image; a feature extraction means for extracting facial feature amounts from the face region; a calculation means for calculating a matching score by matching the facial feature extracted by the feature extraction means with a registered facial feature; a time-stamp information management means for storing at least one matching score associated with a time stamp as time-stamp information; a determination means for identifying the person to be authenticated based on the matching score calculated by the calculation means and the timestamp information, the determining means corrects the time-stamp information at a second time, which is a time earlier than the first time, based on the time-stamp information at a first time of the identified person; 1. An information processing device comprising:

2. The device further includes a notification means for sending the first notification or the second notification to the predetermined opening / closing means, 2. The information processing apparatus according to claim 1, wherein said determining means determines whether or not the time-stamp information of the person specified by said determining means already exists in said time-stamp information management means.

3. The information processing device according to claim 2, characterized in that, if there is no time-stamp information of the person identified by the determination means, the determination means stores the time-stamp information of the person identified by the determination means and controls the notification means to send the first notification to the opening / closing means.

4. the determination means, when there is timestamp information of the person identified by the determination means, determines whether the verification score of the person identified by the determination means is higher than a past verification score included in the timestamp information; 3. The information processing device according to claim 2, wherein, as a result of the judgment, if the matching score of the person identified by the judgment means is higher than the past matching score, the judgment means stores the timestamp information of the person identified by the judgment means at the current time.

5. When the matching score of the person identified by the determining means is lower than past matching scores, the determining means determines whether one or more past matching scores exceed a threshold; 5. The information processing device according to claim 4, wherein, if any of the past matching scores exceeds the threshold, the determination means corrects the timestamp information at the second time based on the past matching score that exceeded the threshold.

6. The information processing device according to claim 2, characterized in that the determination means controls the notification means to send the second notification to the opening / closing means when the face area cannot be detected, when the face feature amount cannot be extracted, or when the matching score of the person with the highest matching score calculated by the calculation means does not exceed a threshold value.

7. 2. The information processing apparatus according to claim 1, wherein the determining means determines that the person to be authenticated is the person in question when the matching score of the person with the highest matching score calculated by the calculating means exceeds a threshold value.

8. The information processing device according to claim 1, characterized in that the determination means corrects the time-stamp information at the third time when multiple time-stamps occur at a third time that is different from the first and second times due to correction of the time-stamp information at the second time.

9. 2. The information processing device according to claim 1, wherein the determining means determines that the identified person at the first time is a person who has not been registered in advance, in accordance with a correction of the time-stamp information at the second time.

10. The information processing device according to claim 1, characterized in that the determination means determines whether one or more past matching scores included in the timestamp information exceed a threshold value, and if none of the past matching scores exceed the threshold value, deletes the person ID of the authentication result from the timestamp information at the second time.

11. The information processing device according to claim 2, characterized in that the determination means controls the notification means to send the second notification to the opening / closing means when the matching score of the person identified by the determination means is lower than the past matching score included in the time-stamp information and the past matching score included in the time-stamp information does not exceed a threshold value.

12. 2. The information processing apparatus according to claim 1, wherein said time-stamp information management means transmits said time-stamp information to said determination means in accordance with at least one condition of the number of records, the number of days, and the time.

13. 2. The information processing apparatus according to claim 1, wherein said timestamp information management means deletes all or part of said timestamp information at any timing, or transmits said timestamp information to an external device.

14. 2. The information processing device according to claim 1, further comprising a display control means for displaying at least one of the image, the personal attribute information including the facial feature amount, the timestamp information, the matching score, and the correction history information of the timestamp information on a screen of a display device.

15. 2. The information processing apparatus according to claim 1, wherein the determining means identifies the person to be authenticated based on time stamp information including information relating to entry or exit.

16. the opening / closing means is a gate device having an opening / closing gate, 3. The information processing apparatus according to claim 2, wherein the first notification is an instruction to open a gate, and the second notification is an instruction to close a gate.

17. A control method for an information processing device, comprising: a face detection step of detecting a face area of ​​a person to be authenticated from an image; a feature extraction step of extracting facial feature amounts from the face region; a calculation step of calculating a matching score by matching the facial feature extracted in the feature extraction step with a registered facial feature; a determination step of identifying the person to be authenticated based on time-stamp information including the comparison score calculated in the calculation step and at least one comparison score linked to a time stamp, In the determining step, the stump information at a second time that is an earlier time than the first time is corrected based on the stump information at a first time of the identified person.

2. A method for controlling an information processing apparatus comprising:

18. A program for causing a computer to execute a control method for an information processing device, the program including: a face detection step of detecting a face area of ​​a person to be authenticated from an image; a feature extraction step of extracting facial feature amounts from the face region; a calculation step of calculating a matching score by matching the facial feature extracted in the feature extraction step with a registered facial feature; a determination step of identifying the person to be authenticated based on time-stamp information including the comparison score calculated in the calculation step and at least one comparison score linked to the time stamp; In the determining step, the stump information at a second time that is an earlier time than the first time is corrected based on the stump information at a first time of the identified person. A program characterized by:

19. An imaging device; and an information processing device according to any one of claims 1 to 16. An information processing system comprising:

Images