Entrance management method, entrance management system, entrance management device, and entrance management program

Abstract

To reduce a load required for face authentication in an entrance management method, an entrance management system, an entrance management device, and an entrance management program.SOLUTION: An entrance management method is implemented by an entrance management system 1 including an entrance camera 10 that captures an image near an entrance of an area S, an area camera 11 that captures an image of a person in the area S, and an entrance management device 55 that holds a registered database 35 and performs face authentication, and the entrance management device 55 is configured to execute: a step of executing first collation of collating a face image of the entrance camera with the registered database and registering the face image in an entrance candidate database 51; a step of collating the face image acquired by the area camera with the entrance candidate database and registering the face image in a collation result database 53; and a transmission step of transmitting information for specifying a person in the area based on at least one of the entrance candidate database and the collation result database.SELECTED DRAWING: Figure 17

Description

[Technical Field]

[0001] The present disclosure relates to an entrance management method, an entrance management system, an entrance management device, and an entrance management program. [Background technology]

[0002] Traditionally, in educational institutions, to manage student attendance, teachers call out the student's name at the beginning of class, and when the student responds, the teacher confirms that the student is in a designated area such as a classroom. However, there are cases where a student leaves the area during class.

[0003] For example, an attendance management system is known that includes a camera installed in a classroom divided into multiple areas and an information processing device communicatively connected to the camera, in which the camera photographs each of the multiple areas in turn, detects the faces of students attending class within the area, and executes a sequence process multiple times during class to photograph all of the multiple areas, and the information processing device has a receiving unit that receives facial image data of the students from the camera multiple times during class, and a control unit that compares the facial image data with registered facial image data of the students and counts the number of times the registered facial image data of the students is photographed by the camera during class (see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7189097 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when managing attendance in multiple classrooms that are held at the same time using the conventional technology described in Patent Document 1, it is necessary to compare the facial image data of all students attending at that time with the images captured by the camera. Therefore, as the number of students attending at the same time increases, the load required for facial recognition increases, and there is a risk that the information processing device will not be able to process it.

[0006] Therefore, the main purpose of the present disclosure is to reduce the load required for facial authentication in an entrance management method, entrance management system, entrance management device, and entrance management program that use facial authentication to manage entrance. [Means for solving the problem]

[0007] The entrance management method disclosed herein is an entrance management method for managing the entry of people into a predetermined area by authentication using registered face information, and is implemented by an entrance management system including: an entrance camera that captures facial images of people near the entrance of the predetermined area; an area camera that captures facial images of people present within the predetermined area; and an entrance management device that holds a registered database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison to compare first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and performs face authentication on the registered face information that is determined to be the same person as the first face information. a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the specified area if it is determined that the first face information or the registered face information exists in the first matching; a second matching step of performing a second matching of comparing face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching in a matching result database; and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database.

[0008] The entrance management system disclosed herein is an entrance management system that manages the entry of people into a predetermined area by authentication using registered face information, and includes an entrance camera that captures facial images of people near the entrance of the predetermined area, an area camera that captures facial images of people present within the predetermined area, and an entrance management device that holds a registered database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and when the registered face information is determined to be the same person as the first face information, The system is configured to execute a first matching step of determining whether a person exists or not, a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the specified area if the first matching determines that a person exists, a second matching step of matching face information based on a face image acquired by the area camera with face information registered in the entry candidate database, a second registration step of recording the result of the second matching step in a matching result database, and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database.

[0009] The entrance management device disclosed herein is an entrance management device that manages the entry of people into a predetermined area by authentication using registered face information, and is connected to an entrance camera that captures face images of people near the entrance of the predetermined area and an area camera that captures face images of people present within the predetermined area, and holds a registered database in which registered face information is recorded, and performs a first comparison to compare first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and determines whether or not there is registered face information that is determined to be the same person as the first face information; If it is determined that this is the case, the system is configured to execute a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the specified area; a second matching step of matching face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in a matching result database; and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database.

[0010] The entrance management program disclosed herein is an entrance management program that manages the entry of people into a predetermined area through authentication using registered face information, and is executed by an entrance management system that includes an entrance camera that captures facial images of people near the entrance to the predetermined area, an area camera that captures facial images of people present within the predetermined area, and an entrance management device that maintains a registered database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and determines that the person is the same person as the first face information. The system is configured to execute the following steps: a first matching step of determining whether the registered face information exists; if it is determined that the registered face information exists in the first matching step, a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the specified area; a second matching step of matching face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in a matching result database; and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database. [Effects of the Invention]

[0011] According to the present disclosure, it is possible to reduce the load required for face authentication in an entrance management method, an entrance management system, an entrance management device, and an entrance management program. [Brief explanation of the drawings]

[0012] [Figure 1] System configuration diagram of the admission management system according to the first embodiment [Figure 2] Schematic diagram of a classroom where attendance is confirmed by the admission management system according to the first embodiment. [Figure 3]Management Server Block Diagram [Figure 4] Management server functional block diagram [Figure 5] Example of a personal information table [Figure 6] Flow diagram showing the process up to when a face image is registered by the management server [Figure 7] Attendance table example [Figure 8] Facial recognition server block diagram [Figure 9] Functional block diagram of the face recognition server [Figure 10] Example of a face information table [Figure 11] Edge Server Block Diagram [Figure 12] Edge server functional block diagram [Figure 13] Example of a collation database [Figure 14] Example of a match results table [Figure 15] Flow chart showing the admission process [Figure 16] Flow diagram showing the flow of attendance confirmation processing [Figure 17] A time chart showing the flow of processing performed by the entrance management system when face authentication is performed upon entry to a classroom. [Figure 18] A time chart showing the flow of processing after the start of class, performed by the entrance management system when face authentication is performed upon entering a classroom. [Figure 19] System configuration diagram of the entrance management system according to the second embodiment [Figure 20] Functional block diagram of a management server according to a second embodiment [Figure 21] Example of a subject information table [Figure 22] Example of a course information table [Figure 23] 10 is a time chart showing the flow of processing performed by the entrance management system according to the second embodiment when face authentication is performed upon entering a classroom. [Figure 24] Example of face information table of admission management system according to the third embodiment [Figure 25]Time chart of the process from when group settings are recorded in the face information database during course registration [Figure 26] System configuration diagram showing a modified example of an entrance management system in which a face recognition server and an edge server are integrated DETAILED DESCRIPTION OF THE INVENTION

[0013] The first invention made to solve the above problem is an entrance management method for managing the entry of people into a predetermined area by authentication using registered face information, and is implemented by an entrance management system comprising: an entrance camera that captures facial images of people near the entrance of the predetermined area; an area camera that captures facial images of people present within the predetermined area; and an entrance management device that holds a registered database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and determines that the person is the same person as the first face information. a first registration step of registering the first facial information or the registered facial information in an entrance candidate database as facial information of a person who may have entered the specified area if it is determined in the first comparison that the registered facial information exists; a second matching step of performing a second comparison of comparing facial information based on a facial image acquired by the area camera with facial information registered in the entrance candidate database; a second registration step of recording the results of the second comparison in a matching result database; and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entrance candidate database and the matching result database.

[0014] According to this, the entrance management device performs face authentication (second matching) by comparing the captured image acquired by the area camera with the entrance candidate database. Since the facial information of people who have been matched in the first matching using the registered database is registered in the entrance candidate database, the amount of facial information registered in the entrance candidate database is less than or equal to the number of facial information in the registered database. Therefore, compared to matching the captured image acquired by the area camera with the registered database, the amount of facial information to be matched can be reduced, and the load required for face authentication can be reduced.

[0015] In addition, the second invention is the first invention, wherein, when the second matching is not possible in the second matching step, the entrance management device performs a third matching in which the captured image acquired by the area camera is compared with the registered database, and when the third matching is possible, the result is recorded in the matching result database.

[0016] This means that even if the entrance camera is unable to properly photograph a person at the time of entry and the person is unable to register the person in the entry candidate database, facial authentication can be performed by comparing the face with the registered database.

[0017] In addition, the third invention is configured such that, in the above-mentioned second invention, the entrance management device registers the facial information corresponding to the captured image acquired by the area camera when the third matching is completed in the second matching step in the entrance candidate database.

[0018] According to this, even if the second matching is not successful in the second matching step, if the third matching is successful, the face information can be registered in the entry candidate database.

[0019] In addition, a fourth invention is the second invention, wherein the entrance management device is configured to output an alert to notify the corresponding person or administrator when the third matching cannot be performed in the second matching step.

[0020] This makes it possible to notify a person who is not registered in the registered database when the area camera captures an image of the person.

[0021] In addition, a fifth invention is configured in the first invention, wherein the entrance candidate database is configured as a group of the face information included in the registered database.

[0022] In addition, in a sixth aspect of the present invention, the entrance management device is configured to include a face authentication server that performs the first matching, and an edge server that performs the second matching.

[0023] This allows the load associated with the matching to be distributed.

[0024] A seventh invention made to solve the above-mentioned problems is an entrance management system that manages the entry of people into a predetermined area by authentication using registered face information, and includes an entrance camera that captures facial images of people near the entrance of the predetermined area, an area camera that captures facial images of people present within the predetermined area, and an entrance management device that holds a registered database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and performs a second comparison that compares the ... second face information with the The system is configured to execute a first matching step of determining whether facial information exists or not, a first registration step of registering the first facial information or the registered facial information in an entry candidate database as facial information of a person who may have entered the specified area if the first matching determines that facial information exists, a second matching step of matching facial information based on a facial image acquired by the area camera with facial information registered in the entry candidate database, a second registration step of recording the results of the second matching step in a matching result database, and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database.

[0025] According to this, the entrance management device performs face authentication (second matching) by comparing the captured image acquired by the area camera with the entrance candidate database. Since the facial information of people who have been matched in the first matching using the registered database is registered in the entrance candidate database, the amount of facial information registered in the entrance candidate database is less than or equal to the number of facial information in the registered database. Therefore, compared to matching the captured image acquired by the area camera with the registered database, the amount of facial information to be matched can be reduced, and the load required for face authentication can be reduced.

[0026] The eighth invention made to solve the above problem is an entrance management device that manages the entry of people into a predetermined area by authentication using registered face information, the entrance management device being connected to an entrance camera that captures face images of people near the entrance of the predetermined area and an area camera that captures face images of people present within the predetermined area, and holding a registered database in which registered face information is recorded, and performing a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registered database, and determining whether or not there is registered face information that is determined to be the same person as the first face information; If it is determined that the person exists in the specified area, the system is configured to execute a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the specified area, a second matching step of matching face information based on a face image acquired by the area camera with face information registered in the entry candidate database, a second registration step of recording the result of the second matching step in a matching result database, and a transmission step of transmitting information identifying the person present in the specified area to the outside based on at least one of the entry candidate database and the matching result database.

[0027] According to this, the entrance management device performs face authentication by comparing the captured image acquired by the area camera with the entrance candidate database. Since the facial information of the person who was first matched using the registered database is registered in the entrance candidate database, the amount of facial information registered in the entrance candidate database is less than or equal to the amount of facial information in the registered database. Therefore, compared to comparing the captured image acquired by the area camera with the registered database, the amount of facial information to be matched can be reduced, and the load required for face authentication can be reduced.

[0028] A ninth invention made to solve the above-mentioned problems is an entrance management program that manages the entry of people into a predetermined area by authentication using registered face information, and is executed by an entrance management system that includes: an entrance camera that captures facial images of people near the entrance of the predetermined area; an area camera that captures facial images of people present within the predetermined area; and an entrance management device that holds a registration database in which registered face information is recorded and performs face authentication using face information based on the face images captured by the entrance camera and the area camera, and the entrance management device performs a first comparison that compares first face information based on the face image captured by the entrance camera with registered face information recorded in the registration database, and determines that the person is the same person as the first face information. a first registration step of registering the first facial information or the registered facial information in an entrance candidate database as facial information of a person who may have entered the specified area if it is determined that the first facial information or the registered facial information exists in the first matching; a second matching step of matching facial information based on a facial image acquired by the area camera with facial information registered in the entrance candidate database; a second registration step of recording the result of the second matching step in a matching result database; and a transmission step of transmitting information identifying a person present in the specified area to the outside based on at least one of the entrance candidate database and the matching result database.

[0029] According to this, the entrance management device performs face authentication (second matching) by comparing the captured image acquired by the area camera with the entrance candidate database. Since the facial information of people who have been matched in the first matching using the registered database is registered in the entrance candidate database, the amount of facial information registered in the entrance candidate database is less than or equal to the number of facial information in the registered database. Therefore, compared to matching the captured image acquired by the area camera with the registered database, the amount of facial information to be matched can be reduced, and the load required for face authentication can be reduced.

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0031] The entrance management method, entrance management system, entrance management device, and entrance management program disclosed herein are used to perform facial recognition on people present within a predetermined area. Specifically, the entrance management method, entrance management system, entrance management device, and entrance management program disclosed herein can be used, for example, in a system for checking attendance at educational institutions (hereinafter referred to as an attendance checking system), or in an entrance management system for a theater, a concert venue, etc.

[0032] Hereinafter, an embodiment will be illustrated and described in which the admission management method, admission management system, admission management device, and admission management program of the present disclosure are applied to admission management to classrooms for attendance confirmation in an educational institution.

[0033] First Embodiment The educational institution to which the admission management system 1 according to the first embodiment is applied is a university, where lectures are held in multiple classrooms, and students attend by entering (entering) the classroom S. Here, the number of students at the educational institution is assumed to be, for example, several thousand, and the number of students attending classes is assumed to be, for example, several tens. Figure 1 shows a top view of classroom S, where attendance is confirmed by the admission management system 1.

[0034] It is assumed that students have registered their facial information in advance, and the admission management system 1 of the present disclosure can be used to perform facial authentication of people in classroom S using the pre-registered facial information and to confirm attendance. The facial information here may be one or more facial images or facial feature values. In addition, the facial information may be any biometric information as long as it is possible to identify a person using the face.

[0035] FIG. 2 is a configuration diagram of the entrance control system 1 according to the first embodiment.

[0036] The admission management system 1 includes a school affairs system 2 that provides lecture information, attendance information, etc. to students and teachers, and an attendance management system 3 that confirms student attendance.

[0037] The school affairs system 2 may include a management server 5 that stores various information related to students, including attendance information, and user terminals 6 that are carried by the students.

[0038] The attendance management system 3 may include an entrance camera 10 (see also Figure 2) installed at the entrance to classroom S, a classroom camera 11 (area camera, see also Figure 2) that captures images inside classroom S, a face recognition server 12 (authentication server) that stores facial information for face recognition of each student, and an edge server 13 that performs face recognition using images captured by classroom camera 11 and determines the attendance of each student.

[0039] The entrance camera 10, classroom cameras 11, edge server 13, and management server 5 are operated by an educational institution. As shown in Figure 1, the entrance camera 10, classroom cameras 11, edge server 13, and management server 5 are each connected to an in-school communication network 15 (local network) installed within the educational institution.

[0040] The user terminal 6 and the face authentication server 12 are each connected to an external communication network 16 such as the Internet. The internal communication network 15 and the external communication network 16 are connected to each other via a router 17.

[0041] Next, the user terminal 6 and management server 5 that may be included in the school affairs system 2 will be described in detail.

[0042] The user terminal 6 is held by each student as a user. The user terminal 6 may be configured, for example, by a smartphone or the like equipped with a processor (CPU, MPU, etc.), memory such as volatile memory (RAM) or non-volatile memory (ROM), a touch panel display, a camera, a communication interface, and storage such as eMMC.

[0043] The user terminal 6 displays various types of information on the touch panel display, and also receives input of various types of information on the touch panel display.

[0044] The management server 5 manages the personal information and attendance information of students. As shown in Fig. 3, the management server 5 may be configured by a computer equipped with a processor 5A (CPU, MPU, etc.), memory 5B such as volatile memory (RAM) or non-volatile memory (ROM), a communication interface 5C, and storage 5D such as an HDD or SSD.

[0045] As shown in FIG. 4, the management server 5 includes, as functional units, a student information storage unit 21, a student information management unit 22, an attendance information storage unit 23, and an attendance information management unit 24.

[0046] The student information storage unit 21 may be configured with a storage 5D. The student information storage unit 21 stores personal information of students. In this embodiment, the student information storage unit 21 holds a personal information database 25 in which personal information of students is recorded. The personal information database 25 may include a personal information table 25A in which contact information such as an ID (student ID number), name, and email address assigned to each student is recorded in association with the student's personal information. FIG. 5 shows an example of the personal information table 25A. The personal information table 25A may also record a facial image of each student in association with the student's ID.

[0047] The student information management unit 22 can be configured by the processor 5A performing processing in accordance with a predetermined program. The student information management unit 22 acquires personal information from the user terminal 6 and stores it in the student information storage unit 21. The student information management unit 22 also updates the personal information stored in the student information storage unit 21 as appropriate. The personal information acquired by the student information management unit 22 may include a facial image. The facial image acquired by the student information management unit 22 may be acquired by a camera of the user terminal 6, or may be acquired by another imaging device and stored in the user terminal 6.

[0048] 6 shows an example of a time chart when the student information management unit 22 acquires face information from the user terminal 6. In the example shown in FIG. 6, it is assumed that the personal information of the student, including the email address, is recorded in advance in the personal information database 25.

[0049] In the example shown in Figure 6, the student information management unit 22 first refers to the personal information database 25, obtains the email addresses of each student, and sends an instruction to obtain facial information (face information acquisition instruction) to the facial authentication server 12 along with information related to the obtained email addresses (S100).

[0050] When the face authentication server 12 receives the face information acquisition instruction, it acquires the email addresses of each student and sends an email requesting face information registration (hereinafter referred to as face registration request email) to each email address (S101). The face registration request email may include the student ID corresponding to each email address and a link to display the face information registration screen.

[0051] In the example shown in Figure 6, the student information management unit 22 is configured to send a facial information acquisition instruction to the facial authentication server 12 and cause the facial authentication server 12 to send an email, but the student information management unit 22 may also be configured to send a facial registration request email to the user terminal 6.

[0052] When a student receives the face registration request email using the user terminal 6 and opens the link included in the face registration request email, the user terminal 6 displays a face image registration screen (S102). The face image registration screen may be generated by either the management server 5 or the face authentication server 12. The face image registration screen may include text indicating the ID of the student corresponding to the user terminal 6, an input field for inputting information (such as a file name) of a file indicating a face image to be registered, and a send button. The face image registration screen may also include text instructing the user to take a face image using the user terminal 6. When a file name indicating a face image is entered and the send button is touched, the user terminal 6 sends the ID of the corresponding student and the face image to the face authentication server 12 (S103).

[0053] When the face authentication server 12 receives the ID and facial image of the corresponding student from the user terminal 6, it associates the ID with the facial image and / or corresponding facial information and stores (registers) them (S104). Thereafter, the face authentication server 12 associates the ID and facial image acquired from the user terminal 6 with each other and transmits them to the management server 5 (S105), and the management server 5 (student information management unit 22) associates the ID and facial image of the corresponding student with each other and records them in the personal information database 25 (S106).

[0054] Note that when a file name indicating a face image is entered on the face image registration screen and the send button is touched, the user terminal 6 may be configured to transmit the ID of the corresponding student and the face image to the management server 5. At that time, the management server 5 may be configured to register the acquired face image in the personal information database 25, and also to transmit the acquired face image and the corresponding ID to the face authentication server 12, and have the face authentication server 12 register the face information.

[0055] The attendance information storage unit 23 may be configured by the storage 5 D. The attendance information storage unit 23 holds an attendance information database 26 in which attendance information for lectures taken by each student is recorded.

[0056] As shown in Fig. 7, the attendance information database 26 may include an attendance information table 26A in which student ID numbers and the attendance information of the corresponding students are stored in association with each other. The attendance information indicates whether the corresponding student attended the lectures (also called classes) that they are taking on each period on each date. In the example of Fig. 7, a circle recorded in the attendance information table 26A indicates that attendance was confirmed, and an × indicates that attendance was not confirmed (i.e., the student was absent or there was no lecture to take).

[0057] The attendance information management unit 24 can be configured by the processor 5A performing processing according to a predetermined program. After each lecture, the attendance information management unit 24 obtains the attendance determination result from the edge server 13 and writes the obtained attendance determination result to the attendance information database 26.

[0058] Next, the entrance camera 10, classroom camera 11, face authentication server 12, and edge server 13 that may be included in the attendance management system 3 will be described in detail.

[0059] As shown in FIG. 1, entrance camera 10 captures facial images of people near the entrance to classroom S (a specified area). More specifically, entrance camera 10 is installed near the entrance to classroom S and captures images of people passing through the entrance to classroom S. The imaging range of entrance camera 10 includes the facial portion of people entering classroom S. As shown in FIG. 2, entrance camera 10 is connected to an in-school communication network 15 installed within the educational institution, and transmits the captured images to edge server 13 via communication network 15.

[0060] As shown in FIG. 1, classroom camera 11 is installed in classroom S and captures facial images of people present in classroom S. More specifically, classroom camera 11 captures images of students attending a lecture in classroom S. The imaging range of classroom camera 11 includes the faces of the students attending the lecture. It is preferable to install multiple classroom cameras 11 in classroom S so that the faces of all students attending the lecture in classroom S can be captured. As shown in FIG. 2, classroom camera 11 is connected to an in-school communication network 15 and transmits the captured images to edge server 13 via that communication network 15.

[0061] As shown in Figure 8, the facial recognition server 12 may be configured by a computer equipped with a processor 12A (CPU, MPU, etc.), memory 12B such as volatile memory (RAM) or non-volatile memory (ROM), a communication interface 12C, and storage 12D such as an HDD or SSD. The facial recognition server 12 is connected to an off-campus communication network 16 and corresponds to a cloud server that executes various processes related to facial recognition. Specifically, the facial recognition server 12 stores facial information for facial recognition, provides the facial information to the edge server 13, and performs processes such as face detection and facial recognition from images acquired by the entrance camera 10.

[0062] As shown in FIG. 9, the face authentication server 12 includes, as functional units, a face information storage unit 31, a face detection unit 32, a feature amount acquisition unit 33, and an entrance processing unit .

[0063] The face information storage unit 31 may be configured by the storage 12D. The face information storage unit 31 holds a face information database 35 in which IDs and face information are recorded in association with each other. The face information database 35 may include a face information table 35A in which the face information of all students entered by the face information storage unit 31 from the user terminal 6 in response to a face registration request email is recorded together with the student's ID.

[0064] An example of face information table 35A is shown in Fig. 10. In face information table 35A, IDs, face information, and the names of students corresponding to the IDs are stored in association with each other. As shown in Fig. 10, the face information may include file names indicating face images and file names of data indicating facial feature amounts.

[0065] When students enroll, they are assigned an ID (student ID number) and their facial information is registered in the facial information database 35. Therefore, when attendance management is performed by the admission management system 1, the facial information database 35 is a registered database in which registered facial information is recorded.

[0066] The face detection unit 32 can be configured by the processor 12A executing a predetermined program. The face detection unit 32 uses a known method to detect the face of a person from the captured image acquired by the entrance camera 10. When a student enters the classroom S, the face of the person entering the classroom S is included in the image capturing range of the entrance camera 10, so the face of the student entering the classroom S is captured and the face detection unit 32 detects the face in the captured image.

[0067] The feature acquisition unit 33 can be configured by the processor 12A executing a predetermined program. The feature acquisition unit 33 acquires (also called generates) facial features using the facial parts detected by the face detection unit 32. Facial features are numerically converted information useful for identifying a person, such as the shape and positional relationship of the eyes, nose, etc., or are numerical data generated from a facial image using a trained model of an AI for generating facial features based on known deep learning technology. By comparing the facial features with previously acquired facial information, it becomes possible to determine whether the person is the same or not.

[0068] The entrance processing unit 34 can be configured by the processor 12A executing a predetermined program. The entrance processing unit 34 uses the feature amounts acquired by the feature acquisition unit 33 and the face information database 35 to perform face matching of the person captured by the entrance camera 10. If the face information database 35 is able to extract (match) face information corresponding to the person captured by the entrance camera 10, the entrance processing unit 34 transmits the ID (student ID number) and face information of that person to the edge server 13. As a result, the ID of the student who entered the classroom S where the entrance camera 10 is installed through the entrance and the corresponding face information are transmitted to the edge server 13. In this embodiment, the entrance processing unit 34 may transmit the name of the person who has been matched to the edge server 13 in addition to the ID and face information of the person.

[0069] 11, the edge server 13 may be configured by a computer including a processor 13A (CPU, MPU, etc.), memory 13B such as volatile memory (RAM) or nonvolatile memory (ROM), a communication interface 13C, and storage 13D such as a storage device. The edge server 13 is installed in each classroom S, performs face authentication using images captured by the classroom camera 11, determines attendance, and transmits the results to the management server 5. Note that the edge server 13 may be installed anywhere within the building in which the classroom S is located, instead of in each classroom S.

[0070] As shown in FIG. 12, the edge server 13 has, as its functional units, a matching data storage unit 41, a matching data management unit 42, a face detection unit 43, a feature acquisition unit 44, a face matching unit 45, a matching result storage unit 46, a matching result update unit 47, a matching data update unit 48, and an attendance determination unit 49.

[0071] The verification data storage unit 41 may be configured by the storage 13D. The verification data storage unit 41 holds a verification database 51 for performing facial recognition of students photographed in the classroom S using the classroom camera 11. As shown in FIG. 13 , the verification database 51 stores the IDs (student ID numbers) of individuals (students) whose faces have been recognized using images captured by the entrance camera 10, and the corresponding facial information, in association with each other. In this embodiment, the verification database 51 stores the names of the corresponding students in addition to the IDs and the corresponding facial information.

[0072] The verification data management unit 42 can be configured by the processor 13A executing a predetermined program. The verification data management unit 42 resets the verification database 51 as necessary after the end of a lecture held before the lecture for which attendance is to be confirmed and before the ID and facial information are transmitted from the entrance processing unit 34. The verification data management unit 42 then acquires the ID and facial information transmitted from the entrance processing unit 34 via the communication interface 13C and writes them to the verification database 51 held by the verification data storage unit 41. As a result, when a student enters classroom S through the entrance and undergoes facial authentication using an image captured by the entrance camera 10, the student's ID and corresponding facial information are recorded in the verification database 51. In other words, the verification database 51 is a database (entry candidate database) that records facial information of people who are believed to have entered classroom S (within the area). The verification data management unit 42 may be configured to discard the data recorded in the verification database 51 and reset the verification database 51 when the lecture ends and attendance determination is completed.

[0073] It should be noted that the facial information of the person who has entered the facility written into the verification database 51 by the verification data management unit 42 is not limited to the facial information registered in the facial information database 35. For example, the facial information of the person who has entered the facility written into the verification database 51 may be facial information corresponding to an image captured by the entrance camera 10 (the captured image, a facial image acquired from the captured image, facial feature amounts acquired from the captured image, etc.).

[0074] The face detection unit 43 can be configured by the processor 13A executing a predetermined program. The face detection unit 43 uses a known method to detect the facial portion of a person from the captured image acquired by the classroom camera 11. Since the classroom camera 11 captures images of the facial portions of all students attending the lecture in the classroom S, the facial portions of all students attending the lecture in the classroom S are captured, and the facial portions of the captured images are detected by the face detection unit 43.

[0075] The feature amount acquiring unit 44 can be configured by the processor 13A executing a predetermined program. The feature amount acquiring unit 44 acquires (also called generates) facial feature amounts using the face portion detected by the face detection unit 43.

[0076] The face matching unit 45 can be configured by the processor 13A executing a predetermined program. The face matching unit 45 acquires an image from the classroom camera 11 at each matching timing (hereinafter referred to as the matching timing) and detects a face portion using the face detection unit 43. The matching timing may be set to a predetermined time (e.g., every 15 minutes) after the start of class. The face matching unit 45 then acquires facial features from the detected face portion using the feature acquisition unit 44. Next, the face matching unit 45 compares all the acquired facial features with the matching database 51 stored in the matching data storage unit 41 and acquires the IDs of all students who have been matched. The face matching unit 45 then outputs the IDs of all students who have been matched to the matching result update unit 47. A matched student means that the student was in the classroom S at the time the image was captured by the classroom camera 11. Furthermore, if there are facial features that could not be matched in the matching database 51, the face matching unit 45 outputs the facial features to the matching data update unit 48.

[0077] The matching result storage unit 46 may be configured by the storage 13D. After the lecture starts, the matching result storage unit 46 stores, as a matching result database 53, information (matching result information) indicating whether matching and face authentication were successful at each matching timing.

[0078] The matching result database 53 may include a matching result table 53A in which matching result information is recorded. An example of the matching result table 53A is shown in Fig. 14. The matching result table 53A records the ID of a student whose entry into classroom S or whose presence in classroom S has been confirmed, in association with a truth value indicating whether or not the student has been matched (face authentication has been performed) by the face matching unit 45 at each matching timing.

[0079] If the face matching unit 45 is able to perform face recognition at the matching timing, it means that the person whose face has been recognized was staying in classroom S. In other words, the matching result database 53 records that the person whose face has been recognized was staying in classroom S at the matching timing.

[0080] The matching result table 53A may also record, for each ID, whether or not face authentication has been performed by the entrance camera 10.

[0081] The matching result update unit 47 can be configured by the processor 13A executing a predetermined program. The matching result update unit 47 associates the matching timing with the IDs of all students who were matched by the face matching unit 45 at that matching timing, writes the association information into the matching result table 53A, and updates the matching result database 53.

[0082] The matching data update unit 48 can be configured by the processor 13A executing a predetermined program. The matching result update unit 47 references the face information table 35A stored in the face authentication server 12 and acquires, from the face information of all students, data that matches the facial features that could not be matched by the face matching unit 45. When face information that matches the facial features that could not be matched by the face matching unit 45 is acquired, the matching result update unit 47 acquires the matching face information, the corresponding ID (student ID number), and the name of the corresponding student, and causes the matching result update unit 47 to update the matching database 51 stored in the matching data storage unit 41. As a result, the ID, facial information, and name of the student who was not properly photographed by the entrance camera 10 at the time of entry but whose face was photographed by the classroom camera 11 and whose face was authenticated are added to the matching database 51, and the matching database 51 is updated.

[0083] When the matching data update unit 48 is able to obtain, from the facial information of all students, facial information that matches the facial feature that could not be matched by the face matching unit 45, it may record in the matching result database 53 the student's ID and the fact that the student corresponding to that ID was facially authenticated in classroom S. In this way, even if a student was not properly photographed by the entrance camera 10 when entering but was photographed by the classroom camera 11 and facially authenticated, it is recorded in the matching result database 53 that the student was staying in classroom S.

[0084] The attendance determination unit 49 can be configured by the processor 13A executing a predetermined program. When the lecture ends, the attendance determination unit 49 determines whether or not a student recorded in the matching result database 53 can be considered to have attended the lecture (attendance determination) based on the matching result database 53. When making this determination, the attendance determination unit 49 may refer to the true / false values ​​recorded in the matching result database 53. For example, the attendance determination unit 49 may take into consideration the determination accuracy of the classroom camera 11 and face matching, and determine that the student was present when 90% or more of the true / false values ​​recorded in the matching result database 53 are true (o). The attendance determination unit 49 performs attendance determination for each ID recorded in the matching result database 53, and transmits the determination result together with the corresponding ID to the management server 5 as an attendance determination result.

[0085] When the management server 5 (attendance information management unit 24) acquires the attendance determination result from the edge server 13 (attendance determination unit 49), the management server 5 writes the acquired attendance determination result in the attendance information database 26.

[0086] Next, an overview of the entrance management processing performed by the entrance management system 1 by executing the entrance management program in order to implement the entrance management method of the present invention will be explained with reference to the flow chart shown in Figure 15 and the flow chart shown in Figure 16.

[0087] 15 shows the flow of processing (hereinafter referred to as entrance processing) executed by the face authentication server 12. The entrance processing starts after the end of a lecture that is being held in the same classroom S before the lecture for which attendance is being confirmed.

[0088] When the entrance process begins, the face authentication server 12 first acquires a captured image from the entrance camera 10 and performs face detection to detect a human face from the captured image (S200). If a face is detected, the face authentication server 12 acquires facial features (first facial information) based on the facial image acquired by the entrance camera 10, and performs a first matching using a face information database 35 (registered database) in which the facial information of all students is registered, to determine whether face authentication is possible (i.e., whether face information determined to be the same person as the person corresponding to the facial features of the captured image exists in the face information database 35) (S201, first matching step).

[0089] If the first matching is successful (i.e., if facial information determined to be the same person as the person corresponding to the facial feature amount of the captured image exists in the facial information database 35, then the answer in S202 is Yes), the facial authentication server 12 transmits the facial information that matches the facial feature amount of the captured image, the corresponding ID (student ID number), and the name to the edge server 13 (S202). If facial authentication is not successful (i.e., if facial information that matches the facial feature amount of the captured image cannot be found in the facial information database 35, then the answer in S202 is No), the facial authentication server 12 performs face detection on the captured image (S200) and determines whether facial authentication is possible (S201).

[0090] When the face authentication server 12 transmits face information that matches the face feature amounts of the image captured by the entrance camera 10, the corresponding ID (student ID number), and the name to the edge server 13, the edge server 13 registers the acquired face information, ID, and name in the matching database 51 as face information of a person who may have entered the classroom S (first registration step). Note that the face information recorded in the matching database 51 may be face information corresponding to the image captured by the entrance camera 10 (such as a face image included in the captured image or face feature amounts acquired from the captured image).

[0091] When the face information that matches the facial feature amount of the captured image, the corresponding ID (student ID number), and the name are sent to the edge server 13, the face authentication server 12 instructs the management server 5 to write in the attendance information database 26 that the entry of the student corresponding to that ID has been confirmed (S203). As a result, when an image is captured by the classroom camera 11 and face authentication is performed, it is recorded in the attendance information database 26 that the student may have entered the classroom S. At this time, the face authentication server 12 may send an entry confirmation notification from the management server 5 to the user terminal 6, indicating that the student may have entered the classroom S, and cause the user terminal 6 to display that entry has been confirmed.

[0092] Next, the face authentication server 12 determines whether or not the end time of the lecture for which attendance is being confirmed has arrived (S204). If the end time of the lecture for which attendance is being confirmed has arrived (Yes in S204), the entrance process is completed, and if the end time has not arrived (No in S204), the face authentication server 12 acquires a captured image again and performs face detection (S200).

[0093] 16 shows the flow of processing (hereinafter referred to as attendance confirmation processing) performed by the edge server 13 by executing the entrance management program. The attendance confirmation processing is a process that takes images using the classroom camera 11 while the lecture is in progress, performs face authentication (second matching step), records that each student is in classroom S at the time of face authentication (second registration step), performs attendance determination to identify people who were in classroom S and attending the lecture, and transmits information identifying those people to an external party (management server 5) (transmission step), and is started after the lecture has started.

[0094] When the attendance confirmation process starts, the edge server 13 first determines whether it is time for verification (S300). If it is not time for verification (No in S300), the edge server 13 waits until it is time for verification. If it is time for verification (Yes in S300), the edge server 13 acquires an image captured by the classroom camera 11 and performs face detection (S301).

[0095] If a face is detected, the edge server 13 acquires facial features from the image of the detected face portion, and performs a comparison (second comparison) between the acquired facial information and facial information recorded in the comparison database 51 to perform facial authentication (S302, second comparison step). If comparison is successful using the comparison database 51 (i.e., face authentication is successful), the edge server 13 acquires the corresponding ID from the comparison database 51 and records in the comparison result table 53A that face authentication was successful (i.e., attendance confirmation was successful) in association with the comparison timing (e.g., comparison number, time) at which the comparison was performed (S303, second registration step). As a result, it is recorded in the comparison result database 53 that the person whose face was recognized at the time of the second comparison was present in classroom S.

[0096] If matching is not possible (face authentication is not possible), the edge server 13 sends a matching command to the face authentication server 12 requesting that the face information (face part or face feature amount in the captured image) obtained from the image captured by the classroom camera 11 be matched with the face information database 35 held in the face authentication server 12 (S304).

[0097] When the face authentication server 12 receives the matching command, it performs a comparison (third comparison) between the face information acquired from the edge server 13 and the face information database 35, and determines whether there is matching face information in the face information database 35. If there is matching face information, the face authentication server 12 sends the matching face information, the ID (student ID number) and the name that match the face information, along with a notification that the comparison was successful, to the edge server 13. If there is matching face information, the face authentication server 12 sends a notification to the edge server 13 that the comparison was not successful.

[0098] When the edge server 13 receives a notification that the matching was successful (Yes in S305), it adds the face information sent from the face authentication server 12 and the corresponding ID and name to the matching database 51 (S306). When the edge server 13 receives a notification that the matching was unsuccessful (No in S305), it again determines whether it is time to perform matching (S300). At this time, the edge server 13 may output (transmit) an alert signal to various terminals including the user terminal 6 to notify the corresponding person, administrator, etc. that the matching was unsuccessful.

[0099] Upon receiving the notification that matching has been successful and completing the addition of facial information, etc. to the matching database 51, the edge server 13 adds a new ID to the matching result table 53A and records the matching timing (matching number, time, etc.) and information indicating that facial authentication was successful in association with each other (S303). As a result, it is recorded in the matching result database 53 that the person whose face was recognized during the third matching was present in classroom S.

[0100] When recording in the matching result table 53A (S303) is completed, the edge server 13 determines whether the matching end time has arrived (or passed) (S307). The matching end time may be set to be the same as the end time of the lecture, or may be set to be slightly earlier than the end time of the lecture. If the matching end time has not arrived, the edge server 13 determines whether it is time to perform face matching again (S300).

[0101] When the matching end time has arrived, the edge server 13 performs attendance determination for students with each ID registered in the database based on at least one of the matching database 51 and the matching result database 53, and transmits information (e.g., ID, etc.) identifying the person who attended (a person whose presence was confirmed in classroom S during class time) to the management server 5 (S308, transmission step). For example, the edge server 13 may perform attendance determination based only on the matching result database 53. Specifically, the edge server 13 may determine that the student was present when information indicating that face authentication was performed at all matching timings is recorded in the matching result database 53. The edge server 13 may also determine that the student was present when information indicating that face authentication was performed at 90% or more of all matching timings is recorded in the matching result database 53.

[0102] When the management server 5 receives the attendance judgment results, it records the received judgment results for each ID in the attendance information database 26. At the same time, the management server 5 transmits the received judgment results to each of the user terminals 6 of the corresponding students. When the user terminal 6 receives the judgment results, it displays that fact on the touch panel display.

[0103] When the notification of the determination result is completed, the edge server 13 ends the attendance confirmation process.

[0104] Next, the admission management process performed by the admission management system 1 to implement the admission management method according to the first embodiment will be described in detail with reference to the time charts shown in Figures 17 and 18. Figures 17 and 18 illustrate an extracted process related to one student.

[0105] The upper half of Fig. 17 shows a time chart after the end of a lecture before attendance confirmation has taken place, but before the start of a lecture for which attendance confirmation is to take place. The lower half of Fig. 17 shows a time chart after the lecture has started, when face authentication is performed upon entering (entering) classroom S and face registration is performed in the verification database 51.

[0106] 17, after the lecture held in the period preceding the lecture for which attendance is to be confirmed has ended, entrance camera 10 takes an image (S400), and face authentication server 12 acquires the captured image taken by entrance camera 10 (S401). Then, face authentication server 12 performs face authentication by comparing the image with face information database 35 in which face information of all students is recorded (S402, first matching step). If face authentication is successful, face authentication server 12 transmits the face information (first face information) recorded in face information database 35, along with the ID and name, to edge server 13 (S403), and records the information in matching database 51 (S404, first registration step). As a result, the face information of the person whose face has been recognized by the image captured by entrance camera 10 is recorded in matching database 51 as face information of a person who may have entered classroom S. The facial information recorded in the matching database 51 may be facial information (registered facial information) in the facial information database 35 (registered database), or may be facial information (facial image or facial features) corresponding to the image captured by the entrance camera 10.

[0107] When the lecture begins, the edge server 13 acquires images (S500) taken by the classroom camera 11 at each matching timing (S501) and performs face authentication using the matching database 51 (S502, second matching step). The edge server 13 records the results of the face authentication in the matching result database 53. The edge server 13 performs face authentication based on the images taken by the classroom camera 11 at each matching timing until the matching end time has passed, and records the results in the matching result database 53 (second registration step). As a result, if face authentication is successful at the matching timing, it is recorded in the matching result database 53 that the person whose face was authenticated was present in the classroom S.

[0108] After the matching end time has passed, the edge server 13 uses at least one of the matching database 51 and the matching result database 53 (in this embodiment, the matching result database 53) to determine attendance for each student (S503), and transmits the attendance determination results to the management server 5 (S504, transmission step).

[0109] When the management server 5 receives the attendance determination results, it records the received determination results for each ID in the attendance information database 26 (S505). At the same time, the management server 5 transmits the received determination results to each of the user terminals 6 of the corresponding students (S506). When the user terminal 6 receives the determination results, it displays that fact on the touch panel display (S507).

[0110] The management server 5 stores the course registration information of each student and generates a list of students attending the lecture for which attendance was confirmed based on the course registration information. The management server 5 may then determine whether each student included in the list was present or absent based on the attendance determination results of the edge server 13. Upon obtaining the attendance / absence determination results, the management server 5 may be configured to transmit the determination results to the user terminal 6.

[0111] 18 shows a time chart for the case where a face is not photographed when entering (entering) classroom S, but is photographed and facial authentication is performed inside classroom S. Even in the case shown in FIG. 18, when the lecture starts, edge server 13 acquires an image (S500) taken by classroom camera 11 at the timing of verification (S501) and performs facial authentication using verification database 51 (S502). In this case, because a face was not photographed when entering, the facial information of the student photographed is not recorded in verification database 51, and therefore facial authentication fails.

[0112] If matching is not possible (face authentication is not possible), the edge server 13 sends a matching command to the face authentication server 12 requesting that the face information obtained from the image captured by the classroom camera 11 be matched against the face information database 35 (S600).

[0113] When the face authentication server 12 receives the matching command, it performs face authentication by using the face information database 35 to determine whether or not there is face information matching the face information acquired from the image captured by the classroom camera 11 (S601). If there is matching face information, the face authentication server 12 transmits the matching face information and the corresponding ID and name to the edge server 13 (S602). When the edge server 13 receives the matching face information, ID, and name from the face authentication server 12, it adds and registers the matching face information to the matching database 51. At this time, if there is matching face information, the edge server 13 adds the person's ID to the matching result database 53, adds a new ID, and records the matching timing and information indicating that face authentication was successful in association with each other. As a result, it is recorded in the matching result database 53 that the person was present in classroom S (S603).

[0114] Thereafter, in the same manner as when face authentication was performed upon entering classroom S, an image taken by classroom camera 11 is acquired by edge server 13 at each matching timing, and face authentication is performed (S500 to S502).

[0115] When the lecture is about to end, the edge server 13 performs attendance determination (S503) in the same way as when face authentication is performed upon entry into classroom S, and transmits the result to the management server 5 (S504). The management server 5 records the determination result in the attendance information database 26 (S505) and transmits the determination result to the user terminal 6 (S506). The user terminal 6 displays the transmitted determination result (S507).

[0116] Next, the effects of the admission control system 1 configured in this manner will be described.

[0117] 17, when a facial image is acquired by the entrance camera 10 (S401), the facial authentication server 12 performs a comparison (first comparison) between the facial image acquired by the entrance camera 10 and the facial information database 35 (registered database) (S402). Thereafter, the facial authentication server 12 transmits the facial information in the facial information database 35 (registered database) that has been matched to the edge server 13 (S403), and the edge server 13 registers the received facial information in the comparison database 51 (entrance candidate database) (S404).

[0118] When the lecture starts, the edge server 13 performs a comparison (second comparison) between the facial image captured by the classroom camera 11 (area camera) and the comparison database 51 (entry candidate database) (S502). The edge server 13 determines whether or not the person attended the lecture based on the result of the comparison (second comparison) of the image captured by the classroom camera 11 using the comparison database 51, i.e., the data in the comparison result database 53, and sends the determination result to the management server 5 (S503, S504). Upon receiving the determination result, the management server 5 records in the attendance information database 26 that the person was determined to have attended as a result of face authentication (S505).

[0119] As a result, of the facial information recorded in the facial information database 35, the facial information corresponding to the student who entered the classroom S is registered in the matching database 51. The entrance management system 1 performs facial authentication by matching the captured image acquired by the classroom camera 11 with the matching database 51. The facial authentication server 12 and the edge server 13 constitute an entrance management device 55 (see FIGS. 17 and 18) that holds the facial information database 35 in which the registered facial information is recorded and performs facial authentication using the captured image acquired by the classroom camera 11.

[0120] Of the facial information recorded in face information database 35, facial information corresponding to students who have entered classroom S is registered in matching database 51, so the amount of facial information registered in matching database 51 is equal to or less than the amount of facial information recorded in face information database 35. For example, the amount of facial information registered in face information database 35 corresponds to the number of students at an educational institution (up to several thousand people), and the amount of facial information that can be registered in matching database 51 is thought to be about the number of students attending classes (approximately several tens to several hundred people). Therefore, compared to matching images captured by classroom cameras 11 with face information database 35, the amount of facial information to be matched can be reduced, thereby reducing the load required for face authentication.

[0121] The entrance management device 55 also includes a face authentication server 12 that performs matching using images captured by the entrance camera 10, and an edge server 13 that performs matching using images captured by the classroom cameras 11. Therefore, face authentication based on images captured by the classroom cameras 11 is performed in the edge server 13, except when there is no face information in the matching database 51. This allows the processing load related to matching required for attendance confirmation to be distributed to the edge server 13, thereby reducing the load on the face authentication server 12.

[0122] After the lecture has started, if the image captured by the classroom camera 11 cannot be matched with the matching database 51, the edge server 13 sends a matching command to the face authentication server 12 (S600). In response, the face authentication server 12 performs a match (third match) between the image captured by the classroom camera 11 and the face information database 35 (S601). If the match is successful, the edge server 13 records in the matching result database 53 that the person was present in the classroom S (S603). After the lecture has ended, the edge server 13 determines whether the person attended the lecture based on the contents recorded in the matching result database 53, and sends the determination result to the management server 5 (S602). Upon receiving the determination result, the management server 5 records in the attendance information database 26 that the person was determined to have attended as a result of face authentication (S603).

[0123] This means that, for example, even if the entrance camera 10 is unable to properly photograph a person at the time of entry and the person is unable to register the person in the matching database 51, facial authentication can be performed by matching the person with the face information database 35, thereby improving the convenience of the entrance management system 1.

[0124] Second Embodiment The admission management system 1 according to the second embodiment differs from the first embodiment in that it compares the matching results obtained by the face authentication server 12 and the edge server 13 with course registration information, including information about the students, and records the results in the attendance information database 26. The admission management system 1 according to the second embodiment also differs in that it includes an administrator terminal 71 held by an administrator who is a teacher, as shown in Fig. 19, and takes the course registration information into consideration and notifies the administrator terminal 71 and the user terminal 6. The other configurations are generally the same as those of the first embodiment, so a description of the similar configurations will be omitted.

[0125] The administrator terminal 71 may be configured, for example, by a tablet terminal equipped with a memory such as a volatile memory (RAM) or a nonvolatile memory (ROM), a touch panel display, a camera, a communication interface, and storage such as a smartphone or a laptop computer.

[0126] The administrator terminal 71 is connected to the school's communication network 15. The administrator terminal 71 is configured to be connectable to the edge server 13 via the school's communication network 15, and is configured to be connectable to the face authentication server 12 via the router 17 and the off-campus communication network 16.

[0127] Similar to the user terminal 6, the administrator terminal 71 displays various information on a touch panel display and also accepts input of various information on the touch panel display.

[0128] As shown in FIG. 20, the management server 5 includes a student information storage unit 21, a student information management unit 22, an attendance information storage unit 23, an attendance information management unit 24, as well as a lecture information management unit 81, a course enrollment information management unit 82, and a management display generation unit 83.

[0129] The lecture information management unit 81 has a lecture information database 85 that holds information about lectures held in educational institutions (hereinafter referred to as lecture information), and stores and manages the lecture information.

[0130] The lecture information database 85 may include a subject information table 85A in which lecture IDs, corresponding subject names, days and periods when the lectures are held, and classroom names are recorded in association with each other as lecture information. An example of the subject information table 85A is shown in Figure 21. The lecture information management unit 81 may update the lecture information recorded in the subject information table 85A by accepting input from the administrator terminal 71.

[0131] The course information management unit 82 acquires and manages data related to the lectures that each student takes from the user terminal 6, etc. The course information management unit 82 holds a course information database 87 in which information related to the lectures that each student takes (hereinafter referred to as course information) is recorded.

[0132] The course information database 87 includes a course information table 87A in which each student's ID, the day of the week, the time period, and the lecture ID of the lecture that the student is taking are recorded in association with each other. An example of the course information table 87A is shown in Figure 22. The course information management unit 82 may update the course information recorded in the course information table 87A by accepting input from the user terminal 6 or the administrator terminal 71.

[0133] The ID (student ID number) functions as a key that associates the personal information table 25A with the course registration information table 87A, and the lecture ID functions as a key that associates the subject information table 85A with the course registration information table 87A. These keys configure a relational database in which the personal information table 25A, the course registration information table 87A, and the subject information table 85A are associated with each other.

[0134] When the lecture held in the period before the lecture for which attendance is to be confirmed has ended, the management display generation unit 83 uses the personal information table 25A, the course registration information table 87A, and the subject information table 85A to extract the students who will be attending the lecture for which attendance is to be confirmed, and generates a student group.

[0135] The management display generation unit 83 stores a student status table in which the attendance status and attendance absence status of each student included in a student group are recorded, and updates the student status table at the start and end of a lecture.

[0136] The student status table stores the ID and name of each student, the corresponding attendance information, and the corresponding attendance information in association with each other. The attendance information is a true / false value, and becomes false when the lecture held in the period before the lecture for which attendance is being confirmed has ended, and becomes true when the student (participant) corresponding to the ID is confirmed to have entered classroom S. The attendance information is also a true / false value, and corresponds to the result of attendance determination by edge server 13.

[0137] When the face authentication server 12 performs face authentication based on the image capture results of the entrance camera 10, the management display generation unit 83 acquires the authentication result. Upon acquiring the authentication result, the management display generation unit 83 uses the generated student group and the authentication result to set the attendance information of the face-authenticated student as true. The management display generation unit 83 then transmits the attendance information of all students in the student group to the administrator terminal 71. Upon receiving the attendance information of all students in the student group, the administrator terminal 71 displays the attendance information of all students in the student group on the touch panel display.

[0138] When the management display generation unit 83 obtains the attendance determination results from the edge server 13, it uses the generated student groups and the attendance determination results to generate attendance information for all students, associating each student's ID and name with the attendance determination results. The management display generation unit 83 then transmits the attendance information for all students to the administrator terminal 71. When the administrator terminal 71 receives the attendance information for all students, it displays the attendance information for all students on the touch panel display.

[0139] FIG. 23 shows a time chart of the entrance management process performed by the entrance management system 1 to implement the entrance management method according to the second embodiment, in which face authentication is performed upon entry into classroom S.

[0140] 23, after the lecture held in the period preceding the lecture for which attendance is to be confirmed has ended, the entrance camera 10 takes an image (S400), and the facial recognition server 12 acquires the image taken by the entrance camera 10 (S401). The facial recognition server 12 then performs facial recognition by comparing the image with the facial information database 35 in which the facial information of all students is recorded (S402). If facial recognition is successful, the facial recognition server 12 transmits the facial information recorded in the facial information database 35, together with the ID and name, to the edge server 13 (S403), and records them in the comparison database 51 (S404).

[0141] Also, at this time, if the face authentication server 12 is successful in face authentication, it sends authentication data including face information, ID, and name to the management server 5 (S700). When the management server 5 receives the authentication data, it compares it with the student group and updates the attendance information of the students whose faces have been authenticated (S701). When the attendance information updating is complete, the management server 5 sends the attendance information of all students to the management terminal (S702). When the administrator terminal 71 receives the attendance information, it displays the received attendance information on the touch panel display (S703).

[0142] When the lecture begins, as in the first embodiment, an image is taken by the classroom camera 11 at each matching timing (S500), the captured image is acquired by the edge server 13 (S501), and facial authentication is performed using the matching database 51 (S502).

[0143] When the collation end time has passed, the edge server 13 performs attendance determination (S503) as in the first embodiment, and transmits the determination result to the management server 5 (S504).

[0144] When the management server 5 acquires the assessment results, it compares them with the student group (S704) and records the attendance assessment results for each student as attendance information in the student status table (S705). Once the attendance information for all students has been recorded, the management server 5 transmits the attendance assessment results for the students to each of the user terminals 6 held by the students (S706). When the user terminal 6 receives the assessment results, it displays the received attendance assessment results on its touch panel display (S707).

[0145] As a result, a message indicating that the student is considered absent is also displayed on the user terminal 6 of the student who was absent from the lecture. In consideration of the fact that face authentication was not performed properly, it is advisable to provide a predetermined input field and a send button on the screen on which the message indicating that the student is considered absent is displayed.

[0146] If there is a student who is not included in the student group but is determined to be present in the attendance assessment, the management server 5 may identify the student's email address and send an email to that email address notifying that the student has not registered for the course. In this case, the email may include a link to an input page for registering for the course or a link to a page for filling out a questionnaire.

[0147] Thereafter, the management server 5 transmits the attendance information of all the students to the administrator terminal 71 (S708). When the management terminal receives the attendance information of all the students, it displays the received attendance information on the touch panel display (S709). At this time, the administrator terminal 71 may display the positions and names of the students whose attendance has been confirmed on the touch panel display.

[0148] When the management server 5 is unable to perform a match using the matching database 51 of the edge server 13, it transmits a matching command to the face authentication server 12, as in the first embodiment. Upon receiving the matching command, the face authentication server 12 performs face matching using the face information database 35 based on the face information that the edge server 13 was unable to match.

[0149] In this case, if matching using the face information database 35 is not possible (i.e., face matching with all students is not possible), the face authentication server 12 notifies the management server 5 of this fact. At this time, the management server 5 sends an alert signal to the administrator terminal 71 indicating that there is a student who cannot be authenticated even with the face information of all students. The administrator terminal 71 displays on its touch panel display that a student who could not be matched with the face information of all students is taking the course.

[0150] The management server 5 may be able to send various signals to the terminals carried by persons whose faces could not be matched using the facial information database 35. In such cases, the management server 5 may send an alert signal indicating that the facial information could not be matched to the terminals carried by persons whose faces could not be matched using the facial information database 35. Upon receiving the alert signal, the terminal may display text, an icon, or the like indicating that facial authentication was not possible on the touch panel display. This makes it possible to notify teachers and persons whose faces could not be matched that their faces could not be matched, thereby improving the convenience of the admission management system 1.

[0151] <Third embodiment> The admission management system 1 of the third embodiment differs from the first and second embodiments in that the facial recognition server 12 also stores the course information of each student. The other configurations are the same as those of the first embodiment (or the second embodiment), so a description of the other similar configurations will be omitted.

[0152] The course information may be recorded, for example, in a face information database 35. Fig. 24 shows an example of the face information database 35 according to the third embodiment. In the face information database 35, information about the classroom S in which the lecture that each student is attending is held, and the corresponding day of the week and time period are recorded as group setting information.

[0153] 25 shows a time chart of the processing during course registration up to the point where group settings are recorded in the face information database 35. When the user inputs course information into the course information input screen displayed on the user terminal 6 and performs the transmission process, the user terminal 6 transmits the course information to the management server 5 (S800). The course information transmitted from the user terminal 6 may be equivalent to the data recorded in the course information table 87A, for example.

[0154] When the course information is sent from the user terminal 6, the management server 5 registers the course information in the course information table 87A (S801). Once registration is complete, the management server 5 generates a group setting for each student based on the course information table 87A and the subject information table 85A, which includes information on the classroom S in which the lecture being attended is held and the corresponding day of the week and time period. The group setting information may be, for example, a list of the day of the week, time period, and classroom S in which the lecture being attended is held, as shown in FIG. 24. Once generation of the group setting information for all students is complete, the management server 5 associates the IDs (student ID numbers) with the corresponding group setting information and transmits them to the face authentication server 12 (S802).

[0155] When the management server 5 receives the group setting information, it associates the group setting information with the ID and registers it in the face information database 35 (S803).

[0156] The group setting information is used by the face authentication server 12 when performing face authentication on the image captured by the entrance camera 10. When performing face authentication on the image captured by the entrance camera 10, the participants of the lecture held in the classroom S where the entrance camera 10 is installed are extracted based on the group setting. The face authentication server 12 then uses the extracted face information of the participants to perform face authentication on the facial parts detected in the image captured by the entrance camera 10. This reduces the amount of face information that the face authentication server 12 must compare when performing face authentication on the image captured by the entrance camera 10, thereby reducing the load on the face authentication server 12 for face authentication.

[0157] The facial recognition server 12 may construct the matching database 51 by extracting students who will attend a lecture for which attendance is to be confirmed based on group setting information and generating groups. In this case, the facial recognition server 12 transmits the constructed matching database 51 to the edge server 13, and the edge server 13 performs face recognition using the matching database 51. This allows the matching database 51 to be generated by a simple method without relying on images captured by the entrance camera 10.

[0158] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. Furthermore, it is also possible to combine the components described in the above embodiments to create new embodiments.

[0159] In the above embodiment, matching based on facial information is performed in each of the face authentication server 12 and the edge server 13, but different thresholds for setting face authentication (face authentication thresholds) that are deemed to be successful when exceeded may be set. In this case, the face authentication threshold in the face authentication server 12 may be configured to be set higher than the face authentication threshold in the edge server 13. This allows face authentication to be performed under stricter conditions at the time of entry to confirm entry, and the level of face authentication after entry can be lowered, thereby reducing the load on face authentication in the edge server 13.

[0160] For example, instead of the edge server 13, the entrance management system 1 may include a cloud server located outside the school that performs the same functions as the edge server 13. The cloud server is preferably connected to an off-campus communication network 16 and configured to be connectable to the entrance camera 10 and classroom camera 11 via a router 17.

[0161] In the above embodiment, the face authentication server 12 and the edge server 13 are configured as separate entities in the entrance management system 1, but this is not limited to this. FIG. 26 shows a system configuration diagram in which the face authentication server 12 and the edge server 13 are integrated. As shown in FIG. 26, the entrance management system 1 may include, instead of the face authentication server 12 and the edge server 13, a single server (hereinafter, referred to as authentication server 100) that executes an entrance management program to perform the processes performed by the face authentication server 12 and the edge server 13. In this case, the authentication server 100 may be a cloud server located outside the school. In this case, the face information database 35 and the matching database 51 are managed by a single authentication server 100. Therefore, it is preferable to create a dedicated tenant for the matching database 51 and manage the matching database 51 in the dedicated tenant.

[0162] In the above embodiment, an example has been described in which the admission management system 1 is applied to a system for managing admission to classrooms for attendance confirmation in an educational institution, but it may also be applied to a system for managing admission to a theater, a theatre, etc. In that case, an entrance camera 10 is installed at the location where tickets are checked, and one or more area cameras equivalent to the classroom cameras 11 are installed inside the theater or theatre. The area camera may be an infrared camera (IR camera). The area camera may be installed in a position where it can capture images of the passageway leading from the entrance to the theater or the interior of the theatre. Alternatively, the area camera may be installed in a lobby or foyer.

[0163] When applied to a system for confirming admission to a theater or a cinema, once face authentication is complete, the face authentication server 12 or the edge server 13 may send a notification to the user terminal 6 held by the face-authenticated audience member by email or the like. At that time, if the audience member is seated in a location different from the corresponding designated seat, the user terminal 6 may notify the audience member that they are seated in a different location. Furthermore, depending on the congestion level in the theater or cinema, the user terminal 6 may notify the audience member of the route to where they should be seated.

[0164] However, the admission confirmation systems to which the admission management system 1 can be applied are not limited to admission confirmation systems for theaters and other venues, and the system may also be applied to admission confirmation systems for stadiums.The admission management system 1 may also be applied to admission confirmation systems for music festivals and other venues where seats are not assigned.

[0165] Additionally, the entrance management system 1 may be applied to an office attendance confirmation system. In that case, the entrance camera 10 is preferably installed near the entrance to the room, and the area camera is preferably installed inside the room. If face authentication is performed multiple times using images captured by the area camera and presence within the area (room) is confirmed multiple times, the face-authenticated employee is determined to have clocked in, and attendance processing is performed. [Industrial Applicability]

[0166] The entrance management method, entrance management system 1, and entrance management device disclosed herein have the effect of reducing the load required for facial authentication, and are useful as an entrance management method, entrance management system, and entrance management device that manage the entry of people into a specified area by authentication using registered facial information. [Explanation of symbols]

[0167] 1: Admission control system 2: School Management System 3: Attendance management system 5: Management Server 5A: Processor 5B: Memory 5C: Communication interface 5D: Storage 6: User terminal 10: Entrance camera 11: Classroom camera 12: Face recognition server 12A: Processor 12B: Memory 12C: Communication interface 12D: Storage 13: Edge server 13A: Processor 13B: Memory 13C: Communication interface 13D: Storage 15: Communication Network 16: Communication Network 17: Router 21:Student information storage department 22:Student Information Management Department 23: Attendance information storage unit 24: Attendance Information Management Department 25: Personal information database 25A: Personal information table 26: Attendance information database 26A: Attendance information table 31: Face information memory unit 32: Face detection unit 33: Feature acquisition unit 34: Entrance processing unit 35: Facial information database (registered database) 35A: Face information table 41: Matching data storage unit 42: Matching data management unit 43: Face detection unit 44: Feature acquisition unit 45: Face matching unit 46: Matching result storage unit 47: Matching result update section 48: Verification data update section 49: Attendance Judgment Department 51: Matching database (entrance candidate database) 53: Matching result database 53A: Matching result table 55: Entrance control device 71: Administrator terminal 81: Lecture Information Management Department 82: Course Information Management Department 83: Management display generation section 85: Lecture information database 85A: Course information table 87: Course registration information database 87A: Course information table 100: Authentication server (entrance management device) S: Classroom (designated area)

Claims

1. An entrance management method for managing the entrance of a person to a predetermined area by authentication using registered face information, comprising: an entrance camera that captures a facial image of a person near the entrance of the predetermined area; an area camera for capturing facial images of people present within the predetermined area; an entrance management device that maintains a registered database in which registered face information is recorded, and performs face authentication using face information based on face images acquired by the entrance camera and the area camera, The entrance management device a first matching step of performing a first matching of matching first face information based on a face image acquired by the entrance camera with registered face information recorded in the registered face database, and determining whether or not there is registered face information that is determined to be the same person as the first face information; a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the predetermined area when it is determined that the person exists in the first comparison; a second matching step of performing a second matching in which face information based on the face image acquired by the area camera is matched with face information registered in the entry candidate database; a second registration step of recording the result of the second matching in a matching result database; an entrance management method that executes a transmission step of transmitting information identifying a person present within the specified area to an external location based on at least one of the entrance candidate database and the matching result database.

2. The entrance management method described in claim 1, wherein, when the second matching is not possible in the second matching step, the entrance management device performs a third matching in which the image captured by the area camera is compared with the registered database, and when the third matching is possible, the result is recorded in the matching result database.

3. The entrance management method according to claim 2, wherein the entrance management device registers the facial information corresponding to the captured image acquired by the area camera in the entrance candidate database when the third matching is successful in the second matching step.

4. The entrance management method according to claim 2, wherein the entrance management device outputs an alert to notify the corresponding person or administrator when the third matching is not possible in the second matching step.

5. 2. The entrance management method according to claim 1, wherein the entrance candidate database is configured as a group of the face information included in the registered database.

6. The entrance management method according to claim 1 , wherein the entrance management device comprises a face authentication server that performs the first matching and an edge server that performs the second matching.

7. An entrance management system that manages the entrance of people to a predetermined area by authentication using registered face information, an entrance camera that captures a facial image of a person near the entrance of the predetermined area; an area camera for capturing facial images of people present within the predetermined area; an entrance management device that holds a registered database in which registered face information is recorded, and performs face authentication using face information based on face images acquired by the entrance camera and the area camera; The entrance management device a first matching step of performing a first matching of matching first face information based on a face image acquired by the entrance camera with registered face information recorded in the registered face database, and determining whether or not there is registered face information that is determined to be the same person as the first face information; a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the predetermined area when it is determined that the person exists in the first comparison; a second matching step of matching face information based on the face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in a matching result database; An entrance management system that executes a transmission step of transmitting information identifying persons present within the specified area to an external location based on at least one of the entrance candidate database and the matching result database.

8. An entrance management device that manages the entrance of a person to a predetermined area by authentication using registered face information, an entrance camera that captures a facial image of a person near the entrance of the predetermined area; an area camera that captures facial images of people present within the predetermined area; maintain a registered database in which registered face information is recorded; a first matching step of performing a first matching of matching first face information based on a face image acquired by the entrance camera with registered face information recorded in the registered face database, and determining whether or not there is registered face information that is determined to be the same person as the first face information; a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the predetermined area when it is determined that the person exists in the first comparison; a second matching step of matching face information based on the face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in a matching result database; an entrance management device that executes a transmission step of transmitting information identifying a person present within the specified area to an external device based on at least one of the entrance candidate database and the matching result database.

9. An entrance management program for managing the entrance of a person to a predetermined area by authentication using registered face information, an entrance camera that captures a facial image of a person near the entrance of the predetermined area; an area camera for capturing facial images of people present within the predetermined area; an entrance management device that maintains a registered database in which registered face information is recorded, and performs face authentication using face information based on face images acquired by the entrance camera and the area camera; The entrance management device a first matching step of performing a first matching of matching first face information based on a face image acquired by the entrance camera with registered face information recorded in the registered face database, and determining whether or not there is registered face information that is determined to be the same person as the first face information; a first registration step of registering the first face information or the registered face information in an entry candidate database as face information of a person who may have entered the predetermined area when it is determined that the person exists in the first comparison; a second matching step of matching face information based on the face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in a matching result database; an entrance management program that executes a transmission step of transmitting to the outside information that identifies a person present within the specified area based on at least one of the entrance candidate database and the matching result database.

Images