Admission control method, admission control system, admission control device, and admission control program

Abstract

To reduce a load required for face authentication in an entrance management method, an entrance management system, an entrance management device, and an entrance management program.SOLUTION: An entrance management method is implemented by an entrance management system 1 including an entrance camera 10 that captures an image near an entrance of an area S, an area camera 11 that captures an image of a person in the area S, and an entrance management device 55 that holds a registered database 35 and performs face authentication, and the entrance management device 55 is configured to execute: a step of executing first collation of collating a face image of the entrance camera with the registered database and registering the face image in an entrance candidate database 51; a step of collating the face image acquired by the area camera with the entrance candidate database and registering the face image in a collation result database 53; and a transmission step of transmitting information for specifying a person in the area based on at least one of the entrance candidate database and the collation result database.SELECTED DRAWING: Figure 17

Description

Technical Field

[0001] The present disclosure relates to an admission management method, an admission management system, an admission management device, and an admission management program.

Background Art

[0002] Conventionally, in educational institutions, in order to manage the attendance of students, at the beginning of a class, the teacher calls the name of each student, and by the student answering, the teacher confirms that the student is within a predetermined area such as a classroom, thereby performing attendance confirmation. However, there are cases where a student leaves the area during the class.

[0003] Therefore, for example, there is known an attendance status management system including a camera installed in a classroom divided into a plurality of areas and an information processing device communicably connected to the camera. The camera sequentially photographs each of the plurality of areas, detects the faces of students attending a class in the area, and executes a plurality of times during the class a sequence process that is a process of photographing a round of the plurality of areas. The information processing device includes a receiving unit that receives face image data of students from the camera a plurality of times during the class, and a control unit that collates the face image data with registered face image data of the students and counts the number of times the students with the registered face image data are photographed by the camera during the class (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] [[ID=**38**]] By the way, when using the prior art described in Patent Document 1 above to manage attendance in multiple classrooms held at the same time, it is necessary to compare the facial image data of all students attending the class at that time with the images captured by the camera. Therefore, as the number of students attending the class at the same time increases, the load required for facial recognition increases, and there is a risk that the information processing device will not be able to process it all.

[0006] Therefore, the main purpose of this disclosure is to reduce the load required for facial recognition in an entry management method, entry management system, entry management device, and entry management program that uses facial recognition for entry management. [Means for solving the problem]

[0007] The admission management method disclosed herein is an admission management method that manages the entry of persons into a predetermined area by authentication using registered facial information, and is implemented by an admission management system comprising: an entrance camera that acquires facial images of persons near the entrance of the predetermined area; an area camera that acquires facial images of persons present within the predetermined area; and an admission management device that maintains a registered database in which registered facial information is recorded and performs facial authentication using facial information based on facial images acquired by the entrance camera and the area camera, wherein the admission management device performs a first comparison by comparing first facial information based on facial images acquired by the entrance camera with registered facial information recorded in the registered database, and the registered facial information determined to be the same person as the first facial information The system is configured to perform the following steps: a first matching step of determining whether or not existing facial information exists; a first registration step of registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area, if it is determined in the first matching that such information exists; a second matching step of performing a second matching between facial information based on facial images acquired by the area camera and facial information registered in the entry candidate database; a second registration step of recording the result of the second matching in the matching result database; and a transmission step of transmitting information to an external source that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0008] The admission management system disclosed herein is an admission management system that manages the entry of persons into a predetermined area by authentication using registered facial information, comprising: an entrance camera that acquires facial images of persons near the entrance of the predetermined area; an area camera that acquires facial images of persons present within the predetermined area; and an admission management device that maintains a registered database in which registered facial information is recorded, and performs facial authentication using facial information based on facial images acquired by the entrance camera and the area camera, wherein the admission management device performs a first comparison by comparing first facial information based on facial images acquired by the entrance camera with registered facial information recorded in the registered database, and the registered facial information that is determined to be the same person as the first facial information The system is configured to perform the following steps: a first matching step of determining whether or not a person exists; a first registration step of registering the first face information or the registered face information in the entry candidate database as face information of a person who may have entered the predetermined area, if it is determined in the first matching step that a person exists; a second matching step of comparing face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting information to an external source that identifies a person who is present in the predetermined area, based on at least one of the entry candidate database and the matching result database.

[0009] The access control device of this disclosure is an access control device that manages the entry of persons into a predetermined area by authentication using registered facial information, and is connected to an entrance camera that acquires facial images of persons near the entrance of the predetermined area, and an area camera that acquires facial images of persons present within the predetermined area, and holds a registered database in which registered facial information is recorded, and performs a first matching step which compares first facial information based on the facial image acquired by the entrance camera with the registered facial information recorded in the registered database, and determines whether or not there is registered facial information that is determined to be the same person as the first facial information, and If it is determined that a person may have entered the predetermined area, the system is configured to perform the following steps: a first registration step of registering the first face information or the registered face information in the entry candidate database as face information of a person who may have entered the predetermined area; a second matching step of comparing face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting information to an external source that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0010] The admission management program of this disclosure is an admission management program that manages the entry of persons into a predetermined area by authentication using registered facial information, and is executed by an admission management system comprising: an entrance camera that acquires facial images of persons near the entrance of the predetermined area; an area camera that acquires facial images of persons present within the predetermined area; and an admission management device that maintains a registered database in which registered facial information is recorded and performs facial authentication using facial information based on facial images acquired by the entrance camera and the area camera, wherein the admission management device performs a first comparison that compares first facial information based on facial images acquired by the entrance camera with registered facial information recorded in the registered database, and determines that the first facial information is the same person. The system is configured to perform the following steps: a first matching step of determining whether or not the registered facial information exists; a first registration step of registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area, if it is determined in the first matching step that such information exists; a second matching step of comparing facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting information to an external source that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database. [Effects of the Invention]

[0011] According to this disclosure, the load required for facial recognition can be reduced in admission management methods, admission management systems, admission management devices, and admission management programs. [Brief explanation of the drawing]

[0012] [Figure 1] System configuration diagram of the admission management system according to the first embodiment [Figure 2] Schematic diagram of a classroom where attendance is checked by the admission management system according to the first embodiment. [Figure 3]Block diagram of the management server [Figure 4] Functional block diagram of the management server [Figure 5] Example of the personal information table [Figure 6] Flowchart showing the flow until a face image is registered by the management server [Figure 7] Example of the attendance information table [Figure 8] Block diagram of the face authentication server [Figure 9] Functional block diagram of the face authentication server [Figure 10] Example of the face information table [Figure 11] Block diagram of the edge server [Figure 12] Functional block diagram of the edge server [Figure 13] Example of the collation database [Figure 14] Example of the collation result table [Figure 15] Flowchart showing the process of entry processing [Figure 16] Flowchart showing the process of attendance confirmation processing [Figure 17] Time chart showing the process flow of the entrance management system when face authentication is performed upon entering the classroom [Figure 18] Time chart showing the process flow of the entrance management system when face authentication is performed upon entering the classroom and the process flow after the start of the class [Figure 19] ]>System configuration diagram of the entrance management system according to the second embodiment [Figure 20] Functional block diagram of the management server according to the second embodiment [Figure 21] ]>Example of the subject information table [Figure 22] Example of the course enrollment information table [Figure 23] Time chart showing the process flow of the entrance management system according to the second embodiment when face authentication is performed upon entering the classroom [Figure 24] Example of the face information table of the entrance management system according to the third embodiment [Figure 25]A timeline of the process from course registration to the recording of group settings in the facial information database. [Figure 26] A system configuration diagram showing a modified version of the access control system when the facial recognition server and edge server are integrated. [Modes for carrying out the invention]

[0013] The first invention made to solve the aforementioned problem is an entry management method for managing the entry of a person into a predetermined area by authentication using registered facial information, and is implemented by an entry management system comprising: an entrance camera that acquires a person's facial image near the entrance of the predetermined area; an area camera that acquires a person's facial image within the predetermined area; and an entry management device that maintains a registered database in which registered facial information is recorded and performs facial authentication using facial information based on the facial images acquired by the entrance camera and the area camera, wherein the entry management device performs a first comparison by comparing the first facial information based on the facial image acquired by the entrance camera with the registered facial information recorded in the registered database, and determines that the person is the same as the first facial information. The system is configured to perform the following steps: a first matching step of determining whether or not the registered facial information exists; a first registration step of registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area, if it is determined in the first matching that such information exists; a second matching step of performing a second matching of facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database; a second registration step of recording the result of the second matching in the matching result database; and a transmission step of transmitting to an external party information that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0014] According to this, the access control system performs facial recognition (second matching) by comparing images captured by area cameras with an access candidate database. Since the facial information of individuals who have been successfully matched in the first matching is registered in the access candidate database, the number of facial information entries in the access candidate database is less than or equal to the number of facial information entries in the registered database. Therefore, compared to matching images captured by area cameras with a registered database, the number of facial information entries to be matched can be reduced, thereby reducing the load required for facial recognition.

[0015] Furthermore, the second invention is that, in the first invention described above, if the second verification is not possible in the second verification step, the entry management device performs a third verification by comparing the captured image acquired by the area camera with the registered database, and if the third verification is successful, records it in the verification result database.

[0016] According to this, even if a person cannot be properly photographed by the entrance camera at the time of entry and therefore cannot be registered in the entry candidate database, facial recognition can still be performed by matching them with the registered database.

[0017] Furthermore, the third invention is configured such that, in the second invention described above, the entry management device registers the face information corresponding to the image captured by the area camera when the third matching is achieved in the entry candidate database.

[0018] According to this, even if the second matching step fails, if the third matching is successful, the facial information can be registered in the candidate database.

[0019] Furthermore, the fourth invention is that, in the second invention described above, the access control device is configured to output an alert to notify the corresponding person or manager if the third verification fails in the second verification step.

[0020] According to this, if a person not registered in the registered database is captured by an area camera, a notification can be issued.

[0021] Furthermore, the fifth invention is configured such that, in the first invention described above, the candidate database for admission is configured as a group of facial information included in the registered database.

[0022] Furthermore, the sixth invention provides an access control device comprising a facial recognition server that performs the first verification and an edge server that performs the second verification.

[0023] According to this, the load related to matching can be distributed.

[0024] The seventh invention, made to solve the aforementioned problems, is an entry management system for managing the entry of persons into a predetermined area by authentication using registered facial information, comprising: an entrance camera that acquires a person's facial image near the entrance of the predetermined area; an area camera that acquires a person's facial image within the predetermined area; and an entry management device that maintains a registered database in which registered facial information is recorded, and performs facial authentication using facial information based on the facial images acquired by the entrance camera and the area camera, wherein the entry management device performs a first comparison by comparing first facial information based on the facial image acquired by the entrance camera with registered facial information recorded in the registered database, and the registered facial information is determined to be the same person as the first facial information The system is configured to perform the following steps: a first matching step of determining whether or not facial information exists; a first registration step of registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area, if it is determined in the first matching step that facial information based on a facial image acquired by the area camera is compared with the facial information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting to an external source information that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0025] According to this, the access control system performs facial recognition (second matching) by comparing images captured by area cameras with an access candidate database. Since the facial information of individuals who have been successfully matched in the first matching is registered in the access candidate database, the number of facial information entries in the access candidate database is less than or equal to the number of facial information entries in the registered database. Therefore, compared to matching images captured by area cameras with a registered database, the number of facial information entries to be matched can be reduced, thereby reducing the load required for facial recognition.

[0026] The eighth invention, made to solve the aforementioned problems, is an access control device that manages the entry of a person into a predetermined area by authentication using registered facial information, and is connected to an entrance camera that acquires a person's facial image near the entrance of the predetermined area, and an area camera that acquires a person's facial image within the predetermined area, and holds a registered database in which registered facial information is recorded, and performs a first comparison that compares first facial information based on the facial image acquired by the entrance camera with the registered facial information recorded in the registered database, and determines whether or not there is registered facial information that is determined to be the same person as the first facial information, and the first comparison If it is determined that a person exists in the area, the system is configured to perform the following steps: a first registration step of registering the first face information or the registered face information in the entry candidate database as face information of a person who may have entered the predetermined area; a second matching step of comparing face information based on a face image acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting information to an external source that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0027] According to this, the access control system performs facial recognition by comparing images captured by area cameras with a database of potential entrants. Since the facial information of individuals who have been successfully matched in the first round is registered in the database of potential entrants, the number of facial information entries in the potential entrant database is less than or equal to the number of facial information entries in the registered database. Therefore, compared to matching images captured by area cameras with the registered database, the number of facial information entries to be matched can be reduced, thereby reducing the load required for facial recognition.

[0028] The ninth invention, made to solve the aforementioned problems, is an entry management program that manages the entry of a person into a predetermined area by authentication using registered facial information, and is executed by an entry management system comprising: an entrance camera that acquires a person's facial image near the entrance of the predetermined area; an area camera that acquires a person's facial image within the predetermined area; and an entry management device that maintains a registered database in which registered facial information is recorded and performs facial authentication using facial information based on the facial images acquired by the entrance camera and the area camera, wherein the entry management device performs a first comparison that compares first facial information based on the facial image acquired by the entrance camera with registered facial information recorded in the registered database and determines that the first facial information is the same person The system is configured to perform the following steps: a first matching step of determining whether or not the specified registered face information exists; a first registration step of registering the first face information or the registered face information in the entry candidate database as face information of a person who may have entered the predetermined area, if it is determined in the first matching step that such information exists; a second matching step of comparing face information based on face images acquired by the area camera with face information registered in the entry candidate database; a second registration step of recording the result of the second matching step in the matching result database; and a transmission step of transmitting information to an external source that identifies a person present in the predetermined area based on at least one of the entry candidate database and the matching result database.

[0029] According to this, the access control system performs facial recognition (second matching) by comparing images captured by area cameras with an access candidate database. Since the facial information of individuals who have been successfully matched in the first matching is registered in the access candidate database, the number of facial information entries in the access candidate database is less than or equal to the number of facial information entries in the registered database. Therefore, compared to matching images captured by area cameras with a registered database, the number of facial information entries to be matched can be reduced, thereby reducing the load required for facial recognition.

[0030] The embodiments of this disclosure will be described below with reference to the drawings.

[0031] The admission management method, admission management system, admission management device, and admission management program disclosed herein are used for facial recognition of individuals present inside a designated area. Specifically, the admission management method, admission management system, admission management device, and admission management program disclosed herein can be used, for example, in systems for checking attendance in educational institutions (hereinafter referred to as attendance confirmation systems), or in admission management systems for theaters, concert venues, etc.

[0032] The following describes and illustrates embodiments in which the admission management method, admission management system, admission management device, and admission management program of this disclosure are applied to classroom admission management for attendance confirmation in educational institutions.

[0033] <First Embodiment> The educational institution to which the admission management system 1 according to the first embodiment is applied is a university, where lectures are held in multiple classrooms, and students attending the lectures enter (go into) the classroom S. Here, the number of students in the educational institution is, for example, on the scale of several thousand, and the number of students attending the lectures is, for example, several tens. Figure 1 shows a top view of classroom S where attendance status is checked by the admission management system 1.

[0034] Students are assumed to have registered their facial information in advance, and the admission management system 1 described herein can be used to perform facial recognition of individuals inside classroom S using the pre-registered facial information and to confirm their attendance. The facial information referred to here may be one or more facial images, or it may be facial features. In addition, the facial information may be any biometric information that allows for identity verification using the face.

[0035] Figure 2 is a diagram showing the configuration of the access control system 1 according to the first embodiment.

[0036] The admission management system 1 includes a school administration system 2 that provides students and faculty with lecture information, attendance information, etc., and an attendance management system 3 for checking student attendance.

[0037] The school administration system 2 may include a management server 5 that stores various information related to students, including attendance information, and user terminals 6 that students possess.

[0038] The attendance management system 3 may include an entrance camera 10 (see also Figure 2) installed at the entrance of classroom S, a classroom camera 11 (area camera, see also Figure 2) that captures images inside classroom S, a face recognition server 12 (authentication server) that stores face information for face recognition of each student, and an edge server 13 that performs face recognition using images captured by the classroom camera 11 and determines the attendance of each student.

[0039] The entrance camera 10, classroom cameras 11, edge server 13, and management server 5 are operated by the educational institution. As shown in Figure 1, the entrance camera 10, classroom cameras 11, edge server 13, and management server 5 are each connected to the school's internal communication network 15 (local network) installed within the educational institution.

[0040] The user terminal 6 and the facial recognition server 12 are each connected to an external communication network 16, such as the internet. The internal communication network 15 and the external communication network 16 are connected to each other via a router 17.

[0041] Next, we will describe the details of the user terminals 6 and the management server 5 that may be included in the school administration system 2.

[0042] Each student, acting as a user, possesses a user terminal 6. A user terminal 6 may consist of, for example, a smartphone equipped with a processor (CPU, MPU, etc.), memory such as volatile memory (RAM) or non-volatile memory (ROM), a touch panel display, a camera, a communication interface, and storage such as eMMC.

[0043] The user terminal 6 displays various information on its touch panel display and also accepts input of various information on the touch panel display.

[0044] The management server 5 manages students' personal information and attendance information. As shown in Figure 3, the management server 5 may consist of a computer equipped with a processor 5A (CPU, MPU, etc.), memory 5B such as volatile memory (RAM) or non-volatile memory (ROM), a communication interface 5C, and storage 5D such as an HDD or SSD.

[0045] As shown in Figure 4, the management server 5 comprises a student information storage unit 21, a student information management unit 22, an attendance information storage unit 23, and an attendance information management unit 24 as functional units.

[0046] The student information storage unit 21 may be composed of storage 5D. The student information storage unit 21 stores the personal information of students. In this embodiment, the student information storage unit 21 holds a personal information database 25 in which the personal information of students is recorded. The personal information database 25 may include a personal information table 25A in which the ID (student ID number), name, and contact information such as email address assigned to each student are recorded as associated personal information. Figure 5 shows an example of the personal information table 25A. In addition, the personal information table 25A may also include each student's facial image, which is recorded in association with the student's ID.

[0047] The student information management unit 22 may be configured by having the processor 5A perform processing according to a predetermined program. The student information management unit 22 acquires personal information from the user terminal 6 and stores it in the student information storage unit 21. The student information management unit 22 also updates the personal information stored in the student information storage unit 21 as appropriate. The personal information acquired by the student information management unit 22 may include facial images. The facial images acquired by the student information management unit 22 may be acquired by the camera of the user terminal 6, or they may be acquired by another imaging device and stored in the user terminal 6.

[0048] Figure 6 shows an example of a time chart for when the Student Information Management Department 22 acquires facial information from the user terminal 6. In the example shown in Figure 6, it is assumed that the student's personal information, including their email address, is already recorded in the personal information database 25.

[0049] In the example shown in Figure 6, the student information management department 22 first refers to the personal information database 25 to obtain each student's email address, and then sends an instruction to the facial recognition server 12 to obtain facial information along with the information related to the obtained email address (facial information acquisition instruction) (S100).

[0050] When the facial recognition server 12 receives a facial information acquisition instruction, it obtains each student's email address and sends an email requesting facial information registration (hereinafter referred to as a facial registration request email) to each email address (S101). The facial registration request email may include the student's ID corresponding to each email address and a link to display the facial information registration screen.

[0051] In the example shown in Figure 6, the Student Information Management Department 22 was configured to send a facial information acquisition instruction to the facial recognition server 12, which then sent an email. However, the Student Information Management Department 22 may also be configured to send a facial registration request email to the user terminal 6.

[0052] When a student uses user terminal 6 to obtain a face registration request email and opens the link provided in the email, user terminal 6 displays a face image registration screen (S102). The face image registration screen may be generated by either the management server 5 or the face authentication server 12. The face image registration screen may include text indicating the student ID corresponding to user terminal 6, input fields for entering information about the file to be registered (file name, etc.), and a send button. The face image registration screen may also include text instructing the user to take a face image using user terminal 6. When the file name of the face image is entered and the send button is touched, user terminal 6 sends the corresponding student ID and face image to the face authentication server 12 (S103).

[0053] When the facial recognition server 12 receives the corresponding student's ID and facial image from the user terminal 6, it associates the ID with the facial image and / or the corresponding facial information and stores (registers) them (S104). Subsequently, the facial recognition server 12 associates the ID and facial image obtained from the user terminal 6 and sends it to the management server 5 (S105), and the management server 5 (student information management department 22) associates the corresponding student's ID with the facial image and records it in the personal information database 25 (S106).

[0054] Furthermore, when a file name indicating a face image is entered on the face image registration screen and the send button is touched, the user terminal 6 may be configured to send the corresponding student ID and face image to the management server 5. In that case, the management server 5 should register the acquired face image in the personal information database 25, and also be configured to send the acquired face image and the corresponding ID to the face authentication server 12, and have the face authentication server 12 register the face information.

[0055] The attendance information storage unit 23 may be composed of storage 5D. The attendance information storage unit 23 holds an attendance information database 26 in which attendance information for each student's lectures is recorded.

[0056] The attendance information database 26 may include an attendance information table 26A in which student ID numbers are associated with and stored attendance information for the corresponding students, as shown in Figure 7. The attendance information indicates whether or not the corresponding student attended the lectures (also called classes) they were enrolled in for each period on each date. In the example in Figure 7, a circle (○) recorded in the attendance information table 26A indicates that attendance was confirmed, and a cross (×) indicates that attendance was not confirmed (i.e., the student was absent or did not have any lectures to enroll in).

[0057] The attendance information management unit 24 may be configured by having the processor 5A perform processing according to a predetermined program. After each lecture, the attendance information management unit 24 obtains the attendance judgment result from the edge server 13 and writes the obtained attendance judgment result to the attendance information database 26.

[0058] Next, we will describe the details of the entrance camera 10, classroom camera 11, facial recognition server 12, and edge server 13, which may be included in the attendance management system 3.

[0059] As shown in Figure 1, the entrance camera 10 acquires facial images of people near the entrance of classroom S (a designated area). More specifically, the entrance camera 10 is installed near the entrance of classroom S and captures images of people entering classroom S by passing through that entrance. The imaging range of the entrance camera 10 includes the face of the person entering classroom S. As shown in Figure 2, the entrance camera 10 is connected to the school's internal communication network 15 installed within the educational institution and transmits the acquired images to the edge server 13 via the communication network 15.

[0060] As shown in Figure 1, the classroom camera 11 is installed in classroom S and acquires facial images of people present in classroom S. More specifically, the classroom camera 11 captures images of students attending lectures in classroom S. The imaging range of the classroom camera 11 includes the faces of students attending lectures. It is preferable to install multiple classroom cameras 11 in classroom S so that the faces of all students attending lectures in classroom S can be captured. As shown in Figure 2, the classroom camera 11 is connected to the school's communication network 15 and transmits the acquired images to the edge server 13 via the communication network 15.

[0061] As shown in Figure 8, the facial recognition server 12 may consist of a computer equipped with a processor 12A (CPU, MPU, etc.), memory 12B such as volatile memory (RAM) or non-volatile memory (ROM), a communication interface 12C, and storage 12D such as an HDD or SSD. The facial recognition server 12 is connected to an external communication network 16 and corresponds to a cloud server that performs various processes related to facial recognition. Specifically, the facial recognition server 12 holds facial information for facial recognition, provides facial information to the edge server 13, and performs processes such as face detection and facial recognition from images acquired by the entrance camera 10.

[0062] As shown in Figure 9, the facial recognition server 12 includes, as functional units, a facial information storage unit 31, a facial detection unit 32, a feature acquisition unit 33, and an entry processing unit 34.

[0063] The face information storage unit 31 may be comprised of storage 12D. The face information storage unit 31 holds a face information database 35 in which IDs and face information are recorded in association. The face information database 35 may include a face information table 35A in which the face information storage unit 31 records the face information of all students entered from the user terminal 6 in response to a face registration request email, along with the student's ID.

[0064] Figure 10 shows an example of a face information table 35A. The face information table 35A stores IDs, face information, and the names of students corresponding to the IDs, in association with each other. As shown in Figure 10, the face information may include filenames for face images and filenames for data showing face features.

[0065] Each student is assigned an ID (student ID number) upon enrollment, and their facial information is registered in the facial information database 35. Therefore, when attendance management is performed by the admission management system 1, the facial information database 35 becomes a registered database containing the registered facial information.

[0066] The face detection unit 32 may be configured by the processor 12A executing a predetermined program. The face detection unit 32 detects the face portion of a person from the image captured by the entrance camera 10 using a known method. When a student enters classroom S, the face of the person entering classroom S is included in the imaging range of the entrance camera 10, so the face portion of the student entering classroom S is captured, and the face portion of the captured image is detected by the face detection unit 32.

[0067] The feature acquisition unit 33 can be configured by the processor 12A executing a predetermined program. The feature acquisition unit 33 acquires (also called generating) face features using the face portion detected by the face detection unit 32. Face features are numerical representations of information useful for identifying a person, such as the shape and positional relationship of eyes, nose, etc., or numerical data generated from a face image using a trained model of a face feature generation AI based on known deep learning technology. By comparing the face features with previously acquired face information, it is possible to determine whether or not they are the same person.

[0068] The entrance processing unit 34 may be configured by having the processor 12A execute a predetermined program. The entrance processing unit 34 uses the feature quantities acquired by the feature quantity acquisition unit 33 and the face information database 35 to perform face matching of persons captured by the entrance camera 10. If it is possible to extract (match) face information corresponding to the person captured by the entrance camera 10 from the face information database 35, the person's ID (student ID number) and face information are sent to the edge server 13. As a result, the ID of the student who entered the classroom S equipped with the entrance camera 10 from the entrance, and the corresponding face information are sent to the edge server 13. In this embodiment, the entrance processing unit 34 may also send the name of the person to the edge server 13 in addition to the ID and face information of the person that was matched.

[0069] As shown in Figure 11, the edge server 13 may consist of a computer equipped with a processor 13A (CPU, MPU, etc.), memory 13B such as volatile memory (RAM) or non-volatile memory (ROM), a communication interface 13C, and storage 13D. The edge server 13 is installed in each classroom S, performs facial recognition using images captured by the classroom camera 11, determines attendance, and transmits the results to the management server 5. Note that instead of installing an edge server 13 in each classroom S, it may be installed at any location within the building where the classroom S is located.

[0070] As shown in Figure 12, the edge server 13 includes, as a functional unit, a matching data storage unit 41, a matching data management unit 42, a face detection unit 43, a feature acquisition unit 44, a face matching unit 45, a matching result storage unit 46, a matching result update unit 47, a matching data update unit 48, and an attendance determination unit 49.

[0071] The matching data storage unit 41 may be composed of storage 13D. The matching data storage unit 41 holds a matching database 51 for performing facial recognition of students captured in classroom S using the classroom camera 11. As shown in Figure 13, the matching database 51 records the ID (student ID number) of a person (student) whose face has been recognized using the image captured by the entrance camera 10, and the corresponding facial information, in association with each other. In this embodiment, in addition to the ID and the corresponding facial information, the matching database 51 also records the name of the corresponding student.

[0072] The matching data management unit 42 may be configured by the processor 13A executing a predetermined program. The matching data management unit 42 resets the matching database 51 as needed after the end of a lecture that was held before the lecture for which attendance confirmation was to be performed, and before the ID and face information are transmitted from the entrance processing unit 34. Subsequently, the matching data management unit 42 acquires the ID and face information transmitted from the entrance processing unit 34 via the communication interface 13C and writes them to the matching database 51 held by the matching data storage unit 41. As a result, when a student enters classroom S from the entrance and is facially authenticated using the image captured by the entrance camera 10, the student's ID and corresponding face information are recorded in the matching database 51. In other words, the matching database 51 is a database (entry candidate database) that records the face information of people who are thought to have entered classroom S (area). The matching data management unit 42 may be configured to discard the data recorded in the matching database 51 and reset the matching database 51 when the lecture ends and attendance determination is completed.

[0073] Furthermore, the facial information of individuals who have entered, as written to the matching database 51 by the matching data management unit 42, is not limited to the facial information registered in the facial information database 35. For example, the facial information of individuals who have entered, as written to the matching database 51, may be, for example, facial information corresponding to images captured by the entrance camera 10 (such as captured images, facial images obtained from captured images, or facial features obtained from captured images).

[0074] The face detection unit 43 may be configured by the processor 13A executing a predetermined program. The face detection unit 43 detects the face portion of a person from the captured image acquired by the classroom camera 11 using a known method. Since the classroom camera 11 captures the faces of all students attending the lecture in classroom S, the faces of all students attending the lecture in classroom S are captured, and the face detection unit 43 detects the faces in the captured image.

[0075] The feature acquisition unit 44 may be configured by the processor 13A executing a predetermined program. The feature acquisition unit 44 acquires (also called generating) face features using the face portion detected by the face detection unit 43.

[0076] The face matching unit 45 may be configured by the processor 13A executing a predetermined program. At each timing when matching should be performed (hereinafter referred to as matching timing), the face matching unit 45 acquires an image from the classroom camera 11 and detects the face portion using the face detection unit 43. The matching timing may be set at predetermined intervals (e.g., every 15 minutes) after the start of class. Subsequently, the face matching unit 45 acquires face features from the detected face portion using the feature acquisition unit 44. Next, the face matching unit 45 compares all acquired face features with the matching database 51 held in the matching data storage unit 41 and obtains the IDs of all students that were matched. Subsequently, the face matching unit 45 outputs the IDs of all students that were matched to the matching result update unit 47. Students who were matched were present in the classroom S at the time the image was captured by the classroom camera 11. In addition, if there are face features that could not be matched in the matching database 51, the face matching unit 45 outputs those face features to the matching data update unit 48.

[0077] The matching result storage unit 46 may be composed of storage 13D. After the lecture has started, the matching result storage unit 46 stores information (matching result information) indicating whether or not matching and facial recognition was successful at each matching timing as a matching result database 53.

[0078] The matching result database 53 may include a matching result table 53A in which matching result information is recorded. Figure 14 shows an example of a matching result table 53A. The matching result table 53A records, in association with the ID of a student who was confirmed to have entered classroom S or who was confirmed to have been inside classroom S, and a truth value indicating whether or not that student was able to be matched (face-recognized) by the face matching unit 45 at each matching timing.

[0079] If the face recognition unit 45 is able to recognize a person's face at the matching timing, it means that the person whose face was recognized was present in classroom S. In other words, the matching result database 53 records that the person whose face was recognized was present in classroom S at the matching timing.

[0080] Furthermore, the matching result table 53A may also record whether or not facial recognition was performed by the entrance camera 10 for each ID.

[0081] The matching result update unit 47 may be configured by the processor 13A executing a predetermined program. The matching result update unit 47 associates the matching timing with the IDs of all students that were matched by the face matching unit 45 at that matching timing, writes them to the matching result table 53A, and updates the matching result database 53.

[0082] The matching data update unit 48 may be configured by the processor 13A executing a predetermined program. The matching result update unit 47 refers to the face information table 35A held in the face recognition server 12 and obtains data from the face information of all students that matches the face features that could not be matched by the face matching unit 45. When face information that matches the face features that could not be matched by the face matching unit 45 is obtained, the matching result update unit 47 obtains the matching face information, the corresponding ID (student ID number), and the corresponding student's name, and causes the matching result update unit 47 to update the matching database 51 held in the matching data storage unit 41. As a result, the ID, face information, and name of students who could not be properly photographed by the entrance camera 10 at the time of entry but were photographed and facially recognized by the classroom camera 11 are added to the matching database 51, and the matching database 51 is updated.

[0083] When the matching data update unit 48 obtains face information from the face information of all students that matches the face features that could not be matched by the face matching unit 45, it should record the student's ID and the fact that the student corresponding to that ID was facially authenticated in classroom S in the matching result database 53. This ensures that even students who were not properly photographed by the entrance camera 10 upon entry but were photographed and facially authenticated by the classroom camera 11 are recorded in the matching result database 53 as having been present in classroom S.

[0084] The attendance determination unit 49 may be configured by the processor 13A executing a predetermined program. When the lecture ends, the attendance determination unit 49 determines whether the student recorded in the matching result database 53 can be considered to have attended the lecture (attendance determination) based on the matching result database 53. In making this determination, the attendance determination unit 49 may refer to the truth values ​​recorded in the matching result database 53. For example, the attendance determination unit 49 may determine that a student was present if 90% or more of the truth values ​​recorded in the matching result database 53 are true (〇), taking into account the accuracy of the classroom camera 11 and face matching. The attendance determination unit 49 performs attendance determination for each ID recorded in the matching result database 53 and sends the determination result along with the corresponding ID as the attendance determination result to the management server 5.

[0085] When the management server 5 (attendance information management unit 24) obtains the attendance determination result from the edge server 13 (attendance determination unit 49), it writes the obtained attendance determination result to the attendance information database 26.

[0086] Next, in order to implement the admission management method according to the present invention, the admission management system 1 will describe the admission management process performed by executing the admission management program, with reference to the flowcharts shown in Figure 15 and Figure 16.

[0087] Figure 15 shows the flow of the process executed by the facial recognition server 12 (hereinafter referred to as the entry process). The entry process begins after the lecture being held in the same classroom S has finished, prior to the lecture in which attendance is taken.

[0088] When the entry process begins, the face recognition server 12 first acquires images from the entrance camera 10 and performs face detection to detect a human face from among them (S200). If face detection is successful, the face recognition server 12 acquires face features (first face information) based on the face image acquired by the entrance camera 10, and performs a first matching using the face information database 35 (registered database) in which the face information of all students is registered, to determine whether face recognition is possible (i.e., whether face information that is determined to be the same person as the person corresponding to the face features of the acquired image exists in the face information database 35) (S201, first matching step).

[0089] If the first matching is successful (i.e., if face information that matches the face features of the captured image exists in the face information database 35, the answer is Yes in S202), the face authentication server 12 sends the face information that matches the face features of the captured image, the corresponding ID (student ID number), and the name to the edge server 13 (S202). If face authentication fails (i.e., if face information that matches the face features of the captured image could not be found in the face information database 35, the answer is No in S202), the face authentication server 12 performs face detection on the captured image (S200) and determines whether face authentication is possible or not (S201).

[0090] When the facial recognition server 12 transmits facial information that matches the facial features of the image captured by the entrance camera 10, along with the corresponding ID (student ID number) and name, to the edge server 13, the edge server 13 registers the acquired facial information, ID, and name in the matching database 51 as facial information of a person who may have entered classroom S (first registration step). Note that the facial information recorded in the matching database 51 may be facial information corresponding to the image captured by the entrance camera 10 (facial images included in the captured image, facial features obtained from the captured image, etc.).

[0091] When the facial information matching the facial features of the captured image, the corresponding ID (student ID number), and the name are sent to the edge server 13, the facial recognition server 12 instructs the management server 5 to write to the attendance information database 26 that the entry of the student corresponding to that ID has been confirmed (S203). As a result, when a student is captured by the classroom camera 11 and facial recognition is performed, the attendance information database 26 records that there is a possibility that the student entered classroom S. At this time, the facial recognition server 12 may send an entry confirmation notice from the management server 5 to the user terminal 6 indicating that there is a possibility that the student entered classroom S, and the user terminal 6 may display a message indicating that entry has been confirmed.

[0092] Next, the facial recognition server 12 determines whether the end time of the lecture for which attendance is being checked has been reached (S204). If the end time of the lecture for which attendance is being checked has been reached (Yes in S204), the entry process is completed. If the end time has not been reached (No in S204), the facial recognition server 12 acquires the captured image again and performs face detection (S200).

[0093] The flowchart in Figure 16 shows the flow of the process (hereinafter referred to as the attendance confirmation process) performed by the edge server 13 when it executes the admission management program. The attendance confirmation process is a process that takes images using the classroom camera 11 while the lecture is in progress, performs facial recognition (second matching step), records whether each student is present in classroom S at the time of facial recognition (second registration step), performs attendance determination to identify the person who was present in classroom S and attending the lecture, and sends the information identifying that person to an external source (management server 5) (transmission step). This process is started after the lecture has begun.

[0094] When attendance confirmation processing begins, the edge server 13 first determines whether it is time for matching (S300). If it is not time for matching (No in S300), the edge server 13 waits until it is time for matching. If it is time for matching (Yes in S300), the edge server 13 acquires the image captured from the classroom camera 11 and performs face detection (S301).

[0095] When a face is detected, the edge server 13 obtains facial features from the image of the detected face, compares the obtained facial information with the facial information recorded in the matching database 51 (second matching), and performs facial authentication (S302, second matching step). When matching is successful by the matching database 51 (i.e., facial authentication is successful), the edge server 13 obtains the corresponding ID from the matching database 51 and records in the matching result table 53A that facial authentication was successful (i.e., attendance was confirmed), associating it with the matching timing (matching number, time, etc.) at which the matching was performed (S303, second registration step). As a result, the matching result database 53 records that the person whose face was authenticated was present in classroom S at the time of the second matching.

[0096] If matching fails (face recognition fails), the edge server 13 sends a matching command to the face recognition server 12 requesting it to match the face information (face portion or face features of the captured image) obtained from the classroom camera 11 with the face information database 35 held in the face recognition server 12 (S304).

[0097] When the facial recognition server 12 receives a matching command, it performs a third-stage matching (third-stage matching) with the facial information obtained from the edge server 13 and determines whether there is any matching facial information in the facial information database 35. If there is matching facial information, the facial recognition server 12 sends a notification that the matching was successful, along with the matching facial information, the ID (student ID number) and name that match that facial information, to the edge server 13. If there is no matching facial information, the facial recognition server 12 sends a notification to the edge server 13 that the matching was unsuccessful.

[0098] When the edge server 13 receives notification that the matching was successful (Yes in S305), it adds the facial information transmitted from the facial recognition server 12, along with the corresponding ID and name, to the matching database 51 (S306). When the edge server 13 receives notification that the matching was unsuccessful (No in S305), it determines again whether it is time for matching (S300). At this time, the edge server 13 may output (send) an alert signal to various terminals, including the user terminal 6, to notify the corresponding person or administrator that the matching was unsuccessful.

[0099] When the edge server 13 receives notification that the matching was successful and has finished adding the facial information to the matching database 51, it adds a new ID to the matching result table 53A and records the matching timing (matching number, time, etc.) and the information indicating that facial recognition was successful in association with each other (S303). As a result, the matching result database 53 records that the person whose face was recognized during the third matching was present in classroom S.

[0100] Once recording to the matching result table 53A (S303) is complete, the edge server 13 determines whether the matching completion time has arrived (or has elapsed) (S307). The matching completion time is set to be the same as the lecture end time, or it may be set to a time slightly earlier than the lecture end time. If the matching completion time has not arrived, the edge server 13 determines whether it is time to perform face matching again (S300).

[0101] When the matching completion time has arrived, the edge server 13 performs attendance determination for each student with an ID registered in the database, based on at least one of the matching database 51 and the matching result database 53, and sends information (e.g., an ID) that identifies the person who was present (the person whose presence was confirmed in classroom S during class time) to the management server 5 (S308, transmission step). For example, the edge server 13 may perform attendance determination based only on the matching result database 53. Specifically, the edge server 13 may determine that a student was present when information indicating that facial recognition was performed at all matching timings is recorded in the matching result database 53. The edge server 13 may also determine that a student was present when information indicating that facial recognition was performed at 90% or more of the matching timings is recorded in the matching result database 53.

[0102] When the management server 5 receives the attendance judgment results, it records the received judgment results in the attendance information database 26 for each ID. At the same time, the management server 5 sends the received judgment results to each of the corresponding student user terminals 6. When a user terminal 6 receives the judgment results, it displays a message to that effect on its touch panel display.

[0103] Once the notification of the judgment result is complete, the edge server 13 finishes the attendance confirmation process.

[0104] Next, in order to implement the admission management method according to the first embodiment, the admission management process performed by the admission management system 1 will be described in detail with reference to the time charts shown in Figures 17 and 18. Figures 17 and 18 illustrate the process for a single student.

[0105] The upper half of Figure 17 shows a time chart after the end of a lecture period before attendance checks are conducted, and before the start of a lecture where attendance checks are conducted. The lower half of Figure 17 shows a time chart after the start of a lecture, when facial recognition is performed upon entering classroom S and the information is registered in the matching database 51.

[0106] As shown in Figure 17, after the lecture held in the period preceding the lecture for which attendance checks are conducted has finished, the entrance camera 10 takes a photograph (S400), and the face recognition server 12 acquires the image captured by the entrance camera 10 (S401). Subsequently, the face recognition server 12 performs face recognition by comparing it with the face information database 35, which contains the face information of all students (S402, first matching step). If face recognition is successful, the face recognition server 12 transmits the face information (first face information) recorded in the face information database 35, along with the ID and name, to the edge server 13 (S403) to be recorded in the matching database 51 (S404, first registration step). As a result, the face information of the person whose face recognition was successful using the image captured by the entrance camera 10 is recorded in the matching database 51 as the face information of a person who may have entered classroom S. The facial information recorded in the matching database 51 may be facial information (registered facial information) from the facial information database 35 (registered database), or it may be facial information (facial image or facial features) corresponding to the image captured by the entrance camera 10.

[0107] When the lecture begins, the edge server 13 acquires images (S500) captured by the classroom camera 11 at each matching timing (S501) and performs facial recognition using the matching database 51 (S502, second matching step). The edge server 13 records the results of facial recognition in the matching result database 53. The edge server 13 continues to perform facial recognition based on the images captured by the classroom camera 11 at each matching timing until the matching end time has passed, and records the results in the matching result database 53 (second registration step). As a result, if facial recognition is successful at a matching timing, the matching result database 53 records that the person whose face was recognized was present in classroom S.

[0108] Once the matching completion time has passed, the edge server 13 uses at least one of the matching database 51 and the matching result database 53 (in this embodiment, the matching result database 53) to determine attendance for each student (S503), and sends the attendance determination result to the management server 5 (S504, transmission step).

[0109] When the management server 5 receives the attendance judgment result, it records the received judgment result in the attendance information database 26 for each ID (S505). At the same time, the management server 5 sends the received judgment result to each of the corresponding student's user terminals 6 (S506). When the user terminal 6 receives the judgment result, it displays a message to that effect on its touch panel display (S507).

[0110] Furthermore, the management server 5 stores each student's course registration information and generates a list of students taking the lectures for which attendance checks have been performed, based on that registration information. Subsequently, the management server 5 may determine whether each student in the list was present or absent, based on the attendance determination results from the edge server 13. Once the attendance / absence determination results are obtained, the management server 5 may be configured to send those results to the user terminal 6.

[0111] Figure 18 shows a time chart for the case where facial imaging is not performed upon entry into classroom S, but is performed inside classroom S and then facial recognition is carried out. Even in the case shown in Figure 18, once the lecture begins, the edge server 13 acquires the image captured by the classroom camera 11 (S500) at the matching timing (S501) and performs facial recognition using the matching database 51 (S502). In this case, since facial imaging was not performed upon entry, the captured student's facial information is not recorded in the matching database 51, resulting in a failure of facial recognition.

[0112] If matching fails (face recognition fails), the edge server 13 sends a matching command to the face recognition server 12 requesting it to match the face information obtained from the classroom camera 11's captured image with the face information database 35 (S600).

[0113] When the face recognition server 12 receives a matching command, it uses the face information database 35 to determine whether there is any face information that matches the face information acquired from the classroom camera 11, thereby performing face recognition (S601). If there is matching face information, the face recognition server 12 sends the matching face information, the corresponding ID and name to the edge server 13 (S602). When the edge server 13 receives the matching face information, ID and name from the face recognition server 12, it adds them to the matching database 51. At this time, when there is matching face information, the edge server 13 adds the person's ID to the matching result database 53, adds the new ID, and records the matching timing and information indicating that face recognition was successful, in association with each other. As a result, the matching result database 53 records that the person was in classroom S (S603).

[0114] Subsequently, similar to the case where facial recognition was performed upon entering classroom S, at each matching timing, the image captured by the classroom camera 11 is acquired by the edge server 13 and facial recognition is performed (S500~S502).

[0115] When the lecture ends, the edge server 13 performs an attendance check (S503), similar to when facial recognition was performed upon entering classroom S, and sends the result to the management server 5 (S504). The management server 5 records the result in the attendance information database 26 (S505) and sends the result to the user terminal 6 (S506). The user terminal 6 displays the transmitted result (S507).

[0116] Next, we will explain the effects of the admission management system 1 configured in this way.

[0117] As shown in Figure 17, when a face image is acquired by the entrance camera 10 (S401), the face recognition server 12 performs a comparison (first comparison) between the face image acquired by the entrance camera 10 and the face information database 35 (registered database) (S402). Subsequently, the face recognition server 12 transmits the face information from the face information database 35 (registered database) that was matched to the edge server 13 (S403), and the edge server 13 registers the received face information in the comparison database 51 (entry candidate database) (S404).

[0118] When the lecture begins, the edge server 13 performs a second comparison (S502) between the facial image acquired by the classroom camera 11 (area camera) and the matching database 51 (attendance candidate database). Based on the results of the second comparison of the image captured by the classroom camera 11 using the matching database 51, i.e., the data in the matching result database 53, the edge server 13 determines whether or not the person was present at the lecture and sends the determination result to the management server 5 (S503, S504). Upon receiving the determination result, the management server 5 records in the attendance information database 26 that the person was determined to be present as a result of facial recognition (S505).

[0119] As a result, facial information corresponding to students who enter classroom S is registered in the matching database 51 from the facial information database 35. The entry management system 1 performs facial authentication by matching the captured images acquired by the classroom camera 11 with the matching database 51. The facial authentication server 12 and edge server 13 constitute an entry management device 55 (see Figures 17 and 18) that maintains the facial information database 35 containing the registered facial information and performs facial authentication using the captured images acquired by the classroom camera 11.

[0120] Of the facial information recorded in the facial information database 35, the facial information corresponding to students who enter classroom S is registered in the matching database 51. Therefore, the number of facial information registered in the matching database 51 is less than or equal to the number of facial information recorded in the facial information database 35. For example, the number of facial information registered in the facial information database 35 corresponds to the number of students in an educational institution (several thousand), while the number of facial information that can be registered in the matching database 51 is thought to be roughly the number of students taking the course (several dozen to a hundred). Therefore, compared to matching images captured by the classroom camera 11 with the facial information database 35, the number of facial information to be matched can be reduced, thus reducing the load required for facial recognition.

[0121] Furthermore, the entrance management device 55 includes a face recognition server 12 that performs matching using images captured by the entrance camera 10, and an edge server 13 that performs matching using images captured by the classroom camera 11. Therefore, face recognition based on images captured by the classroom camera 11 is performed by the edge server 13 unless there is no face information in the matching database 51. This allows the processing load related to the matching required for attendance confirmation to be distributed to the edge server 13, thereby reducing the load on the face recognition server 12.

[0122] If, after the start of the lecture, the image captured by the classroom camera 11 cannot be matched against the matching database 51, the edge server 13 sends a matching command to the face recognition server 12 (S600). The face recognition server 12 then performs a third-stage matching (S601) against the image captured by the classroom camera 11 and the face information database 35. If the matching is successful, the edge server 13 records in the matching result database 53 that the person was present in classroom S (S603). After the lecture ends, the edge server 13 determines whether the person attended the lecture based on the records in the matching result database 53 and sends the determination result to the management server 5 (S602). Upon receiving the determination result, the management server 5 records in the attendance information database 26 that the person was determined to have attended as a result of face recognition (S603).

[0123] This improves the convenience of the entrance management system 1, as it allows for facial recognition even if, for example, the entrance camera 10 fails to properly capture a photograph at the time of entry and the information cannot be registered in the matching database 51, by matching the information with the facial information database 35.

[0124] <Second Embodiment> The admission management system 1 according to the second embodiment differs from the first embodiment in that it compares the matching results from the facial recognition server 12 and the edge server 13 with course registration information, which includes information about the student, and records it in the attendance information database 26. Furthermore, the admission management system 1 according to the second embodiment differs in that, as shown in Figure 19, it includes an administrator terminal 71 held by the administrator, who is a teacher, and notifies the administrator terminal 71 and the user terminal 6, taking the course registration information into consideration. The other configurations are generally the same as those of the first embodiment, so a description of similar configurations will be omitted.

[0125] The administrator terminal 71 may be composed of, for example, a tablet terminal equipped with 13 (CPU, MPU, etc.), memory such as volatile memory (RAM) or non-volatile memory (ROM), a touch panel display, a camera, a communication interface, and storage such as . The administrator terminal 71 may also be composed of a smartphone or a laptop computer.

[0126] The administrator terminal 71 is connected to the school's internal communication network 15. The administrator terminal 71 is configured to connect to the edge server 13 via the school's internal communication network 15, and to connect to the facial recognition server 12 via the router 17 and the external communication network 16.

[0127] The administrator terminal 71, like the user terminal 6, displays various information on its touch panel display and also accepts input of various information on the touch panel display.

[0128] As shown in Figure 20, the management server 5 includes a student information storage unit 21, a student information management unit 22, an attendance information storage unit 23, an attendance information management unit 24, as well as a lecture information management unit 81, a course registration information management unit 82, and a management display generation unit 83.

[0129] The lecture information management unit 81 maintains a lecture information database 85 that holds information related to lectures conducted at educational institutions (hereinafter referred to as lecture information), and stores and manages this lecture information.

[0130] The lecture information database 85 may include a subject information table 85A in which the lecture ID, the corresponding subject name, the day and time the lecture is held, and the classroom name are recorded as associated lecture information. Figure 21 shows an example of such a subject information table 85A. The lecture information management unit 81 may update the lecture information recorded in the subject information table 85A by receiving input from the administrator terminal 71.

[0131] The Course Registration Information Management Unit 82 acquires and manages data related to the lectures that each student is taking from user terminals 6, etc. The Course Registration Information Management Unit 82 maintains a Course Registration Information Database 87 in which information related to the lectures that each student is taking (hereinafter referred to as course registration information) is recorded.

[0132] The course registration information database 87 includes a course registration information table 87A in which each student's ID, the day of the week and time slot, and the lecture ID of the lecture the student is taking are recorded in association. Figure 22 shows an example of the course registration information table 87A. The course registration information management unit 82 may update the course registration information recorded in the course registration information table 87A by receiving input from user terminals 6 and administrator terminals 71.

[0133] The ID (student ID number) functions as a key linking the personal information table 25A and the course registration information table 87A, while the lecture ID functions as a key linking the subject information table 85A and the course registration information table 87A. These keys form a relational database in which the personal information table 25A, the course registration information table 87A, and the subject information table 85A are interconnected.

[0134] When a lecture held in the time slot preceding the lecture for which attendance confirmation is to be conducted has finished, the management display generation unit 83 extracts students who are taking the lecture for which attendance confirmation is to be conducted using the personal information table 25A, the course registration information table 87A, and the subject information table 85A, and generates student groups.

[0135] The management display generation unit 83 stores a student status table in which the attendance and attendance status of each student included in the student group is recorded, and updates the student status table at the start and end of the lecture.

[0136] The student status table stores each student's ID and name, along with their corresponding attendance information and attendance status. The attendance information is a boolean value; it becomes false when the lecture preceding the lecture for which attendance is being checked has finished, and becomes true when the student (student) corresponding to the ID has entered classroom S. The attendance status is also a boolean value and corresponds to the attendance determination result by the edge server 13.

[0137] The management display generation unit 83 acquires the authentication result when the face recognition server 12 performs face recognition based on the image capture results from the entrance camera 10. Upon acquiring the authentication result, the management display generation unit 83 uses the generated student group and the authentication result to set the attendance information of the face-recognized student to true. Subsequently, the management display generation unit 83 transmits the attendance information of all students in the student group to the administrator terminal 71. Upon receiving the attendance information of all students in the student group, the administrator terminal 71 displays the attendance information of all students in the student group on the touch panel display.

[0138] When the management display generation unit 83 obtains the attendance judgment results from the edge server 13, it uses the generated student groups and their attendance judgment results to generate attendance information for all students, associating each student's ID and name with their attendance judgment results. Subsequently, the management display generation unit 83 transmits the attendance information for all students to the administrator terminal 71. Upon receiving the attendance information for all students, the administrator terminal 71 displays the attendance information for all students on its touch panel display.

[0139] Figure 23 shows a time chart of the entry management process performed by the entry management system 1 to implement the entry management method according to the second embodiment, specifically when facial recognition is performed upon entry to classroom S.

[0140] As shown in Figure 23, after the lecture held in the time slot preceding the lecture for which attendance checks are conducted has finished, the entrance camera 10 takes a photograph (S400), and the face recognition server 12 acquires the image captured by the entrance camera 10 (S401). Subsequently, the face recognition server 12 performs face recognition by comparing it with the face information database 35, which contains the face information of all students (S402). If face recognition is successful, the face recognition server 12 transmits the face information recorded in the face information database 35, along with the ID and name, to the edge server 13 (S403) to be recorded in the matching database 51 (S404).

[0141] At this time, if the facial recognition server 12 is able to perform facial recognition, it sends authentication data including facial information, ID, and name to the management server 5 (S700). Upon receiving the authentication data, the management server 5 compares it with the student group and updates the attendance information of the students whose faces have been recognized (S701). Once the attendance information update is complete, the management server 5 sends the attendance information of all students to the management terminal (S702). Upon receiving the attendance information, the administrator terminal 71 displays the received attendance information on the touch panel display (S703).

[0142] When the lecture begins, as in the first embodiment, images are captured by the classroom camera 11 at each matching timing (S500), the captured images are acquired by the edge server 13 (S501), and facial recognition is performed using the matching database 51 (S502).

[0143] Once the matching completion time has elapsed, the edge server 13 performs an attendance determination (S503), similar to the first embodiment, and sends the determination result to the management server 5 (S504).

[0144] When the management server 5 obtains the judgment result, it matches it with the student group (S704) and records the attendance judgment result for each student as attendance information in the student status table (S705). Once the recording of attendance information for all students is complete, the management server 5 sends the attendance judgment result for each student to the user terminal 6 held by the student (S706). When the user terminal 6 receives the judgment result, it displays the received attendance judgment result on the touch panel display (S707).

[0145] As a result, a message indicating that the student was considered absent will be displayed on the user terminal 6 of the student who was absent from the lecture. Furthermore, considering the possibility that facial recognition may not have been performed correctly, it is advisable to include a designated input field and a submit button on the screen displaying the absence notification.

[0146] Although management server 5 is not included in the student group, if there are students who are determined to be present during attendance checks, it may identify the email addresses of those students and send an email to those addresses notifying them that they have not registered for the course. In this case, the email may include links to the input page for course registration and to the page for completing surveys.

[0147] Subsequently, the management server 5 transmits attendance information for all participants to the administrator terminal 71 (S708). Upon receiving the attendance information for all participants, the administrator terminal displays the received attendance information on its touch panel display (S709). At this time, the administrator terminal 71 should display the location and name of each participant whose attendance has been confirmed on its touch panel display.

[0148] Furthermore, if the management server 5 fails to match the face information using the matching database 51 of the edge server 13, it sends a matching command to the face recognition server 12, similar to the first embodiment. Upon receiving the matching command, the face recognition server 12 performs face matching using the face information database 35 based on the face information that could not be matched by the edge server 13.

[0149] In this case, if matching with the facial information database 35 is not possible (i.e., facial matching with all students is not possible), the facial authentication server 12 notifies the management server 5 of this fact. At this time, the management server 5 sends an alert signal to the administrator terminal 71 indicating that there is a person who cannot be authenticated even with the facial information of all students. The administrator terminal 71 displays on its touch panel display that a person who could not be matched with the facial information of all students is enrolled in the course.

[0150] The management server 5 may send various signals to terminals held by individuals whose facial information could not be matched using the facial information database 35. In such cases, the management server 5 may send an alert signal to the terminal held by the individual whose facial information could not be matched using the facial information database 35, indicating that the facial information could not be matched. Upon receiving the alert signal, the terminal may display text or icons on its touch panel display indicating that facial authentication could not be performed. This makes it possible to notify teachers and the individuals whose facial information could not be matched, thereby improving the convenience of the access control system 1.

[0151] <Third Embodiment> The entrance management system 1 according to the third embodiment differs from the first and second embodiments in that the facial recognition server 12 also retains the course registration information of each student. However, the other configurations are the same as those of the first embodiment (or second embodiment), so a description of the other similar configurations will be omitted.

[0152] Course registration information may be recorded, for example, in a facial information database 35. Figure 24 shows an example of a facial information database 35 according to the third embodiment. The facial information database 35 records information related to the classroom S where the lectures that each student is taking are held, along with the corresponding days of the week and time slots, as group setting information.

[0153] Figure 25 shows a time chart of the process from course registration to the recording of group settings in the facial information database 35. When a user enters course information on the course information input screen displayed on the user terminal 6 and performs the transmission process, the user terminal 6 sends the course information to the management server 5 (S800). The course information transmitted from the user terminal 6 may correspond to, for example, the data recorded in the course information table 87A.

[0154] When the management server 5 receives course registration information from the user terminal 6, it registers the course registration information in the course registration information table 87A (S801). Once registration is complete, the management server 5 generates a group setting for each student based on the course registration information table 87A and the subject information table 85A, which includes information about the classroom S where the lectures being taken are held, and the corresponding days and times. The group setting information may be, for example, a list of days, times and classroom S for the lectures being taken, as shown in Figure 24. Once the generation of group setting information for all students is complete, the management server 5 associates the ID (student ID number) with the corresponding group setting information and sends it to the facial recognition server 12 (S802).

[0155] When the management server 5 receives the group configuration information, it associates the group configuration information with an ID and registers it in the face information database 35 (S803).

[0156] The group setting information is used by the face recognition server 12 when performing face recognition on the images captured by the entrance camera 10. When performing face recognition on the images captured by the entrance camera 10, the server extracts students attending lectures held in classroom S where the entrance camera 10 is installed, based on the group setting. Subsequently, the face recognition server 12 uses the facial information of the extracted students to perform face recognition on the facial portions detected in the images captured by the entrance camera 10. This reduces the number of facial information sets that the face recognition server 12 needs to match when performing face recognition on the images captured by the entrance camera 10, thereby reducing the load on the face recognition server 12.

[0157] The facial recognition server 12 may configure the matching database 51 by extracting students attending lectures for which attendance confirmation is required, based on group setting information, and generating groups. In this case, the facial recognition server 12 sends the configured matching database 51 to the edge server 13, and the edge server 13 uses the matching database 51 to perform facial recognition. This allows the matching database 51 to be generated in a simple manner without relying on images captured by the entrance camera 10.

[0158] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments.

[0159] In the above embodiment, the face recognition server 12 and the edge server 13 each performed matching based on face information, but the threshold for face recognition settings (face recognition threshold) that indicates successful matching when the threshold is exceeded may be set differently. In this case, the face recognition threshold in the face recognition server 12 may be set higher than the face recognition threshold in the edge server 13. This allows face recognition to be performed under stricter conditions at the time of entry to confirm entry, thereby lowering the level of face recognition after entry and reducing the load on face recognition in the edge server 13.

[0160] For example, the entrance management system 1 may have a cloud server located outside the school that performs the same functions as the edge server 13, instead of the edge server 13. The cloud server may be connected to an external communication network 16 and configured to connect to the entrance camera 10 and classroom cameras 11 via a router 17.

[0161] In the above embodiment, the face recognition server 12 and the edge server 13 were configured as separate entities in the access control system 1, but the system is not limited to this configuration. Figure 26 shows a system configuration diagram when the face recognition server 12 and the edge server 13 are integrated. As shown in Figure 26, the access control system 1 may have a single server (hereinafter referred to as the authentication server 100) that performs the processing performed by the face recognition server 12 and the processing performed by the edge server 13 by executing an access control program, instead of the face recognition server 12 and the edge server 13. In that case, the authentication server 100 may be a cloud server located outside the school. In that case, since the face information database 35 and the matching database 51 will be managed by a single authentication server 100, it is preferable to create a dedicated tenant for the matching database 51 and manage the matching database 51 in that dedicated tenant.

[0162] In the above embodiment, an example was described in which the admission management system 1 is applied to a system for managing entry to classrooms for attendance confirmation in an educational institution. However, it may also be applied to a system for managing entry to theaters, theaters, etc. In that case, an entrance camera 10 is provided at the place where tickets are checked, and one or more area cameras equivalent to classroom cameras 11 are provided inside the theater or theater. The area cameras may consist of infrared cameras (IR cameras). The area cameras may be installed in a position that can capture images of the passageway connecting the entrance to the inside of the theater or theater. In addition, the area cameras may be installed in the lobby or foyer.

[0163] When applied to a system for verifying entry to theaters, theaters, etc., when facial recognition is successful, the facial recognition server 12 and edge server 13 may send a notification via email or other means to the user terminal 6 held by the recognized audience member. In this case, if the audience member is seated in a location other than their assigned seat, the user terminal 6 may notify the audience member that they are seated in the wrong location. Furthermore, depending on the congestion level in the theater or theater, the user terminal 6 may notify the audience member of the route to where they should be seated.

[0164] However, the admission verification systems to which admission management system 1 can be applied are not limited to admission verification systems for theaters and theatrical performances, but may also be applied to admission verification systems for stadiums, etc. In addition, admission management system 1 may also be applied to admission verification systems for music festivals, etc., where seats are not assigned.

[0165] In addition, the access control system 1 may be applied to an office occupancy verification system. In that case, the entrance camera 10 should be installed near the entrance of the room, and the area camera should be installed inside the room. The system should be configured so that if facial recognition is performed multiple times using the images captured by the area camera, and the presence of the employee in the area (room) is confirmed multiple times, the system determines that the recognized employee is at work and proceeds with the attendance process. [Industrial applicability]

[0166] The admission management method, admission management system 1, and admission management device related to this disclosure have the effect of reducing the load required for facial recognition and are useful as admission management methods, admission management systems, and admission management devices that manage the entry of persons into a predetermined area by authentication using registered facial information. [Explanation of Symbols]

[0167] 1: Access control system 2: School administration system 3: Attendance Management System 5: Management Server 5A: Processor 5B: Memory 5C: Communication Interface 5D: Storage 6: User terminal 10: Entrance camera 11: Classroom Camera 12: Face Recognition Server 12A: Processor 12B: Memory 12C: Communication Interface 12D: Storage 13: Edge Server 13A: Processor 13B: Memory 13C: Communication Interface 13D: Storage 15: Communication Networks 16: Communication Networks 17: Router 21:Student information storage department 22:Student Information Management Department 23: Attendance Information Storage Unit 24: Attendance Information Management Department 25: Personal Information Database 25A: Personal Information Table 26: Attendance Information Database 26A: Attendance Information Table 31: Facial Information Memory Unit 32: Face detection unit 33: Feature extraction unit 34: Entrance Processing Section 35: Facial Information Database (Registered Database) 35A: Face Information Table 41: Matching data storage unit 42: Matching Data Management Department 43: Face detection unit 44: Feature extraction unit 45: Face Recognition Unit 46: Matching Result Storage Unit 47: Matching Result Update Unit 48: Matching Data Update Unit 49: Attendance Determination Department 51: Matching Database (Candidate Database for Admission) 53: Matching Result Database 53A: Matching Result Table 55: Entrance control device 71: Administrator terminal 81: Lecture Information Management Department 82: Course Registration Information Management Department 83: Management display generation section 85: Lecture Information Database 85A: Subject Information Table 87: Course Registration Information Database 87A: Course Information Table 100: Authentication server (access control device) S: Classroom (designated area)

Claims

1. An entry management method that manages the entry of a person into a designated area by authentication using registered facial information, An entrance camera that acquires a person's facial image near the entrance of the aforementioned designated area, An area camera that acquires facial images of people located within the aforementioned predetermined area, This is carried out by an access control system comprising an access control device that maintains a registered database containing registered facial information and performs facial recognition using facial information based on facial images acquired by the entrance camera and the area camera. The aforementioned entry control device is A first matching step is performed to compare first facial information based on a facial image acquired by the entrance camera with registered facial information recorded in the registered database, and to determine whether or not there is registered facial information that is determined to be the same person as the first facial information. If it is determined in the first matching that such a person exists, the first registration step involves registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area. A second matching step involves performing a second matching, which involves comparing facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database. A second registration step involves recording the results of the second matching in the matching results database, An admission management method that includes a transmission step of transmitting to an external party information that identifies a person present in a predetermined area, based on at least one of the admission candidate database and the matching result database.

2. The access control method according to claim 1, wherein, in the second verification step, if the second verification fails, the access control device performs a third verification by comparing the image captured by the area camera with the registered database, and if the third verification is successful, records the result in the verification result database.

3. The admission management method according to claim 2, wherein the admission management device registers the face information corresponding to the image captured by the area camera when the third matching is achieved in the second matching step into the admission candidate database.

4. The access control method according to claim 2, wherein the access control device outputs an alert to notify the corresponding person or manager if the third verification fails in the second verification step.

5. The admission management method according to claim 1, wherein the admission candidate database is configured as a group of facial information included in the registered database.

6. The access control device comprises a face recognition server that performs the first verification and an edge server that performs the second verification, according to claim 1.

7. An access control system that manages the entry of individuals into a designated area by authentication using registered facial information, An entrance camera that acquires a person's facial image near the entrance of the aforementioned designated area, An area camera that acquires facial images of people located within the aforementioned predetermined area, The system includes an access control device that maintains a registered database containing registered facial information and performs facial recognition using facial information based on facial images acquired by the entrance camera and the area camera, The aforementioned entry control device is A first matching step is performed to compare first facial information based on a facial image acquired by the entrance camera with registered facial information recorded in the registered database, and to determine whether or not there is registered facial information that is determined to be the same person as the first facial information. If it is determined in the first matching that such a person exists, the first registration step involves registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area. A second matching step involves comparing facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database. A second registration step in which the results of the second matching step are recorded in the matching result database, An admission management system that performs a transmission step of transmitting to an external party information that identifies a person present in a predetermined area, based on at least one of the admission candidate database and the matching result database.

8. An access control device that manages the entry of persons into a designated area by authentication using registered facial information, An entrance camera that acquires a person's facial image near the entrance of the aforementioned designated area, An area camera that acquires facial images of people within the predetermined area is connected to the area camera, It maintains a registered database containing registered facial information, A first matching step is performed to compare first facial information based on a facial image acquired by the entrance camera with registered facial information recorded in the registered database, and to determine whether or not there is registered facial information that is determined to be the same person as the first facial information. If it is determined in the first matching that such a person exists, the first registration step involves registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area. A second matching step involves comparing facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database. A second registration step in which the results of the second matching step are recorded in the matching result database, An admission management device that performs a transmission step of transmitting to an external party information that identifies a person present in a predetermined area, based on at least one of the admission candidate database and the matching result database.

9. An access control program that manages the entry of individuals into a designated area by authentication using registered facial information, An entrance camera that acquires a person's facial image near the entrance of the aforementioned designated area, An area camera that acquires facial images of people located within the aforementioned predetermined area, This is performed by an access control system comprising: an access control device that maintains a registered database containing registered facial information and performs facial recognition using facial information based on facial images acquired by the entrance camera and the area camera; The aforementioned entry control device is A first matching step is performed to compare first facial information based on a facial image acquired by the entrance camera with registered facial information recorded in the registered database, and to determine whether or not there is registered facial information that is determined to be the same person as the first facial information. If it is determined in the first matching that such a person exists, the first registration step involves registering the first facial information or the registered facial information in the entry candidate database as facial information of a person who may have entered the predetermined area. A second matching step involves comparing facial information based on facial images acquired by the area camera with facial information registered in the entry candidate database. A second registration step in which the results of the second matching step are recorded in the matching result database, An admission management program that performs a transmission step of transmitting to an external party information that identifies a person present in a predetermined area, based on at least one of the admission candidate database and the matching result database.

Images