Entrance / Exit Management System and Entrance / Exit Management Device

Abstract

To manage entrance / exit of entrance / exit persons in a non-contact manner without increasing components.SOLUTION: An entrance / exit management system includes: an imaging device that images an entrance / exit person to acquire image data; a storage device that stores face image data for the entrance / exit person; and a control device that manages entrance / exit of the entrance / exit persons on the basis of the image data for the entrance / exit persons imaged by the imaging device. The control device includes: authentication means that compares face image data included in the image data for the entrance / exit person imaged by the image device with the face image data previously stored in the storage device to perform face authentication; determination means that detects at least one of a position, a direction and movement of the entrance / exit person or an article associated the entrance / exit person included in the image data from the image device for the entrance / exit person having been subjected to the face authentication by the authentication means to determine entrance / exit of the entrance / exit persons on the basis of the detection result; and management means that stores, in the storage device, the determination result by the determination means together with time information in association with the entrance / exit persons as entrance / exit history information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a contactless entrance / exit management system and an entrance / exit management device that utilizes face recognition technology. [Background technology]

[0002] With the advancement of digital technology, face recognition technology that identifies individuals from facial images captured by digital cameras is already known. Utilizing this face recognition technology, for example, entrance / exit management devices that determine entry based on the first face recognition and exit based on the second face recognition are also already known. Furthermore, a time attendance management system that performs such face recognition when punching a time card to manage the times of entry and exit is also known (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-48326 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned entrance / exit management device or system that performs entrance / exit management solely through facial recognition, if it is necessary to distinguish between multiple intermediate entries and exits in addition to determining whether a person is an entry or exit based on the first facial recognition and the next facial recognition, it is necessary to carry out designated operations, including time stamping for intermediate exits and entries, when recognizing the face.

[0005] As a method for performing such a designation operation, making a selection by touching a contact-type button or a touch panel, etc., may be insufficient as a mechanism for preventing so-called contact infection.

[0006] In addition, by using a technology that is already known for forming a pseudo-image in space that appears to the human eye (for example, aerial three-dimensional image display technology), it is possible to prepare buttons for the aforementioned specified operations in space and operate them without contact; however, this has the disadvantage of increasing the size of the device due to the increase in components, such as a display for forming the pseudo-image and a sensor unit for detecting operations in the air.

[0007] The present invention has been made in consideration of the above circumstances, and aims to provide an entrance / exit management system and an entrance / exit management device that can manage the entrance and exit of people entering and exiting a facility in a non-contact manner without adding any additional components. [Means for solving the problem]

[0008] The entrance / exit management system of the present invention comprises an imaging device that images entering and exiting persons to obtain image data, a storage device that stores facial image data of the entering and exiting persons, and a control device that manages the entrance and exit of the entering and exiting persons based on the image data of the entering and exiting persons imaged by the imaging device, and the control device has an authentication means that performs facial authentication by comparing facial image data contained in the image data of the entering and exiting person imaged by the imaging device with facial image data previously stored in the storage device, a determination means that detects at least one of the position, orientation and movement of the entering and exiting person or an object associated therewith contained in the image data from the imaging device of the entering and exiting person whose face has been authenticated by the authentication means, and determines the entry and exit of the entering and exiting person based on the detection result, and a management means that associates the determination result by the determination means with the entering and exiting person together with time information and stores it in the storage device as entrance and exit history information.

[0009] The entrance / exit management device of the present invention comprises an imaging means for imaging entering and exiting persons to obtain image data, a storage means for storing facial image data of the entering and exiting persons, an authentication means for performing facial authentication by comparing facial image data contained in the image data of the entering and exiting person imaged by the imaging means with facial image data previously stored in the storage means, a determination means for detecting at least one of the position, orientation and movement of the entering and exiting person or an object associated therewith contained in the image data from the imaging means of the entering and exiting person whose face has been authenticated by the authentication means, and determining the entry and exit of the entering and exiting person based on the detection result, and a management means for storing the determination result by the determination means in the storage means associating it with the entering and exiting person together with time information as entrance and exit history information. Effect of the Invention

[0010] According to the present invention, it is possible to manage the entry and exit of people in a non-contact manner without increasing the number of components. [Brief description of the drawings]

[0011] [Figure 1] 1 is a diagram showing a configuration of an entrance / exit management system according to an embodiment of the present invention; [Diagram 2] 2 is a diagram showing an internal configuration of a control unit of the entrance / exit management system. FIG. [Diagram 3] FIG. 2 is a front view of a terminal device of the entrance / exit management system. [Figure 4] FIG. 13 is a diagram for explaining the entrance and exit history of people entering and leaving a facility. [Diagram 5] FIG. 2 is a diagram illustrating an example of a configuration of an entry / exit history management database. [Figure 6] 13 is a flowchart showing an entrance / exit management process. [Figure 7] FIG. 13 is a diagram showing an example of a display screen of a terminal device. [Figure 8] FIG. 13 is a diagram showing an example of a display screen of a terminal device. [Figure 9] FIG. 2 is a diagram showing an example of a facial gesture pattern. [Figure 10] FIG. 2 is a diagram showing an example of a facial gesture pattern. [Figure 11] FIG. 13 is a diagram showing an example of a display screen of a terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, an access control system and an access control device according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings. However, the following embodiments do not limit the invention according to each claim, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention. In addition, in the following embodiments, the scale, dimensions, etc. of each component may be shown exaggerated, and some components may be omitted.

[0013] FIG. 1 is a diagram showing the configuration of an entrance / exit management system according to an embodiment of the present invention. FIG. 2 is a diagram showing the internal configuration of a control unit of the entrance / exit management system. As shown in FIG. 1, the entrance / exit management system 1 of this embodiment is a system that records and manages information regarding the entrance / exit of multiple employees (entering / exiting persons) when they enter and exit a room of, for example, a business establishment, together with time information. The entrance / exit management system 1 includes a terminal device 10, an input device 30, an output device 40, a detection unit 50, and an external device 60. In this embodiment, the terminal device 10 of the entrance / exit management system 1 is connected to the external device 60 so as to be able to communicate bidirectionally with the external device 60, for example, when the external device 60 is an entrance / exit management device or the like.

[0014] The terminal device 10 of the entrance / exit management system 1 is installed, for example, as a management terminal at a predetermined location (such as in front of a door) of a business establishment, and records and manages various information related to the entrance and exit of multiple employees (entering and exiting persons). The predetermined location refers to a place where an entrance and exiting person stops temporarily when entering or exiting the establishment.

[0015] The terminal device 10 includes an imaging unit 11, a display unit 12, a control unit 13, a storage unit 14, and a recording medium 15. The imaging unit 11 of the terminal device 10 captures images of people entering and leaving the venue to obtain image data. The imaging unit 11 may be configured, for example, with a photographing device such as a digital camera, and an imaging element such as a CMOS or CCD. The imaging unit 11 may be configured to automatically capture face images from image data of people entering and leaving the venue to obtain still face image data, or may be configured to record images to obtain video image data. The imaging unit 11 may be configured to have a face recognition function and a motion detection function of people entering and leaving the venue, which will be described later, by itself.

[0016] The display unit 12 may be configured as a display device such as a display device, a monitor device, or a touch panel mechanism that is configured to be able to display images (particularly, facial images) of people entering and leaving the venue on the terminal device 10 installed at a predetermined location as described above. Specifically, the display unit 12 displays facial images based on image data of people entering and leaving captured by the imaging unit 11 so as to be visible to at least the people entering and leaving the venue.

[0017] From the viewpoint of only performing entrance and exit management in the entrance and exit management system 1, it is not necessarily necessary to display facial images of people entering and leaving. For this reason, in such a case, it can be said that the display unit (display device) 12 is not an essential component. However, if the display unit 12 is provided, people entering and leaving the venue can enter and leave while checking their own facial images displayed on the display unit 12. This improves convenience for people entering and leaving the venue.

[0018] The storage unit 14 stores image data of entering and exiting persons acquired by the imaging unit 11, and also stores face image data 17 of the entering and exiting persons. The storage unit 14 also stores various programs used for basic processing of the control unit 13 of the entrance and exit management system 1, as well as entrance and exit history information of the entering and exiting persons (entrance and exit history data 19) described later.

[0019] The storage unit 14 also stores various registered data, such as image data captured in advance by the imaging unit 11, face image data 17 of people entering and leaving extracted from image data acquired from an external device or the like via the recording medium 15 or the input / output I / F (interface) 13d, and the above-mentioned entrance / exit history data 19, as well as time data 18 relating to time. Examples of the storage unit 14 include a ROM, a RAM, a HDD, and an SSD.

[0020] The recording medium 15 is a medium such as a memory card or optical disk made of a removable non-volatile memory, configured so that various data stored in the memory unit 14 can be written to it in order to be output to the outside, and various data written externally can be read to it in order to be input.

[0021] The control unit 13 controls the entire entrance / exit control system 1 according to various programs stored in the memory unit 14, and also manages the entrance and exit (entrance and exit) of people entering and leaving the facility based on image data of the people captured by the imaging unit 11. The control unit 13 may be configured by an arithmetic processing device such as a SoC (System on a Chip) equipped with a CPU and a GPU.

[0022] 2, in this embodiment, the control unit 13 is configured to include, for example, an authentication unit 13a, a judgment unit 13b, a management unit 13c, an input / output I / F 13d, and an image processing unit 13e. Note that the control unit 13 may have a minimum configuration including at least the functions of the authentication unit 13a, the judgment unit 13b, and the management unit 13c.

[0023] The authentication unit (authentication means) 13a of the control unit 13 has a face authentication function for performing face authentication by comparing face image data included in the image data of entering and leaving persons captured by the imaging unit 11 with face image data previously stored in the storage unit 14. Specifically, the authentication unit 13a performs known face image extraction processing, face feature amount extraction processing, and the like, based on the image data of entering and leaving persons acquired by the imaging unit 11.

[0024] Then, based on the extracted facial image and its features, etc., the authentication unit 13a reads in, by image search, facial image data 17 of the registered person (entering or leaving person) to be facially authenticated, which has been stored (registered) in advance in the memory unit 14 and whose features, etc. match or are similar to those of the extracted facial image and its features, etc., and compares the facial images based on, for example, template matching processing, etc.

[0025] In this manner, the authentication unit 13a executes face authentication processing of people entering and leaving. Note that, depending on the face authentication result by the authentication unit 13a, for example, if authentication is OK, entry and exit is determined, but if authentication is NG, other processing such as a known re-authentication processing or entry and exit prohibition processing may be executed instead of the entry and exit determination.

[0026] The determination unit (determination means) 13b of the control unit 13 detects at least one of the position (standing position of the entering / exiting person, wearing position of the equipment, etc.), direction (direction (of the face) of the entering / exiting person, direction of the equipment, etc.), and movement (movement of the entering / exiting person's face, hands, head, etc., movement of putting on and taking off the equipment, etc.) of the entering / exiting person or an object accompanying the entering / exiting person (equipment, etc.) included in the image data from the imaging unit 11 of the entering / exiting person whose face has been authenticated by the authentication unit 13a, and determines whether the entering / exiting person has entered or left based on the detection result. Specifically, the determination unit 13b detects the facial movement (gesture) of the entering / exiting person included in the image data from the imaging unit 11, detects, for example, the direction of the facial image of the entering / exiting person, and determines whether the entering / exiting person has entered or left based on the direction of the detected facial image.

[0027] In determining entry / exit by the determination unit 13b, for example, if the facial image of the person entering or leaving after face authentication is detected to be in a state indicating that the person is facing to the left or right in front of the person entering or leaving, it is determined to be "entry," and if it is detected to be in a state indicating that the person is facing to the other left or right, it is determined to be "exit."

[0028] The management section (management means) 13c of the control section 13 associates the judgment result by the judgment section 13b with the entering and leaving persons together with the time information, and stores the association in the memory section 14, for example, as entrance and exit history information. Specifically, when the judgment result indicates "entry", the management section 13c associates the entrance record with the entry time and the registered information of the entering person (registration number, registered person name, registered face image, etc.). On the other hand, when the judgment result indicates "exit", the management section 13c associates the exit record with the exit time and the registered information of the entering and leaving persons.

[0029] The management unit 13c stores and manages various types of information for each entering and leaving person thus associated in the storage unit 14 so that they can be used as entrance and exit history information for each entering and leaving person. Note that the management unit 13c can also manage the entrance and exit records by counting (measuring time) from the changeover time of the day (such as midnight) and judging a judgment result indicating a first entry as the entering and leaving person's "arrival" and judging a judgment result indicating a final (last) exit as the entering and leaving person's "leave work."

[0030] The input / output I / F (input means, output means) 13d of the control unit 13 accepts voice input of people entering and leaving, for example, input via the input device 30. The input device 30 may be configured with various input devices, such as a microphone, keyboard, mouse, touch panel, etc., connected to the input / output I / F 13d. Examples of voice input include voice input of "enter," "exit," "save," "end," etc., uttered by people entering and leaving. The control unit 13 can control each unit and the external device 60, etc., based on such voice input.

[0031] The output device 40 may be configured with an output device such as a speaker, a light, a printer, etc., connected via the input / output I / F 13d. When the output device 40 is a speaker or a light, the control unit 13 may be configured to output the determination result by the determination unit 13b via the input / output I / F 13d in a manner that indicates the result with sound or light, thereby encouraging further understanding by people entering and leaving the facility.

[0032] The control unit 13 may output, for example, a control signal for controlling the external device 60 via the input / output I / F 13d based on at least one of the facial authentication result by the authentication unit 13a and the judgment result by the judgment unit 13b. For example, when the external device 60 is a door opening / closing device for an automatic entrance door, and the facial authentication result is authentication OK and the judgment result indicates "entry," the control unit 13 may output a control signal for opening the entrance door to the door opening / closing device.

[0033] The detection unit (detection means) 50 is composed of, for example, a thermal camera or various human sensors, and detects the biometric information (body temperature, iris pattern, etc.) of the person entering or leaving. The determination unit 13b may determine the entry and exit of the person entering or leaving based on at least one of the voice input result received via the input / output I / F 13d and the detection result detected by the detection unit 50, together with or in addition to the determination based on the detection result described above. In such a case, the control unit 13 can perform other processing, such as not permitting entry even if the person entering or leaving utters the word "enter" if the body temperature detected by the detection unit 50 is equal to or higher than a predetermined value, and presenting a warning on the display unit 12 to encourage re-measurement of the body temperature.

[0034] The image processing unit (image processing means) 13e of the control unit 13 performs various image processing on the image data (in particular, the facial images represented by the facial image data) of people entering and leaving captured by the imaging unit 11 so that the image meets a standard that can be used for facial authentication by the authentication unit 13a.

[0035] For example, when a person entering or leaving is wearing accessories such as a mask, hat, or glasses, the image processing unit 13e generates facial image data of the person entering or leaving without these accessories so that the authentication unit 13a can perform facial authentication taking these accessories into account, or generates facial image data of the person wearing these accessories using the facial image data 17 stored in the memory unit 14, thereby processing the image data so that it can be used for facial authentication regardless of the presence or absence of the accessories.

[0036] Examples of the external device 60 include, in addition to the door opening / closing device described above, an alarm device, a security management device, and an entrance / exit management device. As described above, the control unit 13 outputs various data and control signals to the external device 60 via the input / output I / F 13d based on the face authentication result and the determination result, provides various data to the external device 60 so that they can be used, and can control the operation of the external device 60. If the external device 60 is, for example, an entrance / exit management device, the entrance / exit management device can input and use the output entrance / exit history information, etc., to perform entrance / exit management of employees, etc., including multiple people entering and exiting a business establishment, etc. Note that the entrance / exit management can also be performed by, for example, the control unit 13 executing an entrance / exit management process in accordance with an entrance / exit management program stored in the storage unit 14.

[0037] According to the entrance / exit management system 1 configured in this manner, face authentication is performed and entrance / exit is determined based on face image data included in the image data of the person entering and leaving captured by the imaging unit 11, and further, entrance / exit history information can be generated, so that entrance / exit management of the person entering and leaving can be performed in a non-contact manner. Also, it is possible to configure the entrance / exit management system 1 by using the terminal device 10 alone to execute the entrance / exit management process with the minimum configuration including the imaging unit 11, the storage unit 14, and the control unit 13. This makes it possible to manage the entrance / exit of the person entering and leaving in a non-contact manner without increasing the number of components of the device, etc.

[0038] Fig. 3 is a front view of the terminal device. In the following description including Fig. 3, the same or corresponding components are denoted by the same reference numerals and duplicated description will be omitted. As described above, the terminal device 10 can be configured to be able to realize functions equivalent to those of the above-mentioned entrance / exit management system 1 by itself, for example.

[0039] 3, in terminal device 10, imaging section (imaging means) 11 for capturing images from a digital camera or the like is disposed, for example, on the right side of an upper bezel on the front side of the housing of terminal device 10, and is provided so as to be able to capture images of people entering and leaving and acquire image data. In addition, display screen 12A of display section (display means) 12, such as a display, for displaying and outputting image data acquired by imaging section 11 may be configured to include, for example, face image display area 12a for displaying face images based on face image data included in the image data of people entering and leaving captured by imaging section 11, date display area 12b for displaying text information indicating the date, and time display area 12c for displaying text information indicating the time.

[0040] The display unit 12 is not limited to the above, so long as it has a function capable of displaying and outputting image data captured by the imaging unit 11 and also displaying various information such as date and time. The storage unit 14 is not limited to the above, so long as it has a function capable of storing face image data 17 used when comparing face image data captured by the imaging unit 11 when identifying individuals who enter and exit, and time data 18 and entry / exit history data 19 constituting part of data (management database 20, see FIG. 5) that associates the time and date with various information of registered persons (entering and exiting persons) when recording entry and exit.

[0041] FIG. 4 is a diagram for explaining the entrance / exit history of entering and exiting persons. FIG. 5 is a diagram showing an example of an entrance / exit history management database. As shown in FIG. 4, the entrance / exit history of entering and exiting persons can be expressed as, for example, "Clock in," "Exit 1 to n (n is a positive integer)," "Re-entry 1 to n (n is a positive integer)," and "Clock out." The entrance / exit history data 19 can be configured by associating these entrance / exit histories with each entering and exiting person.

[0042] In this entry / exit history, on a 24-hour timeline, for example, the first (first time) entry of a specific person (attendee) who has been face-recognized by the terminal device 10 at midnight is determined to be an "attendee" based on the determination result and recorded. Also, the last (last) time this specific person exits is determined to be an "left work" based on the determination result and recorded.

[0043] Furthermore, during the time between "Clock in" and "Clock out", if a specific person is determined to have first exited after "Clock in", this is determined and recorded as "Exit 1" based on the determination result, and if a person is determined to have entered after this "Exit 1", this is determined and recorded as "Re-entry 1" based on the determination result. By repeating this type of entry / exit recording process for each person during the time between "Clock in" and "Clock out", the entry / exit history in the entry / exit history data 19 associated with the specific entry / exit person is recorded as an entry / exit record.

[0044] Here, the entry / exit history management database 20 including the facial image data 17, time data 18, and entry / exit history data 19 stored in the storage unit 14 will be described with reference to Fig. 5. As shown in Fig. 5, the management database 20 has columns such as "registration number," "registered person name," "registered facial image," and "entrance / exit record," and the facial image data 17, time data 18, and entry / exit history data 19 are managed and stored for each record.

[0045] For example, for an entering or exiting occupant with the registered name "ABCD" and registration number "0001" in the management database 20, in the record of the management database 20, the registered face image based on the face image data 17 is the image as shown in the figure, and the entry / exit record based on the time data 18 and the entry / exit history data 19 is recorded and managed / stored as "2020 / 11 / 2; 8:27; Arrive at work...2020 / 11 / 2; 19:53; Leave work."

[0046] In this way, in the management database 20, the facial image of a registered person (entering and leaving person), the input registration number and name of the registered person, and information about the registered person (time data 18 and entry / exit history data 19) can be recorded in association with each other in the facial image data 17. The various information for each registered person in this management database 20 can also be additionally recorded in association with the facial image data 17 by inputting data of the registered person created on another terminal device via, for example, a removable recording medium 15. Furthermore, in addition to the facial image of the registered person, it is also possible to add and register other related information other than the registered person's name and registration number to the facial image data 17 in this management database 20.

[0047] In addition, the time data 18 is recorded in association with the data of each registered person together with the entry / exit record (entrance / exit history data 19) of the person who has been authenticated by face authentication for personal identification. The recorded contents in the entry / exit record column are, for example, date, time, and information indicating entry or exit (including clock-in, exit 1-n, re-entry 1-n, and clock-out), and are recorded in date and time order for each record. As described above, since there is no limit to the number of re-entries and exits, the time data 18 and the entry / exit history data 19 are recorded in the management database 20 so that the number of fields in the record increases with each re-entry and exit, for example, exit 1, re-entry 1, exit 2, re-entry 2, exit 3, and re-entry 3.

[0048] In the terminal device 10 configured in this way, when the entrance / exit management process is executed in the control unit 13, it is possible to calculate the working hours per day from information on the arrival time and departure time based on the time data 18, or to calculate detailed working hours per day by adding information on each departure time and re-entry time. For example, the management database 20 or the time data 18 and the entrance / exit history data 19 can be output as is to the recording medium 15 so that they can be used by an external device (entrance / exit management device) 60. Also, entrance / exit management, including calculation of working hours, can be performed in cooperation with entrance / exit management software installed in the external device 60 via the input / output I / F 13d.

[0049] The terminal device 10 is configured to include an imaging unit 11 such as a digital camera for capturing images, and a display unit 12 such as a display for displaying people entering and leaving, but may be configured with various electronic devices including a smartphone, tablet terminal, etc. that have a camera function and a display function. In addition, the display unit 12 may have various configurations as long as it can display image data captured by the imaging unit 11 as an image. As an example, the terminal device 10 is configured to identify individuals (face authentication) after the imaging unit 11 captures an image of people entering and leaving, and record the time when (or after) the entry and exit is confirmed by the entry and exit judgment, and at that time, various operating states of the device can be displayed on the display unit 12.

[0050] Next, a series of entrance / exit management processes from capturing images of people entering and leaving to recording the entrance / exit in the entrance / exit management system 1 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the entrance / exit management process. First, when the entrance / exit management process is started, for example, when the detection unit 50 detects that an entering or leaving person has entered the imaging area of ​​the imaging unit 11 of the terminal device 10, the captured image data is output to the display unit 12 and the entering or leaving person is displayed. Then, when the face of the entering or leaving person facing forward can be accommodated in the facial image display area 12a of the display unit 12 at a predetermined size, for example, the facial image of the entering or leaving person is captured by the imaging unit 11 (step S1).

[0051] Next, based on the captured facial image data of the persons entering and leaving, an image search is performed from the facial image data 17 recorded in the memory unit 14, and facial authentication is performed using the facial authentication function in the authentication unit 13a to determine whether the facial image data of the persons entering and leaving is the registered facial image data 17 (step S2).

[0052] In this face authentication, for example, as shown in FIG. 7, an authentication screen is displayed on the display screen 12A of the display unit 12. On the authentication screen, a face image 17a showing the face of the person entering or leaving the venue facing forward is displayed in a state of being inverted from left to right in the face image display area 12a. If the face authentication is successful, for example, the name of the authenticated registered person 12e is displayed in a variable display area 12d between the face image display area 12a and the date display area 12b and the time display area 12c. By displaying the name of the registered person 12e on the display screen 12A in this way, the person entering or leaving the venue can visually confirm that the procedure for entering or leaving the venue has progressed and that authentication has been completed. By displaying the face image 17a in a state of being inverted from left to right, a gesture to be described later and a display image can be displayed in a visually recognizable manner in a seamless link.

[0053] In this manner, the face authentication by the face authentication function of the authentication unit 13a determines whether the captured entering / exiting person is a registered person or not (step S3), and if it is determined based on the face authentication result that the person is a registered person (Yes in step S3), the process proceeds to the next step S4. On the other hand, if it is determined that the person is not a registered person (unregistered person) (No in step S3), face image data 17 representing the captured face image and time data 18 representing the time are stored in the storage unit 14, and it is determined whether or not to end the entrance / exit management process (step S15). If it is determined that the process is not to be ended (No in step S15), the process proceeds to the above step S1 and the subsequent processes are repeated, but if it is determined that the process is to be ended (Yes in step S15), the series of entrance / exit management processes according to this flowchart is ended.

[0054] In step S4 after it is determined that the person is a registered person as described above, an entry / exit record screen such as that shown in FIG. 8 is displayed on the display screen 12A of the display unit 12 of the terminal device 10 (step S4). Unlike the authentication screen described above, this entry / exit record screen displays arrow icons 21 and 22 representing the exit direction and entry direction for entry / exit determination in the variable display area 12d. Note that the arrow icons 21 and 22 are also superimposed with text information representing "exit" and "entry" so that they can be more clearly seen. The arrow icons 21 and 22 here are one example of a mode of representing the exit direction and entry direction, and other display methods may be adopted as long as they are capable of representing these directions.

[0055] With the entrance / exit record screen thus displayed, the person entering / exiting makes a movement (gesture) expressing the intention of entering or exiting. Specifically, with the entrance / exit record screen displayed as shown in FIG. 8, the person entering / exiting makes a gesture by turning his / her face left or right with respect to the front of the terminal device 10. The gesture of the person entering / exiting is detected, for example, by a motion detection function mounted on the imaging unit 11. When the person entering / exiting turns his / her face to the right, as shown in FIG. 9, the face image 17a displayed in the face image display area 12a is also displayed in a left-right inversion so as to face the right direction as indicated by the white arrow in the figure. Also, when the person entering / exiting turns his / her face to the left, as shown in FIG. 10, the face image 17a displayed in the face image display area 12a is also displayed in a left-right inversion so as to face the left direction as indicated by the white arrow in the figure.

[0056] In the entrance / exit record screen as shown in FIG. 8, the arrow icon 22 representing "enter" points to the right, and the arrow icon 21 representing "exit" points to the left, so that the actual gestures of the entering / exiting person and the movement of the displayed face image 17a are displayed to be linked. The entrance / exit of the entering / exiting person is determined by such gestures (step S5). Note that the face image 17a displayed in the face image display area 12a is displayed in a left-right inverted manner, so that it becomes easier to visually determine whether the entering / exiting person should face left or right to be determined as entering or exiting. However, if the face image 17a displayed in the face image display area 12a is not displayed in a left-right inverted manner, the actual gestures and the movement of the displayed face image are opposite, which is not intuitively easy to visually recognize, but it is also possible to perform the entrance / exit determination in such a manner.

[0057] In the determination result of the entry / exit determination (step S5), if the entering / exiting person makes an entry gesture and is determined to have entered (entry in step S5), it is determined whether or not the entry is the first entry for the entering / exiting person on the determination day (step S6). On the other hand, in the determination result of the entry / exit determination (step S5), if the entering / exiting person makes an exit gesture and is determined to have exited (exited) (exit in step S5), it is determined to have exited, and the time and date are temporarily stored and recorded as an exit record in the entry / exit record in the record of the entering / exiting person in the management database 20 of the storage unit 14 (step S9). Note that the exit record is recorded with a number assigned according to the number of exits, for example, as described above. The determination result of the entry / exit determination in step S5 may be notified to the entering / exiting person by the output device 40 with a sound or light indicating "entry" or "exit".

[0058] The determination in step S6 as to whether or not it is the first entry on the determination day is made by, for example, judging whether it is attendance or re-entry based on whether or not it is the first (first) entry record on that day (determination day) from the history of entry records of time data 18 stored in memory unit 14 in association with the entering and leaving persons. If it is determined to be the first (first) entry on that day (Yes in step S6), it is determined to be attendance, and the time and date are temporarily saved and recorded as attendance records in the entry and exit record in the record of entering and leaving persons in management database 20 of memory unit 14 (step S7).

[0059] On the other hand, if it is determined that the entry is not the first time that day (second or subsequent time) (No in step S6), it is determined to be a re-entry, and the time and date are temporarily saved and recorded as a re-entry record in the entry / exit record in the record of the entry / exit person in the management database 20 of the storage unit 14 (step S8). Note that the re-entry record is recorded with a number assigned according to the number of re-entries, for example as described above.

[0060] After the exit record is made in step S9, it is determined whether or not the exit is the last (final) exit of the entry / exit person on the judgment day (step S10). The determination of whether or not the exit is the last (last) exit record of that day (judgment day) is made based on, for example, the history of the exit records of the time data 18 stored in the memory unit 14 in association with the entry / exit person, and it is determined whether the exit is a clock-out or an exit depending on whether or not the exit is the last (final) exit record of that day (judgment day). If it is determined to be the last (final) exit of that day (Yes in step S10), it is determined to be a clock-out, and the time and date are temporarily saved and stored as an exit record in the entry / exit record in the record of the entry / exit person in the management database 20 of the memory unit 14 (step S11).

[0061] On the other hand, if it is determined that the exit is not the last (final) exit of the day (No in step S10) or after steps S7 and S8, a save confirmation screen as shown in FIG. 11 is displayed on the display screen 12A of the display unit 12 of the terminal device 10 (step S12). Unlike the entrance / exit record screen, this save confirmation screen displays arrow icons 23 and 24 representing the end direction and the save direction for the save end determination in the variable display area 12d. Note that the arrow icons 23 and 24 are also superimposed with text information representing "end" and "save" so that they can be more clearly seen. By visually checking the displayed save confirmation screen, the entering / exiting person can check the entry or exit record contents temporarily stored in the memory unit 14 as described above, and can also confirm that the procedure regarding the entrance / exit has progressed further and the entrance / exit record has been performed. Note that the arrow icons 23 and 24 are one example of a mode of representing the end direction and the save direction, and other display methods may be adopted as long as they can represent these directions.

[0062] With the save confirmation screen thus displayed, the person entering or leaving makes a gesture indicating his / her intention to save or end. The gesture here can be the same as the operation described using the entry / exit record screen at the time of the entry / exit determination in step S5 above. That is, on the save confirmation screen, arrow icons 23, 24 indicating the end direction and the save direction are displayed in the variable display area 12d, so that the person entering or leaving can know that a save or end command can be input to the terminal device 10 by facing in either direction.

[0063] The end of saving is determined based on the gesture of the person entering or leaving while the save confirmation screen is displayed (step S13). For example, when the motion detection function mounted on the imaging unit 11 is used to determine that the person entering or leaving has turned their face to the right and is to be saved (save in step S13), the entry / exit record of the entry or exit temporarily saved in association with the person entering or leaving is saved in the management database 20 of the storage unit 14 in association with the time data 18, and the entry / exit record is ended.

[0064] On the other hand, if the entering / exiting person turns his / her face to the left and it is determined that the process is finished (the end of step S13), the data temporarily stored in association with the entering / exiting person is, for example, temporarily discarded, and the entry / exit recording is finished. In this way, by displaying a save confirmation screen after the entry / exit recording screen for confirmation, even if the entering / exiting person mistakenly makes an opposite gesture when making an "enter" or "exit" gesture, for example, by making an "end" gesture, it is possible to prevent various data such as incorrect date and time from being recorded as an entry / exit record in the management database 20. Note that when the entry / exit recording is finished, it is determined that the entry / exit management process is finished as described above (step S15).

[0065] According to the entrance / exit management system 1 configured in this way, face authentication is performed based on the facial image data of the person entering or leaving the facility captured by the imaging unit 11 of the terminal device 10, and the entrance / exit is determined based on the orientation of the facial image (the orientation associated with the gesture) to create an entrance / exit record, so that it is possible to perform entrance / exit management without contact, as with the above-mentioned effects. Also, since the entrance / exit management can be performed by the terminal device 10 alone, there is no need to increase the number of components.

[0066] In the above-mentioned entry / exit management process, entry / exit determinations, etc. are made based on the movements (gestures) of persons entering and exiting, but this is not limited to this, and entry / exit determinations, etc. may be made using the position, orientation and movement of persons entering and exiting or objects accompanying them, or voice input or biometric information of persons entering and exiting, either alone or in combination, in addition to or instead of gestures, as described above.

[0067] In addition, as a modified example of the entrance / exit management system 1 or the terminal device 10 capable of independently executing the entrance / exit management process, when performing entrance / exit judgment etc. based on the gestures of the entering / exiting person, it is also possible to configure it so that the natural movement along the flow line of the entering / exiting person is recognized as a gesture and various judgments etc. can be performed without displaying information prompting the gesture such as the arrow icons 21, 22 in the variable display area 12d on the display screen 12A. Specifically, this applies to a place where the terminal device 10 where the entrance / exit management process as described above is installed, for example, an entrance door is arranged on the right side and an exit gate is arranged on the left side. In such a place of business, if the entering / exiting person starts moving to the right after the authentication screen is displayed and face authentication is successful, the terminal device 10 can recognize that the entering / exiting person has turned right, and if the entering / exiting person starts moving to the left, it can recognize that the entering / exiting person has turned left. Therefore, it is also possible to execute various judgments, input of control commands, etc. by determining the natural movement of the entering / exiting person as an input by a gesture without displaying the entrance / exit record screen with the arrow icons 21, 22 etc. after face authentication.

[0068] Although several embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the gist of the invention. These embodiments and their modifications are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0069] 1. Entrance / Exit Control System 10 Terminal Equipment 11 Imaging unit 12 Display section 13 Control section 13a Authentication Section 13b Judgment part 13c Management Department 13d input / output I / F 13e Image processing section 14 Storage section 15 Recording media 17 Facial image data 18 Time Data 19 Entry and exit history data 20 Management Database 30 Input Devices 40 Output Device 50 Detection unit 60 External equipment

Claims

1. An imaging device that captures images of people entering and leaving the venue to obtain image data; A storage device that stores face image data of the persons entering and leaving the venue; a control device that manages the entry and exit of the person entering and exiting the facility based on image data of the person entering and exiting the facility captured by the imaging device, The control device includes: an authentication means for performing face authentication by comparing face image data included in the image data of the person entering or leaving the venue captured by the imaging device with face image data previously stored in the storage device; a determination means for detecting at least one of a position, a direction, and a movement of the person or an object accompanying the person, which is included in image data from the imaging device of the person whose face has been authenticated by the authentication means, and determining whether the person is entering or leaving based on the detection result; and a management means for storing the determination result by the determination means together with time information in the storage device as entry / exit history information in association with the persons entering and leaving the facility. Entrance / exit control system.

2. The determination means of the control device detects the orientation of a face image of the person entering or leaving the venue, which is included in the image data from the imaging device, and determines whether the person entering or leaving the venue is entering or leaving based on the orientation of the detected face image. The entrance / exit management system according to claim 1.

3. a display device for displaying the facial images of the persons entering and leaving the venue captured by the imaging device; The entrance / exit management system according to claim 1 or 2.

4. The control device includes: an input means for receiving voice input of the person entering or leaving the venue; A detection means for detecting biological information of the person entering and leaving the facility, The determination means determines whether the person is entering or leaving based on at least one of the voice input result by the input means and the detection result by the detection means. The entrance / exit management system according to any one of claims 1 to 3.

5. the control device has an output means for outputting a control signal for controlling an external device based on at least one of a face authentication result by the authentication means and the determination result; The output means outputs the entrance / exit history information to an external entrance / exit management device. The entrance / exit management system according to any one of claims 1 to 4.

6. The control device has an image processing means for performing image processing on image data of the person entering or leaving the venue captured by the imaging device so that the image data becomes an image that can be used for face authentication by the authentication means. The entrance / exit management system according to any one of claims 1 to 5.

7. An imaging means for imaging people entering and leaving the venue to obtain image data; A storage means for storing face image data of the persons entering and leaving the venue; an authentication means for performing face authentication by comparing face image data included in the image data of the person entering or leaving the venue captured by the imaging means with face image data previously stored in the storage means; a determination means for detecting at least one of a position, a direction, and a movement of the person entering or leaving the facility or an object accompanying the person, which is included in image data from the imaging means of the person entering or leaving the facility whose face has been authenticated by the authentication means, and determining whether the person entering or leaving the facility is entering or leaving based on the detection result; a management means for storing the determination result by the determination means together with time information in the storage means as entry / exit history information in association with the entering / exiting persons. Access control device.

8. The determination means detects the orientation of a facial image of the person entering or leaving the venue, which is included in the image data from the imaging means, and determines whether the person entering or leaving the venue is entering or leaving based on the orientation of the detected facial image. The entrance / exit management device according to claim 7.

9. A display means for displaying the facial images of the persons entering and leaving the venue captured by the imaging means is provided.

9. The entrance / exit management device according to claim 7 or 8.

10. an input means for receiving voice input of the person entering or leaving the venue; A detection means for detecting biological information of the person entering and leaving the facility, The determination means determines whether the person is entering or leaving based on at least one of the voice input result by the input means and the detection result by the detection means. The entrance / exit management device according to any one of claims 7 to 9.

11. an output unit that outputs a control signal for controlling an external device based on at least one of a face authentication result by the authentication unit and the determination result; The output means outputs the entrance / exit history information to an external entrance / exit management device. The entrance / exit management device according to any one of claims 7 to 10.

12. and an image processing means for processing image data of the person entering or leaving the venue captured by the imaging means so that the image data can be used for face authentication by the authentication means. The entrance / exit management device according to any one of claims 7 to 11.

Images