Information processing device and information processing method

Abstract

To provide a technology for specifying an event to watch out for based on both an entry / exit record and a face authentication record.SOLUTION: A management system 10 includes: acquisition means 12 for acquiring a record of a person who has entered / exited a predetermined region from an entry / exit recording device 20; acquisition means 13 for acquiring, from face authentication means 30 for performing face authentication for a person appearing in a video captured by photographing means 31 having a predetermined positional relationship with the entry / exit recording device 20, identification information on the person appearing in the video; checking means 14 for checking the record acquired by the acquisition means 12 against the identification information acquired by the acquisition means 13; and output means 15 for outputting, in a case where the person indicated in the identification information satisfies a predetermined condition in the record, information on a time when the condition is satisfied.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for managing entrance / exit records and face authentication records. [Background technology]

[0002] There are known entrance / exit recording devices or entrance / exit management devices, and face authentication systems. For example, Patent Document 1 describes a video surveillance system that unlocks an electric lock 2 when a person is authenticated as a genuine person when passing through a security gate 3, and stores and manages personal attribute information of the person as meta information of a video captured by a surveillance camera 7. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 4751442 Summary of the Invention [Problem to be solved by the invention]

[0004] The system described in Patent Document 1 simply adds meta information of genuine people entering and exiting a facility to video, and is unable to handle other events.

[0005] In response to this, the present invention provides a technique for identifying events requiring attention based on both entrance / exit records and face recognition records. [Means for solving the problem]

[0006] One aspect of the present disclosure provides an information processing device having a first acquisition means for acquiring a record of a person entering and exiting a specified area from an entrance / exit recording device, a second acquisition means for acquiring identification information of a person appearing in an image captured by an image capturing means that is in a specified positional relationship with the entrance / exit recording device from a face recognition means that performs face recognition on the person appearing in the image, a comparison means for comparing the record acquired by the first acquisition means with the identification information acquired by the second acquisition means, and an output means for outputting information regarding the time when a specified condition was satisfied in the record if the person indicated in the identification information satisfies the specified condition.

[0007] The information processing device may have a database in which users who use the entrance / exit recording device are registered, and the predetermined condition may include a condition that the person is a user registered in the database.

[0008] The information processing device may have a database in which users who use the entrance / exit recording device are registered, and the predetermined condition may include a condition that the person is a user who is not registered in the database.

[0009] The predetermined condition may include a condition that the person is registered in the face authentication means.

[0010] The predetermined condition may include a condition that the person is not registered in the face authentication means.

[0011] The image capturing means may capture an image of at least a part of an interior of the area, and the predetermined condition may include a condition that the record indicates that the person has left the area.

[0012] The image capturing means may capture an image of an area different from the area, and the predetermined condition may include a condition that the record indicates that the person has left the area.

[0013] The record may include information defining conditions for permitting or prohibiting entry into the area, and the predetermined conditions may include a condition that the face recognition means detects that the person was in the area at a time when entry into the area is not permitted.

[0014] Another aspect of the present disclosure provides an information processing method comprising the steps of acquiring a record of a person entering and exiting a specified area from an entrance / exit recording device, acquiring identification information of a person appearing in an image captured by an image capturing means that is in a specified positional relationship with the entrance / exit recording device from a face recognition means that performs face recognition of the person appearing in the image, comparing the record acquired by the first acquisition means with the identification information acquired by the second acquisition means, and, if the person indicated in the identification information satisfies a specified condition in the record, outputting information regarding the time when the condition was satisfied. Effect of the Invention

[0015] According to the present invention, events requiring attention can be identified based on both the entrance / exit records and the face authentication records. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram showing an overview of a security system 1000 according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of a hardware configuration of a management system 1. [Diagram 3] FIG. 1 shows an example of the layout of a security system 1000. [Figure 4] 6 is a sequence chart illustrating an operation relating to entry / exit recording. [Diagram 5] FIG. 2 is a diagram illustrating the contents of a database 241. [Figure 6] FIG. 2 is a diagram illustrating the contents of a database 242. [Figure 7] FIG. 2 is a diagram illustrating an example of data recorded in a database 113. [Figure 8] 4 is a sequence chart illustrating an operation relating to face authentication. [Figure 9] FIG. 3 is a diagram illustrating an example of data recorded in a database 321. [Figure 10] 4 is a flowchart showing the operation of the management system 1. [Figure 11] FIG. 2 is a diagram illustrating the contents of a database 114. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] 1. Configuration 1 is a diagram showing an overview of a security system 1000 according to an embodiment. The security system 1000 is a system that manages entrance / exit records and face authentication records in an integrated manner. The security system 1000 includes an entrance / exit recorder 2, a face authentication system 3, and a management system 1.

[0018] The entrance / exit recording device is a device that records the entrance and exit of a person to a predetermined area. The entrance / exit recording device 2 records the entrance and exit of a person to the area A (one example of a predetermined area). The predetermined area refers to a defined physical area, for example, a building, a room, or a means of transportation (elevator, etc.). The area A is provided with an entrance / exit restriction means 21, an authentication means 22, a communication means 23, a storage means 24, and a processing means 25. The entrance / exit restriction means 21 is a technical means for restricting entry and exit to the area A, for example, a door or gate device with an electric lock installed at the boundary between the area A and another area. The authentication means 22 is a technical means for authenticating a person who is attempting to enter or exit the area A. The authentication in the authentication means 22 may be any type, such as a recording medium (for example, a magnetic card, an IC card, or a card with a QR code (registered trademark) printed thereon), a biometric authentication (for example, face authentication, fingerprint authentication, vein authentication, or iris authentication), or a combination of these. When a person attempting to enter or exit area A is authenticated by the authentication means 22, the entry / exit restriction means 21 permits the person to enter / exit area A (specifically, for example, unlocks the electric lock). The communication means 23 transmits the entry / exit record to the management system 1. The entry / exit record is a record of entry and exit to area A, and includes, for example, identification information of area A, whether the person entered or exited the area, identification information of the person who entered and exited, and the time of entry and exit. The storage means 24 stores various data. Note that in this example, only one entry / exit recording device 2 is illustrated, but the security system 1000 may have multiple entry / exit recording devices 2 each corresponding to a different area (for example, installed in a different room).

[0019] The face authentication means is a system that identifies a person from the face of the person photographed by the photographing means. The face authentication system 3 (an example of a face authentication means) identifies a person from the face of the person photographed by the photographing means 31. The face authentication system 3 has the photographing means 31, the storage means 32, the authentication means 33, and the communication means 34. The photographing means 31 is a technical means for photographing an image, such as a camera. The photographing means 31 is installed in a predetermined positional relationship with the area A. In one example, the photographing means 31 photographs at least a part of the area A (for example, the inside of the area A). The photographing means 31 transmits the photographed image to the authentication means 33. The storage means 32 stores a database in which the facial features of multiple people are recorded. The authentication means 33 compares the features extracted from the image photographed by the photographing means 31 with the features recorded in the database, and identifies the person appearing in the image photographed by the photographing means 31. The communication means 34 transmits the identification information of the identified person to the management system 1. In one example, the image capturing means 31 is a surveillance camera or a security camera, and the storage means 32, authentication means 33, and communication means 34 are implemented in a server. The surveillance camera or security camera has a communication device (not shown) and communicates with the server via a network.

[0020] Although a single image capturing means 31 is illustrated here, the face recognition system 3 may include a plurality of image capturing means 31 arranged at different positions. Note that the image capturing means 31 will be referred to as a camera 31 hereinafter.

[0021] The management system 1 manages the entrance / exit record and the face authentication record in an integrated manner. The management system 1 includes a storage means 11, an acquisition means 12, an acquisition means 13, a comparison means 14, an output means 15, and a processing means 16. The storage means 11 stores various data and programs. The acquisition means 12 acquires the entrance / exit record of a person in the area A from the entrance / exit record device 2. The acquisition means 13 acquires the identification information of a person appearing in the image captured by the image capture means 31 from the face authentication system 3. The comparison means 14 compares the entrance / exit record acquired by the acquisition means 12 with the identification information acquired by the acquisition means 13. If the person indicated by the identification information satisfies a predetermined condition in the entrance / exit record, the output means 15 outputs information indicating the time when the condition was satisfied in the image. The processing means 16 executes various processes.

[0022] In one example, the storage unit 11, the acquisition unit 12, the acquisition unit 13, the collation unit 14, and the output unit 15 are implemented in a server in the management system 1. The management system 1 may have a client (not shown). The client accesses the server to acquire information.

[0023] 2 is a diagram illustrating a hardware configuration of the management system 1. The management system 1 is an information processing device or computer having a CPU (Central Processing Unit) 101, a memory 102, a storage 103, and a communication IF (Interface) 104. The CPU 101 is a processing device that performs various processes according to a program. The memory 102 is a main storage device that functions as a work area when the CPU 101 executes processes, and includes, for example, a RAM (Random Access Memory). The storage 103 is an auxiliary storage device that stores various data and programs, and includes, for example, an SSD (Solid State Drive) and / or an HDD (Hard Disk Drive). The communication IF 104 is a communication device that communicates with other devices according to a predetermined communication standard (for example, Ethernet (registered trademark)), and includes, for example, a NIC (Network Interface Card).

[0024] In this example, the programs stored in the storage 103 include a program (hereinafter referred to as a "server program") for causing a computer to function as a management system 1 in the security system 1000. When the CPU 101 is executing the server program, at least one of the memory 102 and the storage 103 is an example of a storage means 11. The CPU 101 is an example of an acquisition means 12, an acquisition means 13, a comparison means 14, and a processing means 16. The communication IF 104 is an example of an output means 15.

[0025] In this document, the term "system" encompasses both a single device and a collection of multiple devices.

[0026] 2.Operation The operation of security system 1000 will be described below. Prior to the description of the operation, an example layout of security system 1000 will be described. The following description of the operation will be given using this example layout. In the following description, functional elements such as storage means 11 may be described as the subject of processing, which means that hardware elements such as CPU 101 execute processing in cooperation with other hardware elements according to a program.

[0027] FIG. 3 is a diagram showing an example of the arrangement of a security system 1000. In this example, the security system 1000 is installed in an office O of a company. The entrance / exit recording device 2, the face authentication system 3, and the management system 1 are managed and operated by different businesses, and each operates independently. An electric lock L (an example of the entrance / exit restriction means 21) is installed on the door D at the entrance / exit of the office O, and a card reader R (an example of the authentication means 22) is installed on the outside and inside of the door D. When distinguishing between these two card readers, the card reader installed on the outside of the door D is called a card reader Ro, and the card reader installed on the inside is called a card reader Ri. A camera 31 is installed in the space inside the door D, and captures the face of a person entering the office O through the door D.

[0028] 2-1. Entry and exit records FIG. 4 is a sequence chart illustrating an operation related to entry and exit records. A card key C (IC card) is distributed to a person who is permitted to enter and leave the office O. When the card key C is held over the card reader R, the card reader R reads the identification information of the card (or user) from the card key C (step S101). The card reader R is connected to the computer 200 (an example of the communication means 23) by wire or wirelessly. The card reader R outputs an authentication request including the read identification information to the computer 200 (step S102). This authentication request includes the identification information of the card reader R (or the door D) and the identification information of the card key C (or the user). The identification information of the card key C (or the user) includes user attribute information, for example, information indicating a position in a company. When the authentication request is received from the card reader R, the processing means 25 of the computer 200 judges whether or not to authenticate this authentication request (step S103). In the computer 200, the storage means 24 stores a database 241 in which conditions for determining whether or not an authentication request from the card reader R should be authenticated, and a database 242 relating to registered users. The processing means 25 refers to the database 241 and the database 242 to determine whether or not this authentication request should be authenticated.

[0029] FIG. 5 is a diagram illustrating the contents of the database 241. The database 241 includes a plurality of records. Each record includes information for identifying the door (or electronic lock), permission or prohibition, time period, and target user. For example, the top record in FIG. 5 indicates that for the door "Main Entrance," "all registered users" are permitted to enter the building during "weekdays 8:00-20:00." The second record from the top in this diagram indicates that for the door "Main Entrance," "all users" are prohibited from entering the building during "all days 24:00-05:00." The third record from the top in this diagram indicates that "users with a title of manager or higher" are permitted to enter the building during "weekdays 20:00-24:00."

[0030] FIG. 6 is a diagram illustrating an example of data recorded in database 242. Database 242 includes multiple records. Each record includes a user's identification information and user attribute information. The user's attribute information includes the name of the department to which the user belongs, the position, and the work shift. The top record in FIG. 6 indicates that a user with the name "Iwata Shogo", the department to which the user belongs, the position "Section Manager", and the work shift "Regular" has been registered.

[0031] An example of the judgment in step S103 will be described. Consider an example in which the authentication request is for authentication of a person named "Shogo Iwata" to "enter" the building from the "main entrance" at 10:06:54 on March 8, 2021. With reference to database 241, it is recorded that "all registered users" are permitted to enter the building during "weekdays 8:00-20:00." Furthermore, with reference to database 242, a user named "Shogo Iwata" is registered. Therefore, in this case, processing means 25 authenticates this authentication request.

[0032] 4 again. If the authentication result is "authenticated", the processing means 25 outputs an unlocking signal to the entrance / exit restriction means 21 (in this example, the electric lock L of the door D) corresponding to the authentication request (step S104). The electric lock L and the computer 200 are connected by wire or wirelessly. The authentication request includes identification information of the card reader R (or the door D), i.e., identification information of the authentication means 22, and the processing means 25 identifies the corresponding entrance / exit restriction means 21 from this identification information. The correspondence between the authentication means 22 and the entrance / exit restriction means 21 is defined, for example, in a database (not shown) stored in the storage means 24. If the authentication result is "not permitted", the processing means 25 notifies the card reader R, which is the sender of the authentication request, of that effect. The card reader R notifies the user of that effect (for example, by outputting a beep, turning on an error lamp, displaying a message on a display, etc.).

[0033] When the electric lock L receives an unlocking signal from the computer 200, it is unlocked for a predetermined time (step S105). The electric lock L notifies the computer 200 that it has been unlocked (step S106). Upon receiving this notification, the computer 200 generates an entry / exit record (step S107). The entry / exit record includes, for example, the time (i.e., time stamp), the identification information of the door (or electric lock), whether the entry / exit occurred, and the identification information of the person who entered or exited. The computer 200 writes the generated entry / exit record in a database (not shown) in the storage means 24. Furthermore, the computer 200 transmits the generated entry / exit record to the management system 1 (step S108). The storage means 11 of the management system 1 records the received entry / exit record in the database 241 (step S109). The database 113 is a database that accumulates the entry / exit records.

[0034] FIG. 7 is a diagram illustrating an example of data recorded in database 113. Database 113 includes multiple records. Each record includes the time, the identification information of the door (or electronic lock), whether entry or exit occurred, and the identification information of the person who entered or exited. For example, the record in the top row of FIG. 5 indicates that a person named "Iwata Shogo" "entered" through the "main entrance" at 10:06:54 on March 8, 2021. If office O has multiple entrances and a card reader is installed at each entrance and exit, the entry and exit records output from these multiple card readers are aggregated in database 113.

[0035] 2-2.Facial recognition FIG. 8 is a sequence chart illustrating an operation related to face authentication. The camera 31 can communicate with the server 300 (an example of a server in which the storage means 32, the authentication means 33, and the communication means 34 are implemented). In this example, the camera 31 operates all the time (24 hours a day, 365 days a year). That is, the camera 31 outputs video data to the server 300 all the time (step S201). The authentication means 33 of the server 300 performs face recognition processing on image data of a frame obtained from the video data (step S202). A well-known technology can be used for the face recognition processing. The face authentication technology is, for example, a process of identifying an area in a frame in which a person's face is captured, extracting a plurality of feature points from the face captured in the frame, and identifying who the person captured in the frame is by collating the positional relationship of the plurality of feature points with data recorded in a database (not shown). This database records data indicating the identification information (e.g., name) of the person and the positional relationship of the feature points. Note that the person registered in this database does not have to be the same as the user registered in the database 242. For example, there may be a user who is registered in the database of the face authentication means 3 but not in the database 242, and there may be a user who is registered in the database 242 but not in the database of the face authentication means 3.

[0036] The authentication means 33 writes the result of the face authentication process in the database 321 (step S203). If data of a person matching the feature points extracted from the frame is recorded in the database, the authentication means 33 writes the identification information of the person (e.g., the name of the person) in the database 321. If data of a person matching the feature points extracted from the frame is not recorded in the database, the authentication means 33 writes in the database 321 a message that an unregistered person has been detected.

[0037] FIG. 9 is a diagram illustrating an example of data recorded in database 321. Database 321 includes multiple records. Each record includes a time, camera identification information, and person identification information. For example, the top record in FIG. 9 indicates that a person named "Iwata Shogo" was detected at the "main entrance" on March 8, 2021 at 10:07:10. The second record from the top in this diagram indicates that an unregistered person was detected at the "main entrance" on March 8, 2021 at 10:15:20.

[0038] 2-3. Matching entrance and exit records with facial recognition data 10 is a flowchart showing the operation of the management system 1. In step S301, the acquisition means 12 acquires the entrance / exit records. In one example, the acquisition means 12 acquires the entrance / exit records periodically (for example, at 0:00 every day) from the database 113. The entrance / exit records acquired at this time are the entrance / exit records accumulated in the database 113 since the previous acquisition (for example, at 0:00 of the previous day).

[0039] In step S302, the acquisition means 13 acquires a face recognition record. In one example, the acquisition means 13 periodically (for example, at 0:00 every day) acquires a face recognition record from the face recognition system 3 (more specifically, from the database 321). The face recognition record acquired at this time is data stored in the database 321 after the previous acquisition (for example, at 0:00 every day). Note that the timing of acquiring the face recognition record does not need to be synchronized with the timing of acquiring the entry / exit record, and the two may be independent.

[0040] In step S303, the matching means 14 matches the entrance / exit record acquired by the acquisition means 12 with the face authentication record acquired by the acquisition means 13. Here, "matching the entrance / exit record with the face authentication data" means comparing the two and extracting data that requires attention (hereinafter referred to as "attention data"). In this example, the storage means 11 extracts the attention data by referring to the database 114. The database 114 is a database that defines the conditions for the attention data.

[0041] FIG. 11 is a diagram illustrating the contents of the database 114. The database 114 includes a plurality of records. Each record includes a condition in the face recognition record, a condition in the entry / exit record, and a corresponding process. For example, the top record in FIG. 11 is extracted as caution data when "a registered user is detected" in the face recognition record and when "there is no entry record of the person in the 8 hours before and after the time when the person is detected by face recognition" in the entry / exit record. This condition corresponds to, for example, an event in which a registered user enters without using his / her own card key (more specifically, entering the building using someone else's card key, entering the building together with someone else entering the building, etc.). In addition, the second record from the top in FIG. 11 is extracted as caution data when "an unregistered person is detected" in the face recognition record and when "there is no entry record of the registered user in the 5 minutes before and after the time when the person is detected by face recognition" in the entry / exit record. This condition corresponds to, for example, an event in which an unregistered person (e.g., a person from outside the company) enters the building without being accompanied by a registered user (e.g., an employee).

[0042] Referring again to FIG. 10, in step S304, the output means 15 executes a process corresponding to the attention data. In the database 114, a process corresponding to the condition of the attention data is defined. In both the topmost record and the second record from the top in FIG. 11, a process of "notifying the face authentication system of the time of the attention data" is defined. Therefore, the output means 15 outputs the time of the attention data to the face authentication system 3.

[0043] The face authentication system 3 records the video data as a log, and when a problematic event occurs, the video data log is referred to. Since it is impossible to visually check the log of a huge amount of video data, the face authentication system 3 embeds data of a mark (hereinafter referred to as a "trigger point") that serves as a search criterion in the video data. The face authentication system 3 has its own data that defines the conditions for generating a trigger point. The condition for generating a trigger point is, for example, "an unregistered person is detected." However, in an environment with many visitors from outside the company, trigger points are generated frequently, and they become essentially meaningless as trigger points. Alternatively, in this example, a trigger point is not generated for a registered user. However, if the user is a registered user but does not match the entry / exit record, or does not match information defined in relation to the entry / exit record (for example, work shift), there is a possibility that a problematic event has occurred. In this case, it is preferable that a trigger point is generated, but it is difficult for the face authentication system 3 alone to respond to such an event.

[0044] In this embodiment, by generating a trigger point in the video data at the time of the warning data output from the management system 1, the trigger point can be generated by comprehensively taking into consideration the entry / exit records and the facial recognition records.

[0045] The output means 15 may output attributes of the attention data (for example, information specifying the condition that was satisfied when the record was extracted as attention data) in addition to the time of the attention data to the face recognition system 3. By outputting additional information in this way, it is possible to improve the searchability of the trigger point in the face recognition system 3.

[0046] In step S305, the storage means 11 writes the entrance / exit record acquired from the acquisition means 12 and the face authentication record acquired from the acquisition means 13 to the database 115. The database 115 is a database that manages the entrance / exit record and the face authentication record in an integrated manner. In addition to or instead of the processing defined in the database 114, the output means 15 may output warning data to a terminal of a defined administrator in response to a request from the administrator. At this time, the output means 15 may output a list of records recorded in the database 115. Furthermore, at this time, the output means 15 may display the warning data in an appearance different from other data (for example, in a different color, size, or font).

[0047] 3. Other embodiments The present invention is not limited to the above-described embodiment, and various modifications are possible. Some modifications will be described below. Two or more of the features described in the following modifications may be applied in combination.

[0048] 3-1. Positional relationship between the photographing means 31 and the entrance / exit recording device 2 The positional relationship between the photographing means 31 and the entrance / exit recording device 2 is not limited to that exemplified in the embodiment. That is, the photographing means 31 is not limited to photographing an area where entrance / exit is restricted by the entrance / exit restricting means 21 of the entrance / exit recording device 2. The photographing means 31 may photograph an area other than the area where entrance / exit is restricted by the entrance / exit restricting means 21. For example, the entrance / exit restricting means 21 may be installed at the front entrance of an office, and the photographing means 31 may be installed in a position where it photographs a warehouse away from the office. In this way, by using the photographing means 31 that photographs an area other than the area where entrance / exit is restricted by the entrance / exit restricting means 21, it is possible to deal with a wider variety of events.

[0049] 3-2. Conditions for Caution Data The conditions for the attention data are not limited to those exemplified in the embodiment. For example, data that satisfies any of the following conditions (1) to (7) in the face recognition record may be extracted as the attention data. (1) When a registered user is detected in the facial recognition record, there is an exit record in the entry / exit record within a predetermined time (e.g., 30 minutes) before the time when the person was detected in the facial recognition record. This is a situation where a person is captured on the image in a specific area even though he or she should have already left the office according to the entry / exit record. This is particularly useful when the area captured by the image capture means 31 is different from the area to which the entry / exit restriction means 21 restricts entry / exit, as shown in 3-1. This corresponds to, for example, an incident where an employee who should have left the office is doing something in the warehouse.

[0050] (2) When a registered user is detected in the facial recognition record, and the time when the person is detected in the facial recognition is outside the time period defined for that person in the entry / exit record. In this example, the entry / exit record includes information that identifies the time period defined for the person. The "time period defined for the person" is, for example, a work shift if the person is an employee. This corresponds to an event in which the person is detected as being in the office outside of the work shift. For example, if the work shift for person "Shogo Iwata" is defined as "weekdays 9:00-18:00," when Shogo Iwata is detected in the face recognition record at 10:00 on a holiday, the matching means 14 extracts this record as caution data (note that in this example, the entry / exit recorder 2 also permits entry / exit outside the hours defined by the work shift).

[0051] (3) When an unregistered user is detected in the facial recognition record, and the ratio of the total number Nu of unregistered users detected in the facial recognition record within a specified time (for example, 10 minutes before or after) before and after the time when the unregistered user was detected to the total number Nk of registered users whose entry and exit are recorded at that time in the entry and exit record satisfies a specified condition. The predetermined condition is, for example, that the ratio of Nu to Nk is greater than a threshold value. This corresponds to, for example, an event in which so many visitors gather at once that the registered users (employees) cannot guide them all.

[0052] (4) When an unregistered user is detected in the facial recognition record, and the total number Nu1 of unregistered users detected in the facial recognition record within a specified time (for example, 10 minutes before or after) before or after the time when the unregistered user was detected does not match the total number Nu2 of unregistered users whose entry and exit are recorded at that time in the entry and exit record. For example, this corresponds to an event in which an unregistered user (guest) enters and exits without using a card key (for example, together with another person) when the unregistered user should enter and exit using a card key for guests.

[0053] (5) When a registered user is detected in the entry / exit record, and the person whose entry / exit is recorded in the facial recognition record does not match the person whose entry / exit is recorded within a specified time period (for example, 5 minutes before or after) when the registered user is detected in the entry / exit record. This corresponds to an event in which a medium used for authentication in the entrance / exit recording device 2 is lent or a lost medium is used by another person. In this case, an entrance / exit record record that does not have a corresponding face authentication record is extracted as caution data.

[0054] (6) If the number of users recorded as entering the building in the entrance / exit record during a specific period of time (for example, 9:00-17:00 every day) does not match the number of people detected in the facial recognition record. This corresponds to an event in which a person entered the building without being authenticated by the entrance / exit recording device 2. In this case, the matching means 14 erases the entrance / exit record and the face authentication record. The erasure is performed by matching the entrance / exit record with the face authentication record, and if records of the same person and the same time period (for example, within a three minute time difference) are found in both the entrance / exit record and the face authentication record, the record is excluded from the targets of warning data. Records of the entrance / exit record that do not have a corresponding face authentication record, or records of the face authentication record that do not have a corresponding entrance / exit record, are extracted as warning data.

[0055] (7) When an unregistered user (i.e., a guest) is detected in the entry / exit records. This corresponds to an event where a person who is not registered in the entrance / exit recording device 2 enters or exits. For example, if this condition is applied to an entrance / exit where it is expected that unregistered people in the entrance / exit recording device 2 enter and exit less frequently, a trigger point can be set in the video data every time a person who is not registered in the entrance / exit recording device 2 enters or exits.

[0056] 3-3. Adaptive setting of conditions for attention data In the security system 1000, the conditions of the warning data are not limited to being fixed, and may be adaptively set according to the time period, the entry / exit recording device, the entrance / exit, etc. For example, the conditions of the warning data may be set individually for each entrance / exit. Alternatively, the conditions of the warning data may be set according to the time of day or the day of the week of the record.

[0057] 3-4. Processing to respond to attention data The process corresponding to the warning data is not limited to the one exemplified in the embodiment. For example, the output means 15 may transmit information indicating that warning data has occurred or an outline of the warning data to a terminal of a pre-designated manager. The information indicating that warning data has occurred and the information indicating an outline of the warning data are other examples of information regarding the time when the condition of the warning data is satisfied.

[0058] As a process corresponding to the warning data, the output means 15 may notify the entrance / exit recording device 2 (more specifically, the computer 200) that the warning data has been generated. In response to this notification, the computer 200 may set a flag in the entrance / exit record recorded in, for example, a database (not shown). Alternatively, in response to this notification, the computer 200 may rewrite the contents of the database 241 (such as prohibiting the entry / exit of a user recorded as an entrance / exit user in the warning data).

[0059] 3-5. Other embodiments The specific hardware configuration of the security system 1000 is not limited to that exemplified in the embodiment. The security system 1000 may have any hardware configuration as long as the required functions can be implemented. For example, at least a part of the functions described in the embodiment as being implemented in a server on a network may be implemented in a local computer. The server may be a physical server or a virtual server.

[0060] The operations of the security system 1000 described in the embodiment are merely examples. The order of the processes shown in the sequence charts or flowcharts may be partially changed, and other processes may be added.

[0061] The programs executed by a processing device such as CPU 101 may be provided in a state recorded on a computer-readable recording medium such as a CD-ROM (Compact Disc Read only memory), or may be downloaded from a server.

[0062] 1...management system, 11...storage means, 12...acquisition means, 13...acquisition means, 14...matching means, 15...output means, 16...processing means, 2...entrance / exit recording device, 21...entrance / exit restriction means, 22...authentication means, 23...communication means, 24...storage means, 25...processing means, 3...face recognition system, 31...photographing means, 32...storage means, 33...authentication means, 34...communication means, 1000...security system, 101...CPU, 102...memory, 103...storage, 104...communication IF, 113...database, 114...database, 115...database, 200...computer, 241...database, 242...database, 300...server, 321...database, L...electronic lock, O...office, R...card reader

Claims

1. a first acquisition means for acquiring a record of a person who has entered and exited a predetermined area from an entrance and exit recording device; a second acquisition means for acquiring identification information of a person appearing in a video image captured by a photographing means that is in a predetermined positional relationship with the entrance / exit recording device from a face authentication means that performs face authentication on the person appearing in the video image; a verification means for verifying the record acquired by the first acquisition means and the identification information acquired by the second acquisition means; an output means for outputting information relating to the time at which a predetermined condition was satisfied in the record if the person indicated by the identification information satisfies the predetermined condition; An information processing device having the above configuration.

2. a database in which users who use the entrance / exit recording device are registered; The predetermined conditions include a condition that the person is a user registered in the database. The information processing device according to claim 1 .

3. a database in which users who use the entrance / exit recording device are registered; The predetermined condition includes a condition that the person is a user who is not registered in the database. The information processing device according to claim 1 .

4. The predetermined condition includes a condition that the person is registered in the face authentication means. The information processing device according to claim 1 .

5. The predetermined condition includes a condition that the person is not registered in the face authentication means. The information processing device according to claim 1 .

6. The imaging means images at least a part of the interior of the area, The predetermined condition includes a condition that the person's exit from the area is recorded in the record. The information processing device according to claim 1 .

7. The photographing means photographs an area different from the area, The predetermined condition includes a condition that the person's exit from the area is recorded in the record. The information processing device according to claim 1 .

8. the record includes information defining conditions under which access to the area is permitted or prohibited; The predetermined condition includes a condition that the face recognition means detects that the person was in the area at a time when the person was not permitted to enter the area. The information processing device according to claim 1 .

9. A computer comprising: obtaining a record of a person who has entered and exited a predetermined area from an entrance and exit recorder; A step of acquiring identification information of a person appearing in a video from a face authentication means that performs face authentication of the person appearing in the video captured by an image capture means that is in a predetermined positional relationship with the entrance / exit recording device; matching said record with said identification information; if the person represented by the identification information satisfies a predetermined condition in the record, outputting information relating to the time at which the condition was satisfied; An information processing method comprising the steps of:

Images