Information processing system, information processing system control method, and program

Abstract

To prevent people from staying there in a system that restricts entry to a specified area depending on the result of personal authentication.SOLUTION: An information processing system includes: acquisition means for performing verification processing based on characteristics of a person detected from a first image and characteristics of multiple pre-registered people who have been registered in advance to acquire a first degree of certainty that the detected person is a pre-registered person; determination means for determining whether or not to inquire of the pre-registered person on the basis of the first degree of certainty acquired by the acquisition means, a first threshold value, and a second threshold value that is greater than the first threshold value; and inquiry means for making an inquiry to a contact address associated with the pre-registered person on the basis of the result of the determination by the determination means.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an authentication system.

Background Art

[0002] With the development of AI recognition technology, person authentication technologies such as face authentication have been put into practical use. A system for managing entry and exit is one of the most effective applications using this technology, and it is possible to enter and exit without having a key, ID card, ticket, etc.

[0003] For example, when using an entry and exit management system at places such as music festivals, food festivals, stadiums, and food stalls, if the entry check is strict at the time of entry, congestion of the entrants will occur. However, at these places, even if a small number of unauthorized persons enter, it is not likely to cause a major problem. This is because, in many cases at music festivals and stadiums, when entering the seat block after entry, identity verification etc. is often performed again. Also, if staying in a place other than the seat block for a long time, there is a high possibility of being a suspicious person. At food festivals and food stalls, after entering the venue, goods and food are purchased at each store, and it is possible to perform identity verification again at this time. In any case, just having a person without the right to enter the venue enter the venue is not highly beneficial for the entrants and not highly disadvantageous for the organizers.

[0004] In Patent Document 1, face authentication is performed in two stages: entry into the store and the payment stage. At the time of entry into the store, the entrant is identified by comparing with the member database. That is, a person is identified at this stage. Also, it is described that in the face authentication at the payment stage (second stage), authentication is performed only with the person in the store (authentication in the first stage). In Patent Document 2, it is described that authentication is performed using face authentication and other authentication technologies.

[0005] Patent Documents 1 and 2 describe the calculation of the confidence level that a face image is a specific person (1:1 authentication), or the identification of which person among a plurality of registered persons (1:N authentication). Recently, techniques for detecting impersonation, such as tricking person authentication technology by hanging a photo of a specific person or a still image / moving image displayed on a tablet on the authentication camera, or wearing a 3D mask, have been developed and are being applied to payment systems and the like.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0007] In the technology of Patent Document 1, when performing highly reliable authentication when entering a store, there is a problem that the FRR (False Rejection Rate: the rate of rejecting the genuine user) increases, and due to confirmation by staff etc., there will be a delay at the time of entry. On the other hand, if the FRR is lowered, at the time of payment, it is necessary to perform authentication to lower the FAR (False Accept Rate: the rate of accepting an impostor), and there is also a problem that a delay will occur due to confirmation by staff etc. Further, in the technology of Patent Document 2, although a plurality of authentication mechanisms are used to improve the authentication accuracy at the entrance gate, there is a problem that a delay will still occur in order to perform highly reliable authentication. Furthermore, in order to prevent entry by an impersonation act by others, when using an impersonation detection technology, it is difficult to ensure practical accuracy, and there is a problem that the delay will be even greater.

[0008] The present invention has been made in view of the above problems, and an object thereof is to suppress the stay of a person in a system that restricts entry into a predetermined area according to the result of personal authentication.

Means for Solving the Problems

[0009] In order to achieve the above object, an information processing system according to an aspect of the present invention includes an acquisition unit that acquires a first accuracy that the person is a pre-registered person through a collation process based on the characteristics of the person detected from a first image and the characteristics of a plurality of pre-registered persons registered in advance, a determination unit that determines whether it is necessary to inquire of the pre-registered person based on the first accuracy acquired by the acquisition unit, a first threshold value, and a second threshold value that is a threshold value larger than the first threshold value, and an inquiry unit that makes an inquiry to a contact associated with the pre-registered person based on the result of the determination by the determination unit.

Effects of the Invention

[0010] According to the present invention, it is possible to suppress the stay of a person in a system that restricts entry into a predetermined area according to the result of personal authentication.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, all of these plurality of features are not necessarily essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and redundant explanations are omitted.

[0013] <Embodiment 1> First, the situation to which this embodiment is applied will be described below. FIG. 1 is a diagram showing an example of a situation where the information processing system 100 of the embodiment is applied. In this embodiment, a paid food festival with a pre-registration system will be described as an example below. The venue 1 is a venue where the food festival is held. The venue 1 includes a main area 151, a special area 152, and a special area 153. Various types of gates and devices are installed in the venue 1.

[0014] The entrance gates 101 and 102 are gate devices with an automatic opening and closing function that physically permits or restricts entry into the main area 151. Also, a first authentication device 21 described later is arranged at the entrance gates 101 and 102. Although details will be described later, the first authentication device 21 has an imaging unit with an imaging function, and when a person wishing to enter attempts to pass through the entrance gate 101 or 102, they are imaged by the imaging unit. Then, the first authentication device 21 determines whether the person wishing to enter is a person who has pre-registered based on the captured image. And according to the determination result, when the first authentication device 21 permits entry, it opens the gate, and the person attempting to enter can enter the main area 151. Note that when the first authentication device 21 does not permit entry, it does not open the gate.

[0015] There are shops 1011 to 1018 in the main area 151, and authentication devices 131 to 138 having applications operating on smartphones or tablets are arranged in each shop. Since a food festival is held in the area 1 including the main area 151, the shops 1011 to 1018 will be described as food shops below. Also, the authentication devices 131 to 138 refer to the second authentication device 22 described later. That is, in this embodiment, the second authentication device 22 is arranged for each shop. Although details will be described later, the authentication devices 131 to 138 functioning as the second authentication device 22 capture the face of a person wishing to purchase a food item or the like with the imaging unit and perform authentication (face authentication) based on the captured image. And based on this authentication, the purchase of food items or the like in each shop is permitted or not permitted.

[0016] In addition, inside Venue 1, there are Special Area 152 and Special Area 153, which are separate from the area where Shops 1011 to 1018 are located. Special Area 152 and Special Area 153 can be entered by purchasing tickets for the special area in advance or on the day, and it is also possible to have meals inside the special area.

[0017] To enter Special Area 152, it is necessary to pass through Authentication Gate 112. At Authentication Gate 112, a second authentication device 22 and a staff member are arranged. At Authentication Gate 112, the staff member monitors the result of whether the trial entrant can enter (the result of the entry determination) on a screen such as a monitor or a display by means of the second authentication device 22, and permits or restricts the entry of the trial entrant to Special Area 152.

[0018] To enter Special Area 153, it is necessary to pass through Authentication Gate 111. Authentication Gate 111 is a gate with an automatic gate function, and the second authentication device 22 is arranged. Although details will be described later, the second authentication device 22 has an imaging unit with an imaging function, and when a person who wishes to enter the special area attempts to pass through Authentication Gate 111, they are imaged by the imaging unit. Then, the second authentication device 22 determines whether entry is permitted based on the captured image. And the second authentication device 22 makes a judgment on gate opening according to the determination result.

[0019] In this way, entry to the special area may be in either case of an automatic gate without a staff member or a gate with a staff member. In either case, entry to the special area can be permitted or restricted by performing face authentication with a camera (imaging device).

[0020] When entering (returning) from the special area 152 into the main area 151, one can enter (return) into the main area 151 from the gate 121. The gate 121 is a gate for returning from the special area 152 into the main area 151, and in principle, it is not possible to enter the special area 152 from the gate 121. Also, when entering (returning) from the special area 153 into the main area 151, one can enter (return) into the main area 151 from the gate 122. The gate 122 is a gate for returning from the special area 153 into the main area 151, and in principle, it is not possible to enter the special area 153 from the gate 122. Also, for the gates 121 and 122, it is assumed, for example, that they are rotary gates that can only rotate in one direction or gates where attendants are arranged as exits partitioned by ropes or the like, and it is not assumed that they have the same function as the authentication gate 111. That is, when returning into the main area 151, identity verification by face recognition or the like is not performed.

[0021] Furthermore, a shop 1019 is provided in the special area 152. Near the shop 1019, an authentication device 139 is installed. The authentication device 139 is synonymous with the second authentication device 22 described later.

[0022] Also, when leaving the venue 1 from inside the venue 1 to the outside, it is necessary to pass through an exit gate 141 that permits or restricts leaving. The exit gate 141 is a gate with the same automatic gate function as the entrance gates 101 and 102. Also, an authentication device that functions as the second authentication device 22 described later is arranged at the exit gate 141.

[0023] In addition, in the above, for the sake of easy understanding of the explanation, the food festival was used as an example for the explanation. However, it is not limited to this, and in addition to the food festival, music festivals, stadiums, amusement parks, hot spring facilities, rental conference rooms, food stalls, etc. are also acceptable. In addition, the preferable situation in this embodiment is a situation that satisfies all of the following conditions 1 to 3, and if these conditions are satisfied, there is no restriction on the implementation of the above-described venue, amusement facilities, and events.

[0024] The following shows Condition 1, Condition 2, and Condition 3. However, Condition 2 is composed of two elements, Condition 2-1 and Condition 2-2. And if either Condition 2-1 or Condition 2-2 is satisfied, then Condition 2 is considered satisfied. Condition 1: It is necessary to enter the venue from the public space. Condition 2-1: Just entering the venue does not allow one to receive sufficient services. Condition 2-2: If one stays in the venue for a long time just after entering, there is a possibility of being regarded as a suspicious person. Condition 3: Inside the venue, each individual can receive further useful services.

[0025] Here, in the case of the above-mentioned food festival, since Conditions 1, 2-1, and 3 are satisfied, all of Conditions 1 to 3 are satisfied. Also, in the case of a stadium or a rented conference room, since staying outside the seat block or outside the conference room for a long time may make one seem suspicious, all of Conditions 1, 2-2, and 3 are satisfied.

[0026] FIG. 2 is a block diagram showing a configuration example of the information processing system 100 according to the present embodiment. The information processing system 100 in the present embodiment includes a first authentication device 21 and a second authentication device 22. Note that the information processing system 100 in the present embodiment may be configured by the first authentication device 21, the second authentication device 22, and the mobile device 23.

[0027] In the present embodiment, the first authentication device 21 is an information processing device installed at a location for determining entry to Venue 1. For example, it is installed with respect to the entry gates 101 and 102 shown in FIG. 1. Also, the second authentication device 22 is an information processing device installed at a service-providing location inside Venue 1, and can be, for example, a smartphone or a tablet. The second authentication device 22 is arranged near the shops 1011 to 1018 and 1019 shown in FIG. 1. Also, the second authentication device 22 is arranged at a location for leaving Venue 1 or entering a special area. Taking FIG. 1 as an example, the second authentication device 22 is also installed at the exit gate 141 and the authentication gates 111 and 112.

[0028] The mobile device 23 is an electronic device (communication terminal) that each of the attendees registered to enter the venue carries (owns). In this embodiment, the mobile device 23 is assumed to be, for example, a mobile phone, a smartphone, a tablet, or the like.

[0029] The first authentication device 21, the second authentication device 22, and the mobile device 23 are communicably connected via a network 24 so that they can exchange information, signals, etc. The network 24 is a wired or wireless LAN (Local Area Network) or WAN (Wide Area Network).

[0030] The first authentication device 21 is an information processing device having a first imaging unit 211, a first calculation unit 212, a first collation unit 213, a first display unit 214, a first transmission / reception unit 215, an update unit 216, an inquiry unit 217, and a holding unit 218.

[0031] The first imaging unit 211 is composed of an imaging lens and an imaging sensor such as a CCD or CMOS. The first imaging unit 211 images a person or the like who is the subject of face authentication and acquires an image. In this embodiment, an image including a person's face is acquired. In this embodiment, the first imaging unit 211 images a person (an applicant for entry, an authenticated person) who wants to enter a predetermined area, and acquires an image (image data) including the face of the person reflected in the captured image.

[0032] The first calculation unit 212 includes a face feature calculation unit 2121 and an impersonation detection unit 2122. The face feature calculation unit 2121 calculates personal features from the image output from the first imaging unit 211. In this embodiment, a face feature vector (hereinafter referred to as a feature vector) of a person is calculated. A feature vector is a vector characteristic of an individual. The similarity between the feature vectors of images of the same person indicates a high value, and the similarity between the feature vectors of images of other people indicates a low value. The method of calculating the feature vector is known in the field of face authentication. Examples of those that calculate face features include FaceNet and the like. The face feature calculation unit 2121 outputs the extracted feature vector to the first verification unit 213. Also, the impersonation detection unit 2122 obtains an impersonation probability by performing an impersonation detection process and outputs it to the first verification unit 213. That is, the first calculation unit 212 provides the feature vector calculated by the face feature calculation unit 2121 and the impersonation probability calculated by the impersonation detection unit 2122 (if any) to the first verification unit 213. However, in this embodiment, the impersonation detection unit 2122 is not an essential component.

[0033] The first verification unit 213 verifies the feature vector output from the first calculation unit 212 with the feature vectors of all pre-registered ticket purchasers and compares them. In addition, the first verification unit 213 calculates an accuracy as a degree of certainty for identifying whether the person is a registered user.

[0034] The first display unit 214 is a display device such as a display or a monitor, for example. Note that the first display unit 214 may be incorporated in a gate with an automatic gate function capable of opening and closing the door so that the screen can be seen from the outside, or may be the gate itself capable of opening and closing the door. That is, the first display unit 214 can also be an entrance gate, an authentication gate, or an exit gate having an automatic gate function capable of opening and closing the door. When the first display unit 214 corresponds to the gate, the result of determining whether entry is permitted is displayed based on the open / closed state of the gate.

[0035] The first transmission / reception unit 215 communicates information and messages with the second authentication device 22 and the mobile device 23. The update unit 216 updates the trial record of the ticket purchaser (pre-registered person) held in the holding unit 218. The inquiry unit 217 transmits a predetermined inquiry message to the mobile device 23 via the first transmission / reception unit 215. More specifically, the inquiry unit 217 transmits a predetermined inquiry message to the mobile device 23 held by the registrant classified as "on hold" by the first verification unit 213. Then, the inquiry unit 217 receives a response message to the inquiry message. Further, the inquiry unit 217 requests the update unit 216 to update the trial record according to the content of the response message. The holding unit 218 stores (holds) the trial record of the ticket purchaser (pre-registered person).

[0036] The second authentication device 22 is an information processing device configured to include a second imaging unit 221, a second calculation unit 222, a second verification unit 223, a second display unit 224, and a second transmission / reception unit 225.

[0037] The second imaging unit 221 is composed of an imaging lens and an imaging sensor such as a CCD or a CMOS, similar to the first imaging unit 211. The second imaging unit 221 captures an image of a person or the like to be face-authenticated to obtain an image. In this embodiment, an image of an area including a person's face is obtained. In this embodiment, the second imaging unit 221 captures an image of a person (an admission trialist, an authenticated person) who wants to enter a predetermined area, and obtains an image (image data) including the face of the person reflected in the captured image. Note that the second imaging unit 221 may be a fixed camera. Also, a camera embedded in the second authentication device 22 may be used. That is, when the second authentication device 22 is a smartphone or a tablet, it may be a camera built into the smartphone or the tablet.

[0038] The second calculation unit 222 calculates the personal characteristics of the service user from the image output by the second imaging unit 221, and performs a feature vector extraction process. The second matching unit 223 compares the feature vector calculated by the second calculation unit 222 with the values in the feature vector columns of all the tester records held by the holding unit 218, and calculates the similarity.

[0039] The second display unit 224 is, for example, the display screen of a smartphone or a tablet terminal. Note that the second display unit 224 may be built into a gate with an automatic gate function that can open and close the door so that the screen can be seen from the outside, or may be the gate itself that can open and close the door. That is, the second display unit 224 can also be an entrance gate, an authentication gate, or an exit gate having an automatic gate function that can open and close the door. When the second display unit 224 corresponds to a gate, the result of determining whether entry is permitted is displayed based on the open / closed state of the gate. The second transmission / reception unit 225 communicates information and messages with the first authentication device 21 and the mobile device 23.

[0040] Note that the second authentication device 22 may be an imaging device such as a camera having an information processing device with a hardware configuration shown in FIG. 3 described later inside, or may be an electronic device such as a smartphone or a tablet. In the case of a smartphone or a tablet, the second imaging unit 221 can be a camera built into the smartphone or the tablet.

[0041] Hereinafter, a simple flow of the process in the first authentication device 21 will be described. Also, it is assumed that a person (ticket purchaser) who intends to enter the main area 151 tries to enter from the entrance gate 101.

[0042] First, a ticket purchaser who is a pre-registered person tries to enter the venue 151 from the entrance gate 101. At this time, the first imaging unit 211 of the first authentication device 21 arranged at or near the entrance gate 101 captures an image of the face of the ticket purchaser, and acquires a face image. The first imaging unit 211 outputs the acquired face image to the first calculation unit 212.

[0043] Next, the first calculation unit 212 calculates personal characteristics from the face image output by the first imaging unit 211 and performs feature vector extraction processing. After that, the first calculation unit 212 outputs the feature vector extracted from the face image to the first collation unit 213.

[0044] Next, the first collation unit 213 collates the feature vector output from the first collation unit 213 with the feature vectors of all pre-registered ticket purchasers and compares them. Through this collation process, the identification of the registrant and its accuracy are calculated. In the first collation unit 213, further, based on the calculated accuracy, the entry of the person wishing to enter is classified into three levels: "permitted", "held", and "not permitted". And this determination result (judgment result) is held in the holding unit 218 as a trial record.

[0045] When classified as "permitted" or "held" by the first display unit 214 by the first collation unit 213, the first display unit 214 displays that the entry of the person is permitted. When classified as "not permitted" by the first collation unit 213, the first display unit 214 displays that the entry of the person is not permitted. As described above, in the form where the first display unit 214 is the gate itself, when classified as "permitted" or "held" by the first collation unit 213, it indicates that the entry of the person is permitted by opening the gate. On the other hand, when classified as "not permitted" by the first collation unit 213, the first display unit 214 indicates the non-permission of the person's entry by closing the gate.

[0046] Here, in this embodiment, when the admission is classified as "on hold", for the purpose of suppressing stay, the ticket purchaser is permitted to enter the main area 151. However, a process of confirming whether the person is really the registered person through identity verification is also performed. Specifically, when it is classified as "on hold" by the first verification unit 213, the inquiry unit 217 sends a predetermined message to the pre-registered mobile device 23 possessed by the registered person (classified as "on hold") identified by the first verification unit 213. At this time, the predetermined message is sent via the first transceiver unit 215. As the predetermined message, for example, an inquiry message prompting confirmation of entry into the venue, such as "Are you the one who just entered the venue?", is sent.

[0047] In addition, in a registration database (not shown), information such as the person's facial feature amount, the access destination of the possessed mobile device (for example, the SNS account), and the permission to enter a special area is stored in a searchable manner for each registered person in advance. And the registration database is configured to be accessible from the first verification unit 213. Note that SNS is an abbreviation for Social Networking Service.

[0048] When the inquiry unit 217 receives a reply to the message from the pre-registered mobile device 23 possessed by the registered person, it requests the update unit 216 to update (correct) the trial record of the ticket purchaser held in the holding unit 218 according to the reply. The reply sent from the pre-registered mobile device 23 possessed by the registered person is sent to the inquiry unit 217 via the first transceiver unit 215. The above is the process of the first authentication device 21. For the detailed process, reference will be made to FIGS. 4 and 5 and the like and described later.

[0049] FIG. 3 is a diagram showing an example of the hardware configuration of the first authentication device 21. The first authentication device 21 includes a CPU 10, a ROM 11, a RAM 12, an imaging unit 13, a storage device 14, a display unit 15, and a communication unit 16.

[0050] The various functions of the first authentication device 21 are not limited to the example where the CPU 10 executes all of them. For example, by using a predetermined compiler, a dedicated circuit may be automatically generated on the FPGA from a predetermined program. FPGA is an abbreviation for Field Programmable Gate Array. Also, a gate array circuit may be formed in the same manner as the FPGA and realized as hardware. Further, it may be realized by an ASIC (Application Specific Integrated Circuit).

[0051] The CPU 10 is a central processing unit and controls the entire first authentication device 21. Also, the CPU 10 reads a control program recorded in the ROM (Read Only Memory) 11 and executes various processes. The ROM 11 is a non-volatile memory and stores the program of the present embodiment and other programs and data necessary for control.

[0052] The RAM (Random Access Memory) 12 is a volatile memory and is used as a temporary storage area such as a main memory or a work area. The imaging unit 13 is configured to include a lens unit and an image sensor. That is, it can be synonymous with the above-described first imaging unit 211.

[0053] The storage device 14 stores various data and various programs. The storage device 14 is, for example, a non-volatile storage device such as an HDD, a flash memory, or an SD card. The storage device 14 is used as a permanent storage area for an OS, various programs, and various data, and is also used as a storage area for various short-term data.

[0054] The display unit 15 may be composed of a display device such as a display or a monitor. The communication unit 16 performs communication processing with external devices such as the second authentication device 22, the mobile device 23, or a server via the network 24, either wired or wirelessly. Incidentally, the first authentication device 21 may have an input device. In that case, the input device may be composed of a device for the user to manually perform settings, such as a keyboard, a mouse, or a touch panel.

[0055] Incidentally, the second authentication device 22 also has the same hardware configuration as the first authentication device 21. That is, the second authentication device 22 has a CPU 10, a ROM 11, a RAM 12, an imaging unit 13, a storage device 14, a display unit 15, and a communication unit 16. Similarly, the mobile device 23 also has a CPU 10, a ROM 11, a RAM 12, an imaging unit 13, a storage device 14, a display unit 15, and a communication unit 16.

[0056] FIG. 4 and FIG. 5 are flowcharts showing examples of the processing (operation) of the first authentication device 21 in Embodiment 1. The following respective processes (each operation) are realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, the notation of the processes (steps) is omitted by attaching an S at the beginning of each process (step).

[0057] In S401, the first imaging unit 211 images a person attempting to enter the main area 151 (hereinafter referred to as the entry trial person) and acquires an image (the first image). Specifically, a face image (entry-time image) in which at least the face of the entry trial person is captured is acquired. Thereafter, the first imaging unit 211 outputs the acquired image to the first calculation unit 212.

[0058] In S402, the first calculation unit 212 calculates the personal characteristics of the person entering the venue from the image output by the first imaging unit 211 in S401, and performs extraction processing of a feature vector (face feature vector). Note that the processing in S402 is performed by the face feature calculation unit 2121 of the first calculation unit 212. In this way, the personal characteristics calculated by the first calculation unit 212 include a feature vector (face feature vector). Thereafter, the first calculation unit 212 outputs the extracted feature vector to the first collation unit 213.

[0059] In S403, the first collation unit 213 performs a collation process based on the feature vector output from the first calculation unit 212 in S402 and the feature vectors of all pre-registered ticket purchasers (multiple pre-registered persons). By performing the said collation process, the first collation unit 213 can identify which registered person the person entering the venue is. At the time of identification, the first collation unit 213 calculates a certainty (first certainty) as the personal certainty in the identification.

[0060] In S404, the first collation unit 213 compares the certainty calculated in S403 with two threshold values (first threshold value and second threshold value) respectively, and makes a determination regarding the entry of the person entering the venue into the main area 151. Note that in the determination of S404, the determination is made so that the determination result is one of the three: "permitted", "held in abeyance", and "not permitted". If the determination result is "permitted", the process proceeds to S408. On the other hand, if it is "held in abeyance" or "not permitted", the process proceeds to S405. Note that the details of the entry determination process in S404 will be described later.

[0061] An example of the trial participant record held by the holding unit 218 in the present embodiment is shown below as Table 1.

Table 1

[0062] Table 1(A) shows the trial records of five entrance trial participants, that is, the judgment results for five people. The entrance trial participant ID is a different ID assigned to each entrance trial participant, for example, it may be a number assigned in the order of entrance trials. The entrance trial image is an image of the entrance trial participant captured by the first imaging unit 211. Based on this image, the face feature calculation unit 2121 calculates a feature vector.

[0063] Then, the feature vector is compared with the feature vectors of all pre-registered ticket purchasers by the first collation unit 213, and the similarity (the first similarity) is calculated. The registered ID of the registrant with the highest similarity is described in the registered ID column, and the similarity at that time is described in the similarity column, respectively. The feature vector column is the pre-registered feature vector of the registered ID. This can be calculated in advance in the same manner as the face feature calculation unit 2121 by registering one's own photo at the time of membership registration, ticket purchase, or afterwards. The similarity between feature vectors is determined by the face feature calculation unit 2121. For example, Euclidean distance or cosine similarity is used. Since the similarity is a well-known technique in the field of face authentication, the details are omitted here. Here, the similarity takes a value from 0 to 1, and the larger the value, the more similar the two feature vectors are. That is, it is assumed that the two feature vectors are more similar as the value is 1 or closer to 1.

[0064] The personal certainty is a value calculated by the first collation unit 213, and represents the degree of certainty that the person shown in the image at the time of the entrance trial is the person with the registered ID. Although the details will be described later, in this embodiment, the personal certainty is assumed to take the same value as the similarity. That is, the first collation unit 213 determines the calculated first similarity as the first certainty.

[0065] The entrance determination is the determination result of the first collation unit 213. When it is strongly estimated that the entrance trial participant is the person with the registered ID, it becomes "permitted". On the other hand, when it is strongly estimated that the person in the entrance trial is not the person with the registered ID, it becomes "not permitted". And when it is neither "permitted" nor "not permitted", it becomes "pending". That is, the determination result as the entrance determination of the first collation unit 213 has three types: "permitted", "pending", and "not permitted".

[0066] Here, the entry determination in S404 is specifically made based on two threshold values and the person's certainty (the first certainty). Here, in the determination process, the first of the two threshold values is set as the first threshold Th1, and the second threshold value is set as the second threshold Th2. Also, the person's certainty is set as the person's certainty C. The first threshold Th1 is set to a value greater than 0. The second threshold Th2 is set to a value greater than the first threshold Th1 and less than 1. That is, the relationship between the first threshold Th1 and the second threshold Th2 is 0 < Th1 < Th2 < 1.

[0067] As a result of the entry determination in S404, if C < Th1, that is, when the certainty C is less than the first threshold Th1, it is determined as "not permitted" by the first verification unit 213. Also, when Th1 ≤ C < Th2, that is, when the certainty C is greater than or equal to the first threshold Th1 but less than the second threshold Th2, it is determined as "pending" by the first verification unit 213. Further, when Th2 ≤ C, that is, when the certainty C is greater than or equal to the second threshold, it is determined as "permitted" by the first verification unit 213.

[0068] For example, when the first threshold Th1 = 0.30 and the second threshold Th2 = 0.85, the result of the determination process of the first verification unit 213 is an entry determination result as shown in Table 1(A) as the tester record.

[0069] In S405, the first verification unit 213 determines whether the result of the entry determination in S404 is "not permitted". If the result of the entry determination in S404 is "not permitted", it proceeds to S406. On the other hand, if the result of the entry determination in S404 is not "not permitted", that is, if it is "pending", it proceeds to S408.

[0070] Still, the determination result as to whether or not to allow the admission trial participant to enter the main area 151 is displayed on the screen by the first display unit 214. At the time of this display, for example, it may be displayed as "Admission Permitted" or "Admission Not Permitted" on the screen of a tablet other than the first display unit 214 or on the screen of a display device such as a fixed monitor. Still, as described above, when the first display unit 214 is the gate itself, the determination result is displayed by opening and closing the gate.

[0071] Here, when it is displayed as "Admission Permitted" and the admission trial participant attempts to enter through a gate device with an automatic opening and closing function such as the admission gate 101 or the admission gate 102, the first authentication device 21 opens the gate to allow the admission trial participant to enter the venue 151. On the other hand, when it is displayed as "Admission Not Permitted", the first authentication device 21 closes the gate to prevent the admission trial participant from entering the main area 151. Still, the first display unit 214 may function as a gate device with an automatic opening and closing function. In that case, the gate may be opened when it is determined as "Permitted" or "Pending" without displaying the determination result, and the gate may not be opened when it is determined as "Not Permitted".

[0072] Also, when it is displayed as "Admission Permitted" and the admission trial participant attempts to enter through an authentication gate (for example, the authentication gate 112) where a staff member is stationed instead of a gate device with an automatic opening and closing function, the staff member may guide the admission trial participant into the main area 151. On the other hand, when it is displayed as "Admission Not Permitted", the staff member may inform the admission trial participant that they cannot enter the main area 151.

[0073] In this embodiment, as described above, those who are determined to be "permitted" or "held in abeyance" among the entrants are the ones who enter the main area 151. Conventionally, in order to reduce or eliminate the number of people determined to be "held in abeyance", the authentication threshold was raised and tightened, or other media such as paper tickets and electronic tickets were used in combination. Furthermore, when authentication still could not be achieved, the staff would conduct identity verification. However, all of these methods took more time compared to simple face authentication, resulting in delays during entry.

[0074] This embodiment is characterized in that it allows the entrants who are determined to be "held in abeyance" to enter. By allowing those determined to be "held in abeyance" to enter, it is possible to significantly alleviate the delays during entry. However, those determined to be "held in abeyance" are not treated the same as those determined to be "permitted". Therefore, it is confirmed by inquiring the person whether the entrant determined to be "held in abeyance" through identity verification is the same as the person with the registered ID recorded in the entrant record.

[0075] In S406, the update unit 216 updates the entrant record stored in the holding unit 218. Specifically, based on the result of the determination process in S405, the entrant record is updated, and the updated entrant record is stored in the holding unit 218. As a result, the determination result is recorded in the column of the determination result of the entrant ID.

[0076] In S407, the first collation unit 213 rejects the entry of the entrant who is attempting to enter the main area 151, and causes the first display unit 214 to display the determination result in S405. Then the process ends. Note that the first collation unit 213 rejects the entry of the entrant attempting to enter the main area 151 based on the entry determination ("not permitted" determination) in S405.

[0077] In S408, the update unit 216 updates the trial participant record stored in the holding unit 218. Specifically, based on the results of the determination processes in S404 and S405, the trial participant record is updated, and the updated trial participant record is stored in the holding unit 218. As a result, the determination result is recorded in the column for the determination result of the entry trial participant ID.

[0078] In S409, the first collation unit 213 permits the entry of the entry trial participant who is attempting to enter the main area 151, and causes the first display unit 214 to display the determination results in S404 and S405. Note that the first collation unit 213 permits the entry of the entry trial participant into the main area 151 based on the entry determination (“permit” or “hold” determination) in S404 and S405.

[0079] In S410, the first collation unit 213 determines whether the result of the entry determination is “hold”. If the result of the entry determination is “hold” (that is, if NO in S405), the process proceeds to S411. On the other hand, if the result of the entry determination is not “hold”, that is, if it is “permit” (that is, if YES in S404), the process ends.

[0080] In S411, the inquiry unit 217 sends a predetermined notification to the communication terminal (portable device 23) carried (owned) by the person who is pre-registered with the registered ID for which the determination result was saved in S408. Specifically, it sends an inquiry message for identity verification such as "Did you just pass through the gate?" or "Did you just enter the venue?" That is, in S410, the inquiry unit 217 determines whether it is necessary to inquire of the pre-registered person based on the above-described first accuracy and two thresholds (first threshold Th1 and second threshold Th2). Then, in S411, the inquiry unit 217 conducts an inquiry to the contact associated with the pre-registered person. As the inquiry method, a phone call (inquiry by automatic voice), sending an email, a text message, etc. can be used. The inquiry message by the inquiry unit 217 is sent to the portable device 23 via the first transmission / reception unit 215 and the network 24. At this time, in order to make it easier for the recipient of the inquiry message to judge, it is preferable to send the image at the time of the entry trial included in the trial participant record in Table 1(A) as an attached image. Also, not limited to the face image, for example, information such as a full-body image and the time at the time of sending the inquiry message may be sent together.

[0081] However, it is not essential to attach an image or time to the inquiry message. For example, if a face image is not attached to the message, it is not necessary to save the face image in the trial participant record. Also, if a full-body image is to be attached, it is necessary to add it to the trial participant record. What information is necessary depends on the service level to the recipient of the inquiry message and the security risk, and these can be set arbitrarily.

[0082] In S412, the inquiry unit 217 determines whether a reply to the inquiry message sent to the mobile device 23 in S411 has been received. If a reply to the inquiry message sent to the mobile device 23 has been received, the process proceeds to S413. On the other hand, if a reply to the inquiry message sent to the mobile device 23 has not been received, the system waits until a reply to the inquiry message is received. The reply to the inquiry message is passed from the mobile device 23 to the inquiry unit 217 via the network 24 and the first transceiver unit 215.

[0083] In S413, based on the answer (reply) to the inquiry received by the inquiry unit 217, it is determined whether the person on hold (the person who was determined to be NO in S405 and has entered the main area 151) is the registered person. If the person on hold is the registered person, the process proceeds to S415. On the other hand, if the person on hold is not the registered person, that is, if it is another person, the process proceeds to S414.

[0084] In S414, the inquiry unit 217 requests the update unit 216 to update the entry determination part in the trial record of the entry trial person from "on hold" to "not permitted", and the update unit 216 changes the part. Then the process ends.

[0085] In S415, the inquiry unit 217 requests the update unit 216 to update the entry determination part in the trial record of the entry trial person from "on hold" to "permitted", and the update unit 216 changes the part. Then the process ends.

[0086] As described above, in this embodiment, even a person who is determined to be "on hold" when entering the main area 151 can enter the main area 151. Then, after the person determined to be "on hold" enters the main area 151, an identity verification can be performed on the person to make a determination of "permitted" or "not permitted". As a result, in the main area 151, in addition to the persons determined to be "permitted", there may also be persons determined to be "not permitted". However, in this embodiment, a person determined to be "not permitted" cannot purchase hoods or other items within the main area 151. This process will be described later.

[0087] For example, assuming that the trial subject record held in the holding unit 218 is the case of Table 1(A), inquiry messages are sent to the registered persons with registered IDs 31, 55, and 92. If the person with the registered ID 31 replies as "self", the person with the registered ID 92 replies as "other", and the person with the registered ID 55 does not reply, the trial subject record is updated as shown in Table 1(B). In this way, when it is determined that the entry of an entry trial subject into a predetermined area is on hold, the inquiry unit 217 inquires of the mobile device 23 possessed by the entry trial subject as to whether the entry trial subject is the person who registered.

[0088] Note that the remarks column is a column for recording the reasons and the like when the determination such as the entry determination is changed (updated). By providing the remarks column, there is an effect that smooth correspondence can be made when resolving troubles with the entry trial subject, such as "Why is it not permitted?".

[0089] Note that in this embodiment, the inquiry unit 217 inquired whether the person determined to be on hold by the first verification unit 213 is the person who registered. However, it is not limited to this. For example, the process in S410 may be omitted, and the inquiry unit 217 may make an inquiry when the determination result of the first verification unit 213 is "permitted" or "on hold".

[0090] The operation of the first authentication device 21 has been described above. Next, the operation of the second authentication device 22 will be described. As described above, in the main area 151, in addition to the persons determined as "permitted" or "pending", there may also be persons determined as "not permitted" through identity verification. However, in order to actually receive various services such as purchasing a hood within the venue, authentication by the authentication devices 131 to 139, which are the second authentication devices 22, must be passed (it is necessary to obtain a permission determination). In the present embodiment, when the determination result of the trial participant record held in the holding unit 218 is "pending" or "not permitted", the admitted trial participant who has been so determined is prevented from fully enjoying the service.

[0091] Hereinafter, the operation of the second authentication device 22 will be mainly described with reference to FIGS. 1, 2, and 6. Also, hereinafter, processing targeted at an admitted trial participant who can enter the main area 151 and intends to purchase a hood at the shop 1011 (hereinafter referred to as a service - enjoying trial participant) will be described. When purchasing a hood at the shop 1011, the service - enjoying trial participant is authenticated by the authentication device 131, which is the second authentication device 22. FIG. 6 is a flowchart showing an example of the processing (operation) in the second authentication device 22. Each of the following processes (each operation) is realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, the notation of the process (step) is omitted by attaching an S at the beginning to each process (step).

[0092] In S601, the second imaging unit 221 images the service - enjoying trial participant and acquires an image. Specifically, an image (image at the time of service - enjoying trial) in which at least the face of the service - enjoying trial participant is shown is acquired. Then, the second imaging unit 221 outputs the acquired image to the second calculation unit 222.

[0093] In S602, the second calculation unit 222 calculates the personal characteristics of the service user from the image output by the second imaging unit 221 in S601, and performs feature vector extraction processing. After that, the second calculation unit 222 outputs the extracted feature vector to the second matching unit 223. Since the second calculation unit 222 has the same configuration as the first calculation unit 212, detailed description thereof is omitted.

[0094] Note that the first calculation unit 212 and the second calculation unit 222 do not necessarily have the same configuration. For example, a network for face feature calculation unit 2121 of the first calculation unit 212 that has been downsized using a technique such as distillation may be used. In this case, generally the authentication accuracy decreases, but the amount of memory used is significantly reduced, and it becomes possible to operate on smartphones, tablets, etc. When using the distillation technique, the feature vectors for the same image will be similar, so the feature vectors of the trial user records held in the holding unit 218 can be used as they are by the second matching unit 223 described below. It is also possible to use a feature extractor for the second calculation unit 222 that is completely different from the face feature calculation unit 2121. In that case, a feature vector that can be compared by the second matching unit 223 may be included in the trial user record separately. The distillation technique is common in the field of deep learning technology, and including other feature vectors in the trial user record is a well-known technique, so detailed description is omitted.

[0095] In S603, the second matching unit 223 compares the feature vector calculated by the second calculation unit 222 in S602 with the values in the feature vector fields of all the trial user records held by the holding unit 218, and calculates the similarity. To distinguish the similarity calculated in S603 from the above-described similarity (first similarity), hereinafter it is referred to as the second similarity. Also in S603, the trial user records are sent from the holding unit 218 to the second matching unit 223 via the first transceiver 215, the network 24, and the second transceiver 225.

[0096] In S604, the second collating unit 223 refers to the trialist record sent from the holding unit 218 in S603, and identifies the entry trialist ID with the highest second similarity. That is, the person with the highest second similarity is identified from the trialist record.

[0097] In S605, the CPU 10 displays the entry determination result included in the identified trialist record in S604 on the second display unit 224. Then the process ends.

[0098] Incidentally, the second display unit 224 may be the display screen of a smartphone or a tablet terminal as described above. FIG. 7 is an example of the display screen of the second display unit 224 of the second authentication device 22, and is a diagram showing a state where the determination result is displayed on the screen of a tablet or the like. FIG. 7(A) is a diagram showing an example when the determination result is "permitted". FIG. 7(B) is a diagram showing an example when the determination result is "not permitted". FIG. 7(C) is a diagram showing an example when the determination result is "pending". The attendant, the shop clerk of the store, etc. look at this display image and determine whether to provide the service to the service enjoyment trialist. In the case of this example, it is determined whether to sell the hood or not.

[0099] In FIG. 7, in addition to the image captured by the second imaging unit 221, the registered user ID and the entry time are displayed as an example in addition to permitted, not permitted, and pending. However, in this embodiment, it is essential to distinguish between permitted, not permitted, and pending. Therefore, at least information indicating permitted, not permitted, and pending is displayed on the display screen. Other information is supplementary information for smoothly proceeding with the on-site response when not permitted or pending, and although it is preferable to display it because the amount of information increases, it may not be displayed depending on the case.

[0100] Also, for example, additional information such as "image at the time of entry trial (face image, full-body image)", "face image at the time of registration", "similarity", "identity accuracy", "presence or absence of a response to an inquiry", "time of the inquiry or the response to the inquiry", "remarks", "others", etc. may be displayed together. In addition, when using this additional information, corresponding values or information for this information may be entered in the trial participant record. For example, when "not permitted" or "on hold" is displayed on the second display unit 224, if the content of the "remarks" column is also displayed together, it will be possible to understand why such a judgment was made, and subsequent correspondence can be made smoothly.

[0101] In addition, as described above, the second authentication device 22 is installed not only in the shops 1011 to 1018 but also in the authentication gate 111, the authentication gate 112, and the exit gate 141. At the authentication gate 111, the staff member confirms the result of whether the entry trial participant can enter or not on the screen by the second authentication device 22, and permits or restricts the entry of the entry trial participant into the special area 152. That is, the staff member confirms the judgment result as shown in FIG. 7 on the screen and permits or restricts the entry of the entry trial participant into the special area 152.

[0102] The authentication gate 112 is a gate with an automatic gate function, and permits or restricts the entry of the entry trial participant into the special area 153 according to the judgment result. That is, when the judgment result is "permitted" or "on hold", the gate is opened to permit the entry of the entry trial participant into the special area 152. On the other hand, when the judgment result is "not permitted", the entry of the entry trial participant into the special area 152 is restricted without opening the gate. The case of the exit gate 141 can be the same as the case of the authentication gate 112, but there may be differences in the operation of the venue, such as more thorough confirmation by the staff of the person judged as "on hold" or "not permitted" than in the shops 1011 to 1018. Also, for example, a person judged as "not permitted" may be allowed to leave the venue through another exit gate (an exit gate without an automatic gate function). Also, for example, a person judged as "not permitted" who has once left the venue may be recorded in the trial participant record as a person who cannot re-enter.

[0103] So far, the above description has been given by taking the example of a food festival. However, as long as it satisfies the above-mentioned conditions 1 to 3, such as a music festival, a food festival, a stadium, a food stall village, etc., the applicable destination is not limited.

[0104] In addition, in the present embodiment, the face feature calculation unit 2121 of the first calculation unit 212 calculates the face features of a person and extracts a feature vector. However, it is not limited to face features. For example, it can be implemented by any biometric authentication that extracts a feature vector from a biometric image captured by a camera, such as iris authentication and fingerprint authentication.

[0105] As described above, according to the information processing system 100 of the present embodiment, it is possible to ensure practical authentication accuracy by verifying the identity while preventing overstaying at the entrance to a predetermined venue.

[0106] <Embodiment 2> Recently, ticket scalping, which resells tickets at a high price for profit, has become a problem. In Embodiment 2, a configuration effective for preventing ticket scalping will be described. To prevent ticket scalping, an effect can be achieved by improving the verification accuracy of the first verification unit 213 and the second verification unit 223, but it is also possible by other methods. Note that since the configurations and respective functional units of the information processing system 100 including the first authentication device 21 and the second authentication device 22 and the mobile device 23 in Embodiment 2 are the same as those in Embodiment 1, the following will describe the parts and processes different from Embodiment 1. Also, matters not mentioned in Embodiment 2 can be the same as those in Embodiment 1.

[0107] In Embodiment 1, when the determination result of the first verification unit 213 is "pending", the inquiry unit 217 inquires whether the person determined to be pending is the person pre-registered. In Embodiment 2, a time limit from the inquiry to the reply will be set.

[0108] Generally, when a ticket is resold, the seller of the ticket is often not at the venue. Here, if a time limit is set from the inquiry to the reply, the registrant, i.e., the ticket seller, must reply within this time. Even if a reply is made after the time limit has passed, the admission determination of the trial entrant's trial record remains "on hold". Therefore, when the said trial entrant undergoes authentication by the second authentication device 22, the second display unit 224 will display "on hold".

[0109] Since the response is usually made by the regular staff, it can be expected that the ticket purchaser side will avoid obtaining tickets through ticket reselling from the beginning. The seller side of ticket reselling has to wait for a long time to reply to the inquiry message, especially in the case of an event where a large number of people enter the venue at once, and the convenience of ticket reselling (selling) decreases.

[0110] Hereinafter, the processing of the first authentication device 21 in the second embodiment will be described below with reference to FIGS. 8 and 9. FIGS. 8 and 9 are flowcharts showing examples of the processing (operations) of the first authentication device 21 in the second embodiment. Note that the difference from the flowcharts shown in FIGS. 4 and 5 is that there is a process of S813 in the flowcharts shown in FIGS. 8 and 9. That is, S801 to S812 are the same processes as S401 to S412 respectively, and S814 to 816 are the same processes as S413 to S415 respectively. Therefore, the description will be omitted hereinafter. The following respective processes (each operation) are realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, by prefixing each step with S, the notation of the step is omitted.

[0111] In S813, the CPU 10 determines whether the reply to the inquiry received by the inquiry unit 217 has passed a predetermined time since the time when the inquiry was made. If the predetermined time has passed since the time when the inquiry was made, the CPU 10 does not perform the process of identity verification or the like and ends the process. On the other hand, if the predetermined time has not passed since the time when the inquiry was made, the process proceeds to S814.

[0112] In Embodiment 2, the inquiry unit 217 changes the admission determination part in the trial participant record of the relevant admission trial participant from "on hold" to "permitted" only when a reply indicating that the person is the actual person is received within a predetermined time after sending the inquiry. Note that even if a reply is received within the predetermined time after sending the inquiry, if the reply is not from the actual person, that is, if the reply indicates that it is from someone else, the admission determination part in the trial participant record of the relevant admission trial participant is changed from "on hold" to "not permitted".

[0113] Note that the inquiry time is set to be included in the reply to the inquiry. Therefore, when sending the inquiry, the current time is also sent together. And the mobile device 23 may just include this time (the time when the inquiry was made) in the reply.

[0114] As described above, in Embodiment 2, in addition to the same effects as in Embodiment 1, by providing a time limit from the inquiry for identity confirmation to the reply, it is possible to provide an information processing system that is effective in preventing ticket reselling.

[0115] <Embodiment 3> In Embodiment 3, a method for enhancing the accuracy of the identity certainty by having the forgery detection unit 2122 perform forgery detection processing will be described. Forgery means, for example, pretending to be a registered person by hanging a photo of the registered person obtained in advance or a still image / moving image displayed on a tablet on the authentication camera, or wearing a 3D mask. Note that since the configurations and respective functional parts of the information processing system 100 including the first authentication device 21 and the second authentication device 22 and the mobile device 23 in Embodiment 3 are the same as those in Embodiment 1, the differences and processes from Embodiment 1 will be described below. Also, matters not mentioned in Embodiment 3 can be the same as in Embodiment 1.

[0116] The forgery detection process (detection technology) is a process (technology) for detecting that a person is forging an identity. Depending on the characteristics of interest, there are several methods. For example, when using a specific photo or tablet, things that should not originally exist, such as the edges of the photo or the edges of the tablet, are captured. Therefore, there is a method for detecting the presence or absence of these. Other methods include detecting non-living bodies such as photos and 3D masks. In this case, forgery is detected by paying attention to whether it is "a living thing". In any method, the output is a value that approaches 1 if it is a living body (human) and approaches 0 if it is a non-living body (non-human). In Embodiment 3, the forgery detection unit 2122 may use any method.

[0117] The following shows an example of five trial participant records held in the holding unit 218 of Embodiment 3 in Table 2. Table 2 shows the forgery probabilities of five trial participant records obtained by the forgery detection unit 2122. In Embodiment 3, the forgery probability is a value from 0 to 1, and it is assumed that the larger the value, the more likely it is considered to be a forgery.

Table 2

[0118] The identity certainty in Embodiment 3 is calculated from the forgery probability and the similarity. Specifically, similar to Embodiment 1, the extracted feature vector and the feature vectors of all pre-registered ticket purchasers are compared by the first matching unit 213, and the similarity (the first similarity) is calculated. Then, the forgery detection unit 2122 performs a forgery detection process to obtain a forgery probability (the second probability). Then, the first matching unit 213 calculates the identity certainty (the first probability) based on the similarity and the forgery probability. Note that the method for calculating the identity certainty in Embodiment 3 is not particularly limited, but it can be calculated, for example, by the following formula (1).

[0119] Identity certainty = (1 - forgery probability) × similarity 2 ···(1)

[0120] Table 2 shows the degree of self-identity calculated according to the above formula (1). Here, for example, in the same manner as in Embodiment 1, when the first threshold Th1 = 0.1 and the second threshold Th2 = 0.4, and the determination is made in the same manner as in Embodiment 1, the result is an admission determination as shown in Table 2. Thereby, the accuracy of the degree of self-identity can be improved.

[0121] As described above, in Embodiment 3, in addition to the same effects as in Embodiment 1, by calculating the degree of self-identity from the degree of forgery and the degree of similarity, it is possible to provide an information processing system that can further improve the accuracy of the degree of self-identity.

[0122] <Embodiment 4> In Embodiment 1, in the first collation unit 213, the extracted feature vector is collated with the feature vectors of all pre-registered ticket purchasers, and the degree of similarity (the first accuracy) is calculated by comparing them. Then, in the inquiry unit 217, an inquiry message for self-confirmation is sent to the target mobile device 23.

[0123] Here, when a plurality of very similar people are registered, even though the person taking the entry trial is registered, the registered ID with the highest degree of similarity may be different from the registered ID of the person taking the entry trial. For example, it may be found among very similar people such as twins. Therefore, in Embodiment 4, in order to avoid incorrect authentication due to such a phenomenon, the inquiry unit 217 sends an inquiry message to all or some of the registered persons with a degree of self-identity equal to or higher than a certain level.

[0124] Specifically, as shown in Table 3(A) below, assume that there are two registered IDs, registered ID1 and registered ID2, whose degree of self-identity is equal to or higher than the second threshold Th2 (0.4). Here, assume that the registered ID1 (degree of self-identity = 0.722) has the highest degree of self-identity, but the person taking the entry trial is the registered ID2 (degree of self-identity 0.692). In this case, an inquiry message is sent not only to the registered ID1 (120) but also to the registered ID2 (121).

Table 3

[0125] The processing flow in this Embodiment 4 will be described below with reference to FIGS. 10 and 11. Note that since the configurations and respective functional units of the information processing system 100 including the first authentication device 21 and the second authentication device 22 and the mobile device 23 in Embodiment 4 are the same as those in Embodiment 1, the differences and processes from Embodiment 1 will be described below. Also, matters not mentioned in Embodiment 4 can be the same as those in Embodiment 1.

[0126] FIGS. 10 and 11 are flowcharts showing examples of the processing (operations) of the first authentication device 21 in Embodiment 4. Each of the following processes (each operation) is realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, the notation of the steps is abbreviated by attaching an S at the beginning of each step (step).

[0127] In S1010, the inquiry unit 217 selects candidate persons. That is, when 0 < first threshold Th1 < second threshold Th2 ≤ third threshold Th3 < 1, if there are a plurality of persons with Th3 or more (Case 1), the inquiry unit 217 sets all of those persons as candidate persons. Further, when there are no persons with Th3 or more and there are persons with Th1 or more (Case 2), the inquiry unit 217 sets the person with the highest accuracy among them as the candidate person.

[0128] In S1011, the inquiry unit 217 determines whether it corresponds to Case 1 or 2. If it is determined that it does not correspond to Case 1 or 2, the processing in FIG. 11 ends. If it is determined that it corresponds to Case 1 or 2, the process proceeds to S1012.

[0129] In S1012, the inquiry unit 217 makes an inquiry to the communication terminal (portable device 23) similar to that in the first embodiment. Specifically, in the case of the above case 1, for all registered IDs whose personal accuracy is equal to or higher than the third threshold Th3, the portable device 23 owned by the person, which was registered in advance in the same manner as the processing in the first embodiment, is sent an inquiry message such as "Did you just pass by?" In the fourth embodiment, when the determination result is "pending" or "permitted", the inquiry unit 217 makes an inquiry to the portable device 23 owned by the registered ID with a predetermined accuracy or higher. On the other hand, in the case of the above case 2, an inquiry message as described above is sent to the portable device 23 corresponding to the registered ID with the highest personal accuracy among the registered IDs whose personal accuracy is equal to or higher than the first threshold Th1.

[0130] In S1013, the inquiry unit 217 determines whether a response to the inquiry message has been received. If it is determined that a response has been received, the process proceeds to S1014. In S1014, the inquiry unit 217 determines whether identity verification can be performed based on the response message. If identity verification can be performed, the process proceeds to S1017. In S1017, the inquiry unit 217 requests the update unit 216 to change the admission determination part in the trial record of the admission trial person from "pending" to "permitted", and the update unit 216 changes the part. In addition, the inquiry unit 217 deletes those other than the registered ID. Then the process ends.

[0131] Here, for example, pay attention to the person with the admission trial person ID of 6 in Table 3(A). Regarding the persons with the admission trial person IDs of 1 to 5 in Table 3(A), since the number of registered IDs with a personal accuracy equal to or higher than the second threshold Th2 is one or less, there is no description in the registered ID 2, similarity 2, and personal accuracy 2.

[0132] When a reply indicating that the registrant ID is the actual person is received from the registrant ID 121 (Registrant ID 2), if the registrant ID is other than 121, in this case, the content with the registrant ID 120 (Registrant ID 1) is deleted. This is shown in Table 3(B). Note that in Tables 3(B) to 3(D), the trial records of the trial participants with the trial participant IDs 1 to 5, which are not relevant to the description of this embodiment, are omitted. In Table 3(B), the values related to the registrant ID 120 are deleted, but the item itself may be deleted as in Table 3(C). Also, in order not to increase the items, an implementation may be made such that the feature vector, registrant ID, similarity, and authenticity probability are held as a list as in Table 3(D). Note that in Embodiment 4, it is not restricted to these implementations. Any implementation may be used as long as it deletes the registrant IDs other than the one that replied as the actual person.

[0133] On the other hand, in S1014, if the identity cannot be confirmed based on the reply message (that is, if it is not the actual person), the process proceeds to S1015. In S1015, the inquiry unit 217 determines whether all the reply messages have been received. If it is determined that all the response messages have been received, the process proceeds to S1016. In S1016, the inquiry unit 217 requests the update unit 216 to update the entry determination part in the trial record of the trial participant from "pending" to "not permitted", and the update unit 216 changes the part.

[0134] As described above, in Embodiment 4, in addition to the same effects as in Embodiment 1, even when a plurality of very similar people such as twins are registered, an information processing system that can link the trial participant to the correct registrant ID can be provided.

[0135] <Embodiment 5> In Embodiment 1, when a trial participant is determined to be "pending" by the first collation unit 213, there was no means other than inquiry to the actual person to correct this association. For example, if malicious Mr. A impersonates Mr. B and can enter as "pending". Here, when Mr. B himself enters, both Mr. A and Mr. B will be associated with the system as Mr. B.

[0136] When Mr. A enters, if Mr. B replies "someone else" to the inquiry, Mr. A will be "not permitted", but if Mr. B can also enter, the merit for Mr. B to make Mr. A "not permitted" is not necessarily large, and in that case, Mr. A will not be "not permitted". In some cases, it may be possible for multiple siblings or multiple people with similar faces to enter with a single registration.

[0137] However, there is often a significant difference in the degree of certainty of identity between the case where Mr. A impersonates Mr. B for authentication and the case where Mr. B himself authenticates. In Embodiment 5, a method for preventing multiple trial entrants from being associated with the same registrant in such cases will be described below with reference to FIGS. 12 to 14 and Table 4. Note that since the configuration and each functional part of the information processing system 100 including the first authentication device 21 and the second authentication device 22 and the mobile device 23 in Embodiment 5 are the same as those in Embodiment 1, the parts and processes different from those in Embodiment 1 will be described below. Also, matters not mentioned in Embodiment 5 can be the same as those in Embodiment 1.

[0138] FIGS. 12 to 14 are flowcharts showing examples of the processing (operations) of the first authentication device 21 in Embodiment 5. Note that the difference from the flowcharts shown in FIGS. 4 and 5 is that there are processes S1209 to S1213 in the flowcharts shown in FIGS. 12 to 14. That is, S1201 to S1208 are the same processes as S401 to S408 respectively, and S1214 to S1220 are the same processes as S409 to S415 respectively. Therefore, the description will be omitted below. The following respective processes (each operation) are realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, by prefixing each step with S, the notation of the step is omitted.

[0139] Table 4(A) shows two trialist records held in the holding unit 218 in a situation where the trialist with an admission trialist ID of 7 has already entered the main area 151 by a "permitted" determination, and the trialist with an admission trialist ID of 8 is about to enter just now.

Table 4

[0140] In S1209, the CPU 10 determines whether there is a trialist with the same registered ID. If there is a trialist with the same registered ID, the identity accuracy (first accuracy) of the trialist with the same registered ID is obtained respectively, and the process proceeds to S1210. On the other hand, if there is no trialist with the same registered ID, the process proceeds to S1214. Specifically, when another person having the same registered ID as the person determined to be on hold or permitted by the first collation unit 213 is determined to be on hold or permitted by the first collation unit 213, the identity accuracy of each person is obtained.

[0141] Here, an example of the process of S1209 will be described with reference to Table 4. At the time of determination, the registrant ID column of the determination result (the trialist record in Table 4(A) (admission trialist ID = 8)) is confirmed to obtain the registrant ID = 35. Then, it is searched from the holding unit 218 whether there is a trialist with the same registrant ID of 35. Then, a trialist record with an admission trialist ID = 7 is found. Here, since a trialist record with an admission trialist ID = 7 is found, the identity accuracy of the trialist records with admission trialist IDs of 7 and 8 is obtained. Then, from the trialist with an admission trialist ID of 7, an accuracy of 0.448 can be obtained, and from the trialist with an admission trialist ID of 8, an accuracy of 0.648 can be obtained.

[0142] In S1210, the first collation unit 213 calculates the difference in identity accuracy from each identity accuracy obtained in S1209. Then, it is determined whether the difference in identity accuracy is equal to or greater than a predetermined threshold (fourth threshold). If the difference in identity accuracy is equal to or greater than the predetermined threshold (fourth threshold), the process proceeds to S1211. On the other hand, if the difference in identity accuracy is less than the predetermined threshold (fourth threshold), the process proceeds to S1213.

[0143] Here, an example of the process of S1210 will be described when the probability of 0.448 is obtained when the ID of the entrant candidate is 7 or higher, and the probability of 0.648 is obtained when the ID of the entrant candidate is 8 or higher as described above. As the difference in the probability of the person, it becomes 0.648 - 0.448 = 0.2. Here, when the fourth threshold Th4, which is a predetermined threshold, is set to 0.1, it becomes 0.2 ≧ 0.1 = Th4. In this case, since the difference in the probability of the person is equal to or greater than the fourth threshold Th4, the process proceeds to S1211.

[0144] In S1211, the inquiry unit 217 changes the determination result on the side where the value of the probability of the person is small (the side where the probability of the person is low) to "pending" or "not permitted". That is, the entry determination of the ID of the entrant candidate in the target trial candidate record is rewritten to "pending" or "not permitted". In the case of the above example, the entry determination of the entrant candidate ID 7 with a probability of 0.448 is changed to "pending" or "not permitted". Here, whether to set it to "pending" or "not permitted" is determined according to the operation policy and varies depending on the situation, so the manufacturer of the device implementing the present invention or the person actually using the device may decide. In this process, it is set to "not permitted".

[0145] In S1212, the CPU 10 determines whether or not the entry determination of the target to be entered in S1211 has been changed to "pending". If the entry determination has been changed to "pending" in S1211, the process proceeds to S1214. On the other hand, if the entry determination has not been changed to "pending" in S1211, that is, if it has been changed to "not permitted", the process proceeds to S1207.

[0146] In S1213, the determination results of both targets are set to "pending". That is, the entry determination of the ID of the entrant candidate in each target trial candidate record is set to "pending".

[0147] In addition, when determining "pending" or "not permitted" for the entrant candidate with a low probability of the person, the inquiry unit 217 may make an inquiry to the person with the registered ID (in this case, 35). It is considered that this can perform a more accurate determination of the person.

[0148] In the above-described Embodiment 5, when both the person who has impersonated a specific registrant and the registrant himself / herself attempt to enter the venue, the impersonator is set to "not permitted" or "on hold". Therefore, when the impersonator attempts to receive a new service, the second authentication device 22 does not give "permission", and it becomes necessary to perform identity verification again. As a result, in addition to the same effects as in Embodiment 1, it is possible to provide an information processing system capable of reducing the benefits of non-registered persons impersonating others.

[0149] <Embodiment 6> In Embodiment 1, the feature vector of the trial participant record stored in the holding unit 218 is the feature vector registered in advance as a registrant. In Embodiment 6, a method will be described in which, instead of this feature vector, the feature vector calculated by the face feature calculation unit 2121 from the image at the time of entry trial (first image) is used. Note that since the configurations and respective functional units of the information processing system 100 including the first authentication device 21 and the second authentication device 22 and the mobile device 23 in Embodiment 6 are the same as those in Embodiment 1, the following will describe the parts and processes different from those in Embodiment 1. Also, matters not mentioned in Embodiment 6 can be the same as those in Embodiment 1.

[0150] When performing authentication with the second authentication device 22, generally, due to makeup, ornaments, aging, etc., the authentication accuracy may be higher when using the feature vector calculated from the image taken on the day rather than the feature vector registered in advance. Here, as described in Embodiment 1, the face authentication accuracy of the second collation unit 223 of the second authentication device 22 may be reduced by using a distillation technique or the like. Therefore, it may be inferior to the face authentication accuracy of the first collation unit 213 of the first authentication device 21. In such cases, the authentication accuracy can be increased by using the feature vector calculated from the image taken on the day rather than the feature vector registered in advance.

[0151] As described above, in Embodiment 6, the feature vector calculated by the first calculation unit 212, that is, the face feature calculation unit 2121, is described in the feature vector column of the trialist record. The second collation unit 223 compares the feature vectors of all the trialist records held by the holding unit 218, that is, the feature vectors calculated from the entrance images (first images) of all the entrance trialists, with the values of the feature vectors calculated by the second calculation unit 222, and calculates the second similarity.

[0152] As described above, in Embodiment 6, the calculation of the second similarity is performed using the feature vector extracted from the entrance trial image taken on the day, rather than the pre-registered feature vector, for the second similarity calculation. Thereby, in addition to the same effect as in Embodiment 1, it is possible to provide an information processing system capable of enhancing the accuracy of the second collation unit 223 in the second authentication device 22.

[0153] <Embodiment 7> In an event such as a food festival venue as described in Embodiment 1, the authentication at the time of entrance is performed once, but the authentication after entrance is performed multiple times. Also, in many cases, the entrance is made without the intervention of a person at the time of entrance, but the authentication after entrance may involve a person (a person on the organizer side). Therefore, the act of impersonating a registered person by hanging a photo of the registered person, a still image or a moving image displayed on a tablet on the authentication camera, or wearing a 3D mask is considered to be more frequently performed when there is no person on the organizer side than when there is.

[0154] That is, it is considered that the possibility of impersonation is lower in the authentication after entrance than at the time of entrance. Also, in the case of disguise by wearing a wig or makeup, etc., the similarity of the face feature vector and the impersonation probability may not be stable with the current technology.

[0155] Therefore, by recording the person accuracy each time at all authentication opportunities and observing the change in the history of the person accuracy, it is possible to determine whether or not it is the registered person.

[0156] Situation 1) The person accuracy is high only at the time of entrance and not so high thereafter Situation 2) There are large fluctuations in the certainty of the individual

[0157] When situations 1) or 2) as described above are observed, the possibility that it is not the registered individual himself / herself increases. In such cases, it is conceivable to hold off on making an entry determination for the person taking the entry trial, or to display an alert on the second display unit 224 during authentication by the second authentication device 22.

[0158] The processing flow in this Embodiment 7 will be described below with reference to FIGS. 15, 16, and Table 5. Note that since the configurations and respective functional units of the information processing system 100 including the first authentication device 21 and the second authentication device 22, and the mobile device 23 in Embodiment 7 are the same as those in Embodiment 1, the differences and processing from Embodiment 1 will be described below. Also, matters not mentioned in Embodiment 7 can be the same as those in Embodiment 1.

[0159] The certainty (certainty of the individual) calculated as a result of performing the process of S403 in FIG. 4 is the certainty of the individual at the time of entry (the first time). The authentication by the second authentication device 22 in Embodiment 7 is performed every time the service trial participant attempts to receive the service, and each time, the certainty of the individual is recorded in the trial participant record.

[0160] FIG. 15 is a flowchart showing an example of the processing (operation) of the second authentication device 22 in Embodiment 7. Note that the differences from the flowchart shown in FIG. 6 are that there are processes of S1505, S1506, and S1508 in the flowchart shown in FIG. 15. That is, S1501 to S1504 are the same processes as S601 to S604 respectively, and S1507 is the same process as S605. Therefore, the description will be omitted below. The following respective processes (each operation) are realized by the CPU 10 executing a program stored in the ROM 11 or the like. Also, the notation of the process (step) is omitted by prefixing each process (step) with S.

[0161] In the second authentication device 22, in the second matching unit 223, the second calculation unit 222 calculates the personal characteristics, and compares them with the values in the feature vector fields of all the trial participant records held by the holding unit 218 to calculate the second personal certainty. Then, the trial participant record with the highest second personal certainty is identified. Incidentally, when the second calculation unit 222 is configured in the same manner as the face feature calculation unit 2121, the calculated similarity (second similarity) becomes the second personal certainty.

[0162] In S1505, the CPU 10 adds the second personal certainty to the identified trial participant record and updates the trial participant record. Further, the updated trial participant record is saved in the holding unit 218.

[0163] Table 5(A) is an example in which the second personal certainty is added to the identified trial participant record several times. In Embodiment 7, each time the second authentication device 22 authenticates, the second personal certainty is added to the trial participant record. Incidentally, as shown in Table 5, in Embodiment 7, by having the personal certainty column in a list format, it is possible to have multiple values.

Table 5

[0164] Incidentally, having the personal certainty column in a list format is only an example, and it is not necessarily required to be in a list format. For example, columns such as "Personal Certainty 1" and "Personal Certainty 2" can be dynamically added to add the second personal certainty.

[0165] In S1506, the CPU 10 determines whether there is an abnormal variation in the person authentication accuracy. Specifically, it determines whether the above-described situation 1) or situation 2) exists. If there is no abnormal variation in the person authentication accuracy, the process proceeds to S1507. On the other hand, if there is an abnormal variation in the person authentication accuracy, the process proceeds to S1508. Note that it may be determined whether there is an abnormal variation in the person authentication accuracy by using a threshold value. For example, when the subsequent person authentication accuracy is less than the threshold value with respect to the person authentication accuracy at the time of entry, it may be considered that the situation 1) is satisfied and there is an abnormal variation in the person authentication accuracy. Also, for example, when the variation between the person authentication accuracies is equal to or greater than the threshold value, it may be considered that the situation 2) is satisfied and there is an abnormal variation in the person authentication accuracy.

[0166] Here, for example, in the case of Table 5(A), the person authentication accuracy is 0.648 at the time of entry, and then changes to 0.439, 0.412, 0.451, and 0.443 in order. FIG. 16 is a diagram for explaining a graph of the transition of the person authentication accuracy. FIG. 16(A) is a diagram for explaining a graph of the transition of the person authentication accuracy in the case of Table 5(A). Here, in the case of FIG. 16(A), it corresponds to the above-described situation 1). This is because the person authentication accuracy after the second time is uniformly lower than the first person authentication accuracy.

[0167] Note that in the seventh embodiment, since the person authentication accuracy may vary depending on the installation location of the second authentication device 22 due to the influence of lighting or the like depending on the location, in such a case, the person authentication accuracy is corrected in advance. As an example of the correction method, it is conceivable to correct for each location using a linear expression, but since it is a known method, the explanation is omitted. Also, as another solution, it is possible to omit the correction itself by making the lighting environments of the installation locations of the plurality of second authentication devices 22 similar to each other.

[0168] The determination at this time can be made by examining whether the expression (2) shown in FIG. 16 is satisfied when the person authentication accuracy at the nth time is Cn, the maximum value of the number of authentication times is M, and α (>1) is a predetermined constant. For example, in this case, when α = 1.4, 0.648 > 1.4 × 0.43625 = 0.61075, so it corresponds to the situation 1).

[0169] Also, another example is shown in Table 5(B). FIG. 16(B) is a diagram for explaining the transition of the person certainty in the case of Table 5(B) in the form of a graph. Here, the case like FIG. 16(B) corresponds to the above-described situation 2). This is because the variation between the person certainties is large and the person certainty is not stable.

[0170] The determination at this time can be made by examining whether or not the expression (3) shown in FIG. 16 is satisfied when 0 < β < 1 is a given constant. For example, in this case, if β = 0.2, then 0.648 - 0.421 = 0.227 > 0.2, so it corresponds to situation 2).

[0171] In S1507, without performing the alert display described later, a display as shown in FIG. 7 is performed on the screen of the second display unit 224 in the same manner as in the first embodiment. In S1508, the CPU 10 displays a predetermined alert on the second display unit 224. Specifically, an alert display such as "Possibility of impersonation, requiring confirmation of the person" as shown in FIG. 17 is displayed (notified) on the screen of the second display unit 224. FIG. 17 is a diagram showing an example of the display screen of the second display unit 224 of the second authentication device 22 in the seventh embodiment.

[0172] In addition, after adding the second person certainty to the identified trial participant record in S1505, if it is determined in S1506 that there is an abnormal variation in the person certainty, then the "entry determination" is further changed to "pending", and then the trial participant record is updated. The further updated trial participant record may be stored in the holding unit 218.

[0173] As described above, in the seventh embodiment, in addition to the same effects as in the first embodiment, by performing impersonation or the like, when an abnormality in the person certainty occurs, an information processing system that can reconfirm whether the person is the actual person can be provided.

[0174] As described above, the preferred embodiments of the present invention have been described in detail. However, the present invention is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0175] Moreover, by executing the program code read by the computer, not only the functions of the foregoing embodiments are realized. Based on the instructions of the program code, an operating system (OS) or the like running on the computer also performs part or all of the actual processing.

[0176] Furthermore, the functions of the foregoing embodiments are also realized in the following manner. The program code read from the recording medium is written into the memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instructions of the program code, a CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing.

[0177] When the present invention is applied to the above recording medium, the recording medium stores program codes corresponding to the respective flowcharts described above.

[0178] The disclosure of this embodiment includes the following system, method, and program.

[0179] (Configuration 1) An acquisition means for acquiring a first accuracy that the person is a pre-registered person by a collation process based on the characteristics of the person detected from the first image and the characteristics of a plurality of pre-registered persons registered in advance; A determination means for determining whether it is necessary to inquire of the pre-registered person based on the first accuracy acquired by the acquisition means, a first threshold value, and a second threshold value that is a threshold value larger than the first threshold value; An inquiry means for performing an inquiry to the contact associated with the pre-registered person based on the result of the determination by the determination means. An information processing system characterized by the above.

[0180] (Configuration 2) When the first accuracy is less than the first threshold value, the determination means determines that entry of the person into a predetermined area is not permitted. When the first accuracy is equal to or greater than the first threshold value but less than the second threshold value, the determination means determines that entry into the predetermined area is suspended and also determines that an inquiry to the pre-registered person is required. When the first accuracy is equal to or greater than the second threshold value, the determination means determines that entry into the predetermined area is permitted. The information processing system according to Configuration 1, characterized in that.

[0181] (Configuration 3) When the determination means determines that entry into the predetermined area is permitted or suspended, the determination means permits passage through a gate for entry into the predetermined area. When the determination means determines that entry is not permitted, the determination means does not permit passage through the gate. The information processing system according to Configuration 2, characterized in that.

[0182] (Configuration 4) The acquisition means acquires the first accuracy according to the degree of similarity between the feature vector of the person detected from the first image and the feature vector of the pre-registered person. The information processing system according to any one of Configurations 1 to 3, characterized in that.

[0183] (Configuration 5) The information processing system according to any one of Configurations 1 to 4, characterized in that it has a notification means for notifying the determination result of the determination means.

[0184] (Configuration 6) The acquisition means further acquires a second accuracy that the person is a pre-registered person by a collation process based on the features of the person after entering a predetermined area detected from a second image and the features of the pre-registered person. Based on the second accuracy, the determination means determines whether or not the person can pass through an authentication gate existing within the predetermined area. The information processing system according to any one of Configurations 1 to 4, characterized in that.

[0185] (Configuration 7) A holding means for holding the determination result of the determination means, a changing means for changing the determination result of the determination means; When there is a response indicating that the person is the actual person to the inquiry by the inquiry means, the changing means changes the determination result of the person held in the holding means from being on hold to being permitted, and when there is a response indicating that the person is not the actual person to the inquiry by the inquiry means, the changing means changes the determination result of the person held in the holding means from being on hold to being not permitted. The information processing system according to Configuration 2.

[0186] (Configuration 8) The notification means is at least one of a display device or a gate capable of physically permitting and rejecting the passage of a person. The information processing system according to Configuration 5.

[0187] (Configuration 9) Based on the second accuracy, there is a notification means for notifying the result of the determination by controlling at least one of a display device or a gate capable of physically permitting and rejecting the passage of a person. The information processing system according to Configuration 6.

[0188] (Configuration 10) When there is a response indicating that the person is the actual person to the inquiry by the inquiry means within a predetermined time, the changing means changes the determination result of the person held in the holding means from being on hold to being permitted. The information processing system according to Configuration 7.

[0189] (Configuration 11) The obtaining means calculates the first accuracy using a first similarity between the person and a pre-registered person obtained by a collation process based on the feature vector of the person and the feature vectors of the plurality of pre-registered persons, and a second accuracy indicating the accuracy of person impersonation. The information processing system according to any one of Configurations 1 to 5.

[0190] (Configuration 12) When the first accuracy is higher than the first threshold value, the inquiry means makes an inquiry to the contact information associated with each of two or more pre-registered persons whose probability of being the same person as the person among the plurality of pre-registered persons is equal to or higher than a predetermined threshold value. The information processing system according to any one of Configurations 1 to 11, characterized in that.

[0191] (Configuration 13) When a plurality of persons whose probability of being the same person as the pre-registered person is equal to or higher than the first threshold value are detected, and the difference between the respective first accuracies acquired by the acquisition means is equal to or higher than a third threshold value, the change means changes the determination result of the person with the lower first accuracy to hold or not permit. The information processing system according to Configuration 7, characterized in that.

[0192] (Configuration 14) When the difference between the respective first accuracies acquired by the acquisition means is less than the third threshold value, the change means holds the determination result of each person. The information processing system according to Configuration 13, characterized in that.

[0193] (Configuration 15) When the difference between the respective first accuracies acquired by the acquisition means is equal to or higher than the third threshold value, the inquiry means inquires whether the person is the registered person himself / herself to the contact information associated with the person with the lower first accuracy. The information processing system according to Configuration 14, characterized in that.

[0194] (Configuration 16) The acquisition means calculates the second accuracy based on the characteristics of the person after entering the predetermined area detected from the second image and the characteristics of the person detected from the first image. The information processing system according to Configuration 6, characterized in that.

[0195] (Configuration 17) The information processing system according to Configuration 6, further comprising notification means for notifying an alert when the first accuracy is higher than the second accuracy and the difference therebetween is larger than a predetermined value, or when the fluctuation of the second accuracy is equal to or higher than a threshold value.

[0196] (Configuration 18) A method for controlling an information processing system, comprising: an acquisition step of obtaining a first probability that the person is a pre-registered person by a collation process based on the characteristics of the person detected from the first image and the characteristics of a plurality of pre-registered persons registered in advance; a determination step of determining whether an inquiry to the pre-registered person is necessary based on the first probability obtained in the acquisition step, a first threshold value, and a second threshold value that is greater than the first threshold value; an inquiry step of performing an inquiry to the contact associated with the pre-registered person based on the result of the determination in the determination step. A method for controlling an information processing system, characterized by the above.

[0197] (Configuration 19) A program for causing a computer to execute an acquisition procedure of obtaining a first probability that the person is a pre-registered person by a collation process based on the characteristics of the person detected from the first image and the characteristics of a plurality of pre-registered persons registered in advance; execute a determination procedure of determining whether an inquiry to the pre-registered person is necessary based on the first probability obtained in the acquisition procedure, a first threshold value, and a second threshold value that is greater than the first threshold value; execute an inquiry procedure of performing an inquiry to the contact associated with the pre-registered person based on the result of the determination in the determination procedure.

Explanation of Signs

[0198] 21 First authentication device 22 Second authentication device 23 Mobile device 24 Network

Claims

1. An acquisition means for acquiring a first probability that the person is a pre-registered person by a collation process based on the characteristics of the person detected from the first image and the characteristics of a plurality of pre-registered persons registered in advance; A determination means for determining whether an inquiry to the pre-registered person is necessary based on the first probability acquired by the acquisition means, a first threshold value, and a second threshold value that is a threshold value larger than the first threshold value; An inquiry means for making an inquiry to the contact associated with the pre-registered person based on the result of the determination by the determination means. An information processing system characterized by the above.

2. When the first probability is less than the first threshold value, the determination means determines that entry of the person into a predetermined area is not permitted. When the first probability is equal to or greater than the first threshold value but less than the second threshold value, the determination means determines that entry into the predetermined area is suspended and that an inquiry to the pre-registered person is necessary. When the first probability is equal to or greater than the second threshold value, the determination means determines that entry into the predetermined area is permitted. The information processing system according to claim 1, characterized by the above.

3. When the determination means determines that entry into the predetermined area is permitted or suspended, the determination means permits passage through a gate for entry into the predetermined area. When the determination means determines that entry is not permitted, the determination means does not permit passage through the gate. The information processing system according to claim 2, characterized by the above.

4. The acquisition means acquires the first probability according to the similarity between the feature vector of the person detected from the first image and the feature vector of the pre-registered person. The information processing system according to claim 1, characterized by the above.

5. The information processing system according to claim 1, characterized by having a notification means for notifying the determination result of the determination means.

6. The acquisition means further acquires a second probability that the person is a pre-registered person by a collation process based on the characteristics of the person after entering a predetermined area detected from a second image and the characteristics of the pre-registered person. Based on the second probability, the determination means determines whether the person can pass through an authentication gate existing within the predetermined area. The information processing system according to claim 1, characterized by the above.

7. A holding means for holding the determination result of the determination means; An alteration means for altering the determination result of the determination means. When there is a reply indicating that the person is the actual person in response to the inquiry by the inquiry means, the change means changes the determination result of the person held in the holding means from on hold to permitted, and when there is a reply indicating that the person is not the actual person in response to the inquiry by the inquiry means, the change means changes the determination result of the person held in the holding means from on hold to not permitted. The information processing system according to claim 2, characterized in that.

8. The information processing system according to claim 5, characterized in that the notification means is at least one of a display device or a gate capable of physically permitting and rejecting the passage of a person.

9. Based on the second accuracy, the information processing system according to claim 6, characterized in that it has a notification means for notifying the result of the determination by controlling at least one of a display device or a gate capable of physically permitting and rejecting the passage of a person.

10. When there is a reply indicating that the person is the actual person in response to the inquiry by the inquiry means within a predetermined time, the change means changes the determination result of the person held in the holding means from on hold to permitted. The information processing system according to claim 7, characterized in that.

11. The acquisition means calculates the first accuracy using the first similarity between the person and the pre-registered persons obtained by a collation process based on the feature vector of the person and the feature vectors of the plurality of pre-registered persons, and the second accuracy indicating the accuracy of person impersonation. The information processing system according to claim 1, characterized in that.

12. When the first accuracy is higher than the first threshold value, the inquiry means makes an inquiry to the contact information associated with each of two or more pre-registered persons whose accuracy of being the same person as the person among the plurality of pre-registered persons is equal to or higher than a predetermined threshold value. The information processing system according to claim 1, characterized in that.

13. When a plurality of persons whose accuracy of being the same person as the pre-registered person is equal to or higher than the first threshold value are detected, and the difference between the first accuracies acquired by the acquisition means is equal to or higher than a third threshold value, the change means changes the determination result of the person with the lower first accuracy to on hold or not permitted. The information processing system according to claim 7, characterized in that.

14. The information processing system according to claim 13, wherein the changing means holds the determination result of each of the persons when the difference in the first accuracy obtained by the obtaining means is less than the third threshold value.

15. The information processing system according to claim 14, wherein the inquiry means inquires whether the person is the registered person to the contact associated with the person with the lower first accuracy when the difference in the first accuracy obtained by the obtaining means is greater than or equal to the third threshold value.

16. The information processing system according to claim 6, wherein the obtaining means calculates the second accuracy based on the characteristics of the person after entering the predetermined area detected from the second image and the characteristics of the person detected from the first image.

17. The information processing system according to claim 6, further comprising notification means for notifying an alert when the first accuracy is higher than the second accuracy and the difference is greater than a predetermined value, or when the fluctuation of the second accuracy is greater than or equal to a threshold value.

18. A control method for an information processing system, comprising: an obtaining step of obtaining a first accuracy that the person is a pre-registered person by a collation process based on the characteristics of the person detected from a first image and the characteristics of a plurality of pre-registered persons registered in advance; a determination step of determining whether an inquiry to the pre-registered person is necessary based on the first accuracy obtained in the obtaining step, a first threshold value, and a second threshold value that is greater than the first threshold value; an inquiry step of performing an inquiry to the contact associated with the pre-registered person based on the result of the determination in the determination step. The control method for an information processing system is characterized by the above.

19. A program for causing a computer to execute an obtaining procedure for obtaining a first accuracy that the person is a pre-registered person by a collation process based on the characteristics of the person detected from a first image and the characteristics of a plurality of pre-registered persons registered in advance; a determination procedure for determining whether an inquiry to the pre-registered person is necessary based on the first accuracy obtained in the obtaining procedure, a first threshold value, and a second threshold value that is greater than the first threshold value; an inquiry procedure for performing an inquiry to the contact associated with the pre-registered person based on the result of the determination in the determination procedure.

Images