Information processing device, information processing system, information processing method and program

Abstract

Multi-factor authentication using image recognition is preferably implemented. [Solution] An information processing device 10 includes a registration unit 11, an acquisition unit 12, an authentication control unit 13, and a gate control unit 14. The registration unit 11 registers first appearance information for registration in association with facial information for registration for each person to be registered, and updates the first appearance information for registration associated with the facial information for registration when a predetermined condition is met. The acquisition unit 12 acquires photographed image data of the person to be authenticated. The authentication control unit 13 performs personal authentication of the person to be authenticated based on a first determination result and a second determination result. The first determination result indicates whether the facial information for authentication generated based on the photographed image data corresponds to the facial information for registration. The second determination result indicates whether the first appearance information for authentication generated based on the photographed image data corresponds to the first appearance information for registration. The gate control unit 14 allows passage through the gate when personal authentication is successful.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing system, an information processing method, and a program. [Background technology]

[0002] In recent years, image recognition technology has advanced, and authentication using image recognition has attracted attention. Here, multi-factor authentication using image recognition has been proposed to ensure authentication accuracy while ensuring user convenience. For example, Patent Document 1 discloses an information processing device that performs face identification from a facial image of a person to be authenticated, recognizes a password generated based on the lip movements of the person to be authenticated, and performs personal authentication based on the results of the face identification and the password recognition. Patent Document 2 also discloses an image forming device that performs face recognition and gesture recognition, performs authentication based on the results of the face recognition and the gesture recognition, and starts secure printing if authentication is successful. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-203992 [Patent Document 2] Japanese Patent Application Publication No. 2016-018264 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technologies described in Patent Documents 1 and 2, if an authentication operation is performed in a place where there are people around, such as an entrance / exit gate, there is a possibility that the authentication operation may be discovered by those around. In addition, in such a case, if image recognition of multiple elements is performed with people around, there is a possibility that erroneous authentication may occur.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing device, an information processing method, and a program that suitably perform multi-factor authentication using image recognition. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present disclosure includes: a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of a person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; Equipped with.

[0007] An information processing system according to one aspect of the present disclosure includes: an authentication terminal that generates a photographed image of a person to be authenticated; an information processing device communicably connected to the authentication terminal; Equipped with. The information processing device includes: a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of the person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; It has.

[0008] An information processing method according to one aspect of the present disclosure includes: A step of registering first appearance information for registration in association with face information for registration for each person to be registered; acquiring photographed image data of a person to be authenticated; performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; allowing the person to pass through the gate if the personal authentication is successful; updating the first appearance information for registration associated with the face information for registration for each person to be registered when the person satisfies a predetermined condition; Equipped with.

[0009] A program according to one aspect of the present disclosure includes: On the computer, A process of registering first appearance information for registration in association with face information for registration for each person to be registered; A process of acquiring photographed image data of a person to be authenticated; a process of performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; If the personal authentication is successful, the process of allowing the person to pass through the gate is For each person to be registered, if a predetermined condition is satisfied, a process of updating the first appearance information for registration associated with the face information for registration; Execute the following. [Effects of the Invention]

[0010] The present disclosure can provide an information processing device, an information processing method, and a program that suitably perform multi-factor authentication using image recognition. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment. [Figure 2] FIG. 2 is a diagram showing the flow of an information processing method according to the first embodiment. [Figure 3] FIG. 10 is a block diagram showing the overall configuration of an information processing system according to a second embodiment. [Figure 4] FIG. 10 is a block diagram showing the configuration of a face authentication device according to a second embodiment. [Figure 5] 10 is a flowchart showing the flow of face information registration processing according to the second embodiment. [Figure 6] 10 is a flowchart showing the flow of face authentication processing according to the second embodiment. [Figure 7] FIG. 10 is a block diagram showing the configuration of an appearance authentication device according to a second embodiment. [Figure 8] 10 is a flowchart showing the flow of appearance authentication processing according to the second embodiment. [Figure 9] FIG. 10 is a block diagram showing the configuration of an authentication terminal according to a second embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of a user terminal according to the second embodiment. [Figure 11] FIG. 10 is a block diagram showing the configuration of an information processing device according to a second embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a data structure of registration history information according to the second embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a data structure of authentication history information according to the second embodiment. [Figure 14] 10 is a flowchart showing the flow of a registration process according to the second embodiment. [Figure 15] 10 is a flowchart showing the flow of personal authentication processing according to the second embodiment. [Figure 16] 10 is a flowchart showing the flow of appearance information update processing according to the second embodiment. [Figure 17] FIG. 10 is a sequence diagram showing the flow of a registration process according to the second embodiment. [Figure 18] FIG. 10 is a diagram illustrating an example of a display on a user terminal according to the second embodiment. [Figure 19] FIG. 10 is a sequence diagram showing the flow of personal authentication processing according to the second embodiment. [Figure 20] FIG. 10 is a diagram for explaining an example of personal authentication processing according to the second embodiment. [Figure 21] FIG. 10 is a diagram for explaining an example of personal authentication processing according to the second embodiment. [Figure 22] FIG. 10 is a diagram for explaining an example of personal authentication processing according to the second embodiment. [Figure 23] FIG. 10 is a sequence diagram showing the flow of an update process according to the second embodiment. [Figure 24] FIG. 10 is a diagram illustrating an example of a display on a user terminal according to the second embodiment. [Figure 25] FIG. 10 is a block diagram showing the configuration of an appearance authentication device according to a third embodiment. [Figure 26] 11 is a flowchart showing the flow of appearance authentication processing according to the third embodiment. [Figure 27] FIG. 11 is a diagram illustrating an example of a data structure of registration history information according to the third embodiment. [Figure 28] 11 is a flowchart showing the flow of element number addition processing according to the third embodiment. [Figure 29] 11 is a flowchart showing the flow of personal authentication processing according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0013] <Embodiment 1> First, a first embodiment of the present disclosure will be described. FIG. 1 is a block diagram showing the configuration of an information processing device 10 according to the first embodiment. The information processing device 10 is an information processing device that performs multi-factor authentication. Here, the information processing device 10 is connected to a network (not shown). The network may be wired or wireless. Also connected to the network are a gate driving device (not shown) that opens and closes the gate, and an authentication terminal (not shown) that is installed near the gate and captures an image of a person to be authenticated and generates a captured image. In other words, the information processing device 10 is connected to the gate driving device and the authentication terminal via the network so as to be able to communicate with them. Note that the captured image includes at least the facial area of ​​the person to be authenticated.

[0014] The information processing device 10 includes a registration unit 11, an acquisition unit 12, an authentication control unit 13, and a gate control unit .

[0015] The registration unit 11 registers registration appearance information associated with registration face information for each person to be registered. Here, the appearance information is also called first appearance information, and is information that indicates the appearance of the person to be authenticated. It is preferable that the appearance of the appearance information can be easily changed by the person to be authenticated at their own will. For example, the appearance information may be movement information that indicates the gestures (movements) of the person to be authenticated, or clothing information that indicates the clothing of the person to be authenticated. Then, the registration unit 11 updates the registration appearance information associated with the registration face information for each person to be registered when a predetermined condition is satisfied.

[0016] The acquisition unit 12 acquires photographed image data of the person to be authenticated from the authentication terminal via the network, and then supplies the photographed image data to the authentication control unit 13.

[0017] The authentication control unit 13 performs personal authentication of the person to be authenticated based on the first and second judgment results based on the captured image data. Here, the first judgment result indicates whether or not the face information for authentication generated based on the captured image data corresponds to the face information for registration. Therefore, the first judgment result may be called a face authentication result or a face judgment result. The second judgment result indicates whether or not the appearance information for authentication generated based on the captured image data corresponds to the appearance information for registration. Therefore, the second judgment result may be called an appearance authentication result or an appearance judgment result. The authentication control unit 13 supplies the result of the personal authentication to the gate control unit 14.

[0018] If personal authentication is successful, the gate control unit 14 allows the person to be authenticated to pass through the gate. For example, if personal authentication is successful, the gate control unit 14 sends a control signal to the gate drive device via the network to open the gate. Also, for example, if personal authentication fails, the gate control unit 14 sends a control signal to the gate drive device via the network to close the gate. Also, for example, if personal authentication is successful, the gate control unit 14 sends a control signal to the gate drive device via the network to output (for example, display or audio output) a message indicating that passage through the gate is permitted. Also, for example, if personal authentication fails, the gate control unit 14 sends a control signal to the gate drive device via the network to output a message indicating that passage through the gate is restricted.

[0019] FIG. 2 is a diagram showing the flow of the information processing method according to the first embodiment. First, the registration unit 11 registers appearance information of the person to be registered in association with the face information of the person to be registered (S10). Next, the acquisition unit 12 acquires photographed image data of the person to be authenticated from the authentication terminal via the network (S11). Next, the authentication control unit 13 performs personal authentication based on the face authentication result and the appearance authentication result (S12) and determines whether the personal authentication is successful or not (S13). Specifically, the authentication control unit 13 determines that the personal authentication is successful if both the face authentication and the appearance authentication are successful. On the other hand, the authentication control unit 13 determines that the personal authentication is unsuccessful if either one of them fails. If the authentication control unit 13 determines that the personal authentication is successful (Yes in S13), the gate control unit 14 allows the person to be authenticated to pass through the gate (S14), and the process proceeds to step S15. On the other hand, if the authentication control unit 13 determines that the personal authentication has failed (No in S13), the process proceeds to step S15. In step S15, the registration unit 11 determines whether or not a predetermined condition has been met (S15). If the registration unit 11 determines that the predetermined condition has been met (Yes in S15), it updates the appearance information associated with the face information of the person to be registered to new appearance information (S16).

[0020] In this way, the information processing device 10 according to the first embodiment performs multi-factor authentication using face authentication and appearance authentication using image recognition, and updates the appearance information related to the appearance authentication when a predetermined condition is met. Therefore, while improving the authentication accuracy through multi-factor authentication, it is possible to reduce the risk of personal information, such as appearance information, being leaked, thereby increasing the security level. This is particularly effective when personal authentication is performed in a place where people are around. As a result, the information processing device 10 can suitably perform multi-factor authentication using image recognition.

[0021] The information processing device 10 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program that implements the processing of the information processing method according to this embodiment. The processor then loads the computer program from the storage device into the memory and executes the computer program. This allows the processor to implement the functions of a registration unit 11, an acquisition unit 12, an authentication control unit 13, and a gate control unit 14.

[0022] Alternatively, the registration unit 11, acquisition unit 12, authentication control unit 13, and gate control unit 14 may each be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and programs. Furthermore, a CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (Field-Programmable Gate Array), etc., may be used as the processor.

[0023] Furthermore, when some or all of the components of the information processing device 10 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, in a form in which each is connected via a communication network. Furthermore, the functions of the information processing device 10 may be provided in a SaaS (Software as a Service) format.

[0024] <Embodiment 2> Next, a second embodiment of the present disclosure will be described. Fig. 3 is a block diagram showing the overall configuration of an information processing system 1000 according to the second embodiment. The information processing system 1000 is a computer system that permits or restricts a user U, who is a target of authentication, from passing through a gate by multi-factor authentication using image recognition. The information processing system 1000 includes a face authentication device 100, an appearance authentication device 200, an information processing device 300, authentication terminals 400-1 to 400-n (n is a natural number equal to or greater than 1), a user terminal 500, and gate driving devices 600-1 to 600-n. The devices and terminals are connected to each other via a network N. Here, the network N is a wired or wireless communication line.

[0025] Each of the authentication terminals 400-1 to 400-n is installed near a gate A1, A2, ..., An so as to be able to capture at least the face of a user U attempting to pass through the gate. The gates are entrance / exit gates. The gates may function, for example, as station ticket gates, theme park entrance / exit gates, or event venue entrance / exit gates. Some of the gates A1, A2, ..., An may function as station ticket gates, and others as theme park entrance gates. When the gates function as station ticket gates, the gates may be installed at different stations, or some gates may be installed at the same station. Note that the authentication terminals 400-1 to 400-n are preferably installed so as to be able to capture at least the face of the user U by walk-through. Each of the gate driving devices 600-1 to 600-n controls the opening and closing of the gates A1, A2, ..., An. Alternatively, each of the gate driving devices 600-1 to 600-n outputs a signal to permit or restrict user U from passing through gates A1, A2, ..., An. Hereinafter, when the authentication terminals 400-1, 400-2, ..., 400-n are referred to without distinction, they may be simply referred to as authentication terminals 400. Furthermore, when the gate driving devices 600-1, 600-2, ..., 600-n are referred to without distinction, they may be simply referred to as gate driving devices 600.

[0026] A user U, who is a person to be registered, registers in advance to use the service using the user terminal 500 and registers his or her facial information. Each user U also registers appearance information in advance, or the appearance information is registered automatically. As described above, the appearance information is movement information indicating the user U's movements or clothing information indicating the user U's clothing. For example, if the appearance information is movement information, the user U performs a predetermined movement when entering at gate A1. Then, personal authentication is performed using image recognition. If the personal authentication is successful, the user U can pass through gate A1. Furthermore, the user U performs a predetermined movement when exiting at gate A2. Then, personal authentication is performed using image recognition. If the personal authentication is successful, the user U can pass through gate A2. Note that the entry and exit of the user U through the above personal authentication is recorded, and thus the user U may be able to automatically pay transportation fares and admission fees without having to present a magnetic card, IC card, or barcode.

[0027] Here, the face recognition device 100 is an information processing device that stores facial feature information of multiple people. In addition, in response to a face recognition request received from the outside, the face recognition device 100 compares the face image or facial feature information included in the request with the facial feature information of each user, and returns the comparison result (face recognition result) to the request source.

[0028] FIG. 4 is a block diagram showing the configuration of a face authentication device 100 according to the second embodiment. The face authentication device 100 includes a face information database (DB) 110, a face detection unit 120, a feature point extraction unit 130, a registration unit 140, and an authentication unit 150. The face information DB 110 stores a user ID 111 and facial feature information 112 corresponding to the user ID in association with each other. The facial feature information 112 is a collection of feature points extracted from a face image and is an example of face information. The face authentication device 100 may delete the facial feature information 112 from the facial feature DB 110 in response to a request from a registered user of the facial feature information 112. Alternatively, the face authentication device 100 may delete the facial feature information 112 after a certain period of time has elapsed since the registration of the facial feature information 112.

[0029] The face detection unit 120 detects a face area included in a registration image for registering face information and supplies the detected face area to the feature point extraction unit 130. The feature point extraction unit 130 extracts feature points from the face area detected by the face detection unit 120 and supplies face feature information to the registration unit 140. The feature point extraction unit 130 also extracts feature points included in a face image received from the information processing device 300 and supplies the face feature information to the authentication unit 150.

[0030] The registration unit 140 issues a new user ID 111 when registering facial feature information. The registration unit 140 associates the issued user ID 111 with facial feature information 112 extracted from the registered image and registers them in the facial information DB 110. The authentication unit 150 performs facial authentication using the facial feature information 112. Specifically, the authentication unit 150 compares the facial feature information extracted from the facial image with the facial feature information 112 in the facial information DB 110. The authentication unit 150 returns a match of the facial feature information to the information processing device 300. The match of the facial feature information corresponds to the success or failure of authentication. The match of the facial feature information (match present) refers to a case where the degree of match is equal to or greater than a predetermined value.

[0031] FIG. 5 is a flowchart showing the flow of face information registration processing according to the second embodiment. First, the face authentication device 100 acquires a registration image of the user U included in a face information registration request (S21). For example, the face authentication device 100 receives the face information registration request from the user terminal 500 via the network N. Note that the source of the face information registration request is not limited to this, and may be the information processing device 300 that received the user registration request from the user terminal 500. Next, the face detection unit 120 detects a face area included in the registration image (S22). Next, the feature point extraction unit 130 extracts feature points from the face area detected in step S22 and provides facial feature information to the registration unit 140 (S23). Finally, the registration unit 140 issues a user ID 111 and associates the user ID 111 with facial feature information 112, and registers the user ID 111 in the face information DB 110 (S24). The face authentication device 100 may receive the face feature information 112 from the source of the face information registration request, and register it in the face information DB 110 in association with the user ID 111.

[0032] FIG. 6 is a flowchart showing the flow of face authentication processing by the face authentication device 100 according to the second embodiment. First, the feature point extraction unit 130 acquires facial feature information for authentication (S31). For example, the face authentication device 100 receives a face authentication request from the information processing device 300 via the network N and extracts facial feature information from the captured image included in the face authentication request as in steps S21 to S23. Alternatively, the face authentication device 100 may receive the facial feature information from the information processing device 300. Next, the authentication unit 150 compares the acquired facial feature information with the facial feature information 112 in the face information DB 110 (S32). If the facial feature information matches, that is, if the degree of match between the facial feature information is equal to or greater than a predetermined value (Yes in S33), the authentication unit 150 identifies the user ID 111 of the user whose facial feature information matches (S34). Then, the authentication unit 150 returns a message indicating that face authentication was successful and the identified user ID 111 as a face authentication result to the information processing device 300 (S35). At this time, if there are multiple user IDs 111 whose facial feature information matches a predetermined value or more, the authentication unit 150 returns a facial authentication result including a message that facial authentication was successful and the multiple user IDs 111 to the information processing device 300. The facial authentication result may also include information that identifies the detected facial area, for example, position information of the facial area in the captured image. If there is no matching facial feature information (No in S33), the authentication unit 150 returns a message that facial authentication failed to the information processing device 300 as the facial authentication result (S36).

[0033] In step S32, the authentication unit 150 does not need to attempt matching with all of the facial feature information 112 in the facial information DB 110. For example, the authentication unit 150 may preferentially attempt matching with facial feature information registered during the period from the day the facial authentication request was received to several days prior. This may improve the matching speed. Furthermore, if the preferential matching fails, it may be possible to perform matching with all of the remaining facial feature information.

[0034] Returning to Fig. 3, the explanation will be continued. The appearance authentication device 200 is an information processing device that stores appearance feature information for each appearance (each movement and each outfit). In addition, in response to an appearance authentication request received from the outside, the appearance authentication device 200 compares the person image or appearance feature information included in the request with the stored appearance feature information, and returns the comparison result (appearance authentication result) to the request source.

[0035] 7 is a block diagram showing the configuration of an appearance authentication device 200 according to the second embodiment. The appearance authentication device 200 includes an appearance information DB 210, a detection unit 220, a feature point extraction unit 230, a registration unit 240, and an authentication unit 250. The appearance information DB 210 stores an appearance ID 211 and appearance feature information 212 corresponding to the appearance ID in association with each other. The appearance ID 211 is information that identifies the appearance of the authentication target. The appearance feature information 212 is a collection of feature points extracted from an image of a person having an appearance corresponding to the appearance ID. The appearance ID 211 and the appearance feature information 212 are sometimes referred to as appearance information.

[0036] The detection unit 220 detects a person area included in a sample image for registering appearance information or a captured image acquired from the information processing device 300, and supplies the detected person area to the feature point extraction unit 230. The feature point extraction unit 230 extracts feature points from the person area detected by the detection unit 220, and supplies appearance feature information to the registration unit 240. The feature point extraction unit 230 also extracts feature points included in the captured image received from the information processing device 300, and supplies the appearance feature information to the authentication unit 250.

[0037] The registration unit 240 issues a new appearance ID 211 when registering appearance feature information. The registration unit 240 associates the issued appearance ID 211 with appearance feature information 212 extracted from the sample image and registers them in the appearance information DB 210. The authentication unit 250 performs appearance authentication using the appearance feature information 212. Specifically, the authentication unit 250 acquires the appearance ID registered by the user U as an authentication target from the information processing device 300, and compares the appearance feature information 212 stored in the appearance information DB 210 in association with the acquired appearance ID with the appearance feature information extracted from the captured image. The authentication unit 250 returns a match between the appearance feature information and the information processing device 300. The match between the appearance feature information and the appearance feature information corresponds to the success or failure of authentication. Note that the appearance feature information matches (match exists) when the degree of match is equal to or greater than a predetermined value.

[0038] FIG. 8 is a flowchart showing the flow of appearance authentication processing according to the second embodiment. First, the detection unit 220 acquires an appearance authentication request from the information processing device 300 via the network N (S51). The appearance authentication request includes a captured image of the user U and an appearance ID registered as an authentication target by the user U whose face has been successfully authenticated. The appearance authentication request may also include information identifying the facial region of the user U whose face has been successfully authenticated. The detection unit 220 then identifies a human region included in the captured image as the detection region (S52). For example, the detection unit 220 may identify a predetermined region of the human region as the detection region according to the acquired appearance ID. Furthermore, when the appearance authentication request includes information identifying the facial region of the user U, the detection unit 220 may identify a region corresponding to the facial region of the user U, for example, a region within a predetermined distance (a predetermined number of pixels) from the facial region, as the detection region. This allows the appearance of the person whose face has been successfully authenticated to be identified even when multiple people are captured in the captured image, thereby avoiding erroneous authentication. The feature point extraction unit 230 then extracts appearance feature information from the detection area (S53). Alternatively, the appearance authentication device 200 may receive the appearance feature information from the information processing device 300, in which case the processes of S51 to S53 are omitted. Next, the authentication unit 250 compares the acquired appearance feature information with the appearance feature information 212 associated with the acquired appearance ID in the appearance information DB 210 (S54). If the appearance feature information matches, that is, if the degree of match of the appearance feature information is equal to or greater than a predetermined value (Yes in S55), the authentication unit 250 replies to the information processing device 300 that appearance authentication was successful (S56). At this time, the authentication unit 250 may also transmit the appearance ID and user ID to the information processing device 300. On the other hand, if the appearance feature information does not match (No in S55), the authentication unit 250 replies to the information processing device 300 that appearance authentication failed (S57).

[0039] Continuing the explanation, returning to Figure 3, each of the authentication terminals 400-1, 400-2, ..., 400-n is an information terminal including a camera and a display device.

[0040] The authentication terminal 400 takes an image for authentication to be used for personal authentication of the user U. For example, the authentication terminal 400 takes an image of the user U at each gate installed therein and uses the image for authentication. The authentication terminal 400 transmits a personal authentication request including the authentication image to the information processing device 300 via the network N. At this time, the authentication terminal 400 may include a gate ID that identifies the gate where the authentication terminal 400 is installed in the personal authentication request. The authentication terminal 400 may also include the time of the image capture in the personal authentication request. The authentication terminal 400 may receive the personal authentication result from the information processing device 300 via the network N and may display this information on a screen as necessary.

[0041] Next, the authentication terminal 400 will be described in detail. Fig. 9 is a block diagram showing the configuration of the authentication terminal 400 according to embodiment 2. The authentication terminal 400 includes a camera 410, a storage unit 420, a communication unit 430, a display unit 440, and a control unit 450.

[0042] The camera 410 is an imaging device that takes images under the control of the control unit 450. The memory unit 420 is a storage device that stores programs for realizing each function of the authentication terminal 400. The communication unit 430 is a communication interface with the network N. The display unit 440 is a display device. The display unit 440 may be configured integrally with an input unit (not shown). As an example, the display unit 440 is a touch panel. The control unit 450 controls the hardware of the authentication terminal 400. The control unit 450 includes an imaging control unit 451, an authentication control unit 453, and a display control unit 454. Note that the display unit 440 and the display control unit 454 are not essential.

[0043] The photographing control unit 451 controls the camera 410 to take an authentication image of the user U. The authentication image is an image that includes at least the face area of ​​the user. The photographing control unit 451 also supplies the authentication image to the authentication control unit 453.

[0044] The authentication control unit 453 transmits a personal authentication request including an authentication image to the information processing device 300 via the network N, and receives a personal authentication result. The authentication control unit 453 may then supply the personal authentication result to the display control unit 454. The display control unit 454 may display content corresponding to the personal authentication result on the display unit 440.

[0045] Returning to Fig. 3, the explanation will continue. The user terminal 500 is an information terminal used by the user U. The user terminal 500 is, for example, a mobile phone terminal, a smartphone, a tablet terminal, a PC (Personal Computer) equipped with or connected to a camera, etc. The user terminal 500 is associated with the user ID of the user U. In other words, the user terminal 500 is an information terminal that can be identified by the user ID in the information processing device 300. For example, the user terminal 500 is a terminal into which the user U has already logged in using his or her own user ID.

[0046] The user terminal 500 transmits a service registration request to the information processing device 300 via the network N. The user terminal 500 also transmits a registration image to be used for facial authentication of the user U to the face authentication device 100, and requests facial information registration. The user terminal 500 may also transmit facial feature information extracted from the registration image to the face authentication device 100, and request facial information registration. The user terminal 500 may also transmit the registration image and facial feature information to the face authentication device 100 via the information processing device 300. The user terminal 500 also transmits attribute information and appearance information for registration of the user U to the information processing device 300 via the network N, for example, when registering for the service.

[0047] Next, the user terminal 500 will be described in detail. Fig. 10 is a block diagram showing the configuration of the user terminal 500 according to the second embodiment. The user terminal 500 includes a camera 510, a storage unit 520, a communication unit 530, a display unit 540, a control unit 550, and an input unit 560.

[0048] The camera 510 is an imaging device that takes images under the control of the control unit 550. The memory unit 520 is a storage device that stores programs for realizing each function of the user terminal 500. The communication unit 530 is a communication interface with the network N. The display unit 540 is a display device. The input unit 560 is an input device that accepts input. The display unit 540 and the input unit 560 may be configured integrally. As an example, the display unit 540 and the input unit 560 are touch panels. The control unit 550 controls the hardware of the user terminal 500. The control unit 550 includes an imaging control unit 551, a registration unit 552, an acquisition unit 553, and a display control unit 554.

[0049] The photographing control unit 551 controls the camera 510 to photograph a registration image of the user U. The photographing control unit 551 outputs the registration image to the registration unit 552.

[0050] The registration unit 552 transmits a face information registration request including a registration image to the face authentication device 100 via the network N. The registration unit 552 may transmit the face information registration request to the information processing device 300 via the network N. The registration unit 552 also transmits a service registration request to the information processing device 300 via the network N. The registration unit 552 may transmit user attribute information received by the input unit 560 at the time of user registration to the information processing device 300 via the network N. The user attribute information may be included in the user registration request.

[0051] The acquisition unit 553 acquires one or more pieces of appearance information (appearance ID) set for the user U from the information processing device 300 via the network N, and outputs the information to the display control unit 554. The display control unit 554 displays the appearance information on the display unit 540. When the acquisition unit 553 acquires multiple pieces of appearance information from the information processing device 300, the display control unit 554 displays the multiple pieces of appearance information on the display unit 540 so that the user U can select one. In this case, the registration unit 552 transmits the appearance ID selected by the user U, which is received from the user U by the input unit 560, to the information processing device 300 via the network N.

[0052] Returning to Figure 3, the explanation will continue. The information processing device 300 is an information processing device that performs personal authentication using an image captured of the user U at the gate A1 or the like, and permits or restricts the user U from passing through the gate. The information processing device 300 may be redundantly configured with multiple servers, and each functional block may be realized by multiple computers.

[0053] Next, the information processing device 300 will be described in detail. Fig. 11 is a block diagram showing the configuration of the information processing device 300 according to the second embodiment. The information processing device 300 includes a storage unit 310, a memory 320, a communication unit 330, and a control unit 340. The storage unit 310 is a storage device such as a hard disk or a flash memory. The storage unit 310 stores a program 311, registration history information 313, authentication history information 314, gate information 315, and appearance-related information 316. The program 311 is a computer program that implements the processing of the information processing method according to the second embodiment.

[0054] The registration history information 313 is history information of the registration of user attribute information and appearance information (appearance ID). FIG. 12 illustrates an example of the data structure of the registration history information 313 according to the second embodiment. As illustrated in this figure, the registration history information 313 is information that associates a user ID, a registration or update date and time, user attribute information, an appearance group ID, and an appearance ID for registration. The user ID is information that identifies the user U and is notified when face information is registered in the face authentication device 100. The user attribute information may include at least one of the user U's residential area, the nearest station, a planned station to be used, and a planned line to be used. The user attribute information may also include a purpose of use. For example, the purpose of use may be railway use, theme park use, event venue use, or commuter pass use, one-day pass use, or one-time use. The user attribute information may also include personal payment information of the user U, such as the user U's name, address, contact information, and credit card information. The appearance group ID is an ID that identifies an appearance group and is also referred to as appearance information. An appearance group includes multiple appearance IDs and is also referred to as an appearance information group. The appearance ID is one registration appearance ID selected by the user U from the appearance IDs included in the appearance group to which the user belongs. When the appearance type is an action, the appearance ID is also called an action ID, and may be, for example, "raise left hand," "raise right hand," or "touch ear, nose, and mouth in that order." When the appearance type is clothing, the appearance ID is also called an clothing ID, and may be, for example, "wearing red clothing," "carrying blue luggage," "wearing a hat," or "wearing a red wristband on the right hand." The registration appearance ID, i.e., the registration appearance information, is associated with the facial feature information 112 (registration face information) in the face information DB 110 of the face recognition device 100 via the user ID. Note that the registration history information 313 in this figure is configured so that each user U is associated with one registration appearance information, but the user U may be associated with different appearance information for each usage scenario. For example, the user U may be assigned different registration appearance information for each planned station. The user U may also be assigned different registration appearance information for each usage purpose.

[0055] The authentication history information 314 is history information of personal authentication at a gate. FIG. 13 is a diagram showing an example of the data structure of the authentication history information 314 according to the second embodiment. As shown in this diagram, for example, the authentication history information 314 is information in which a user ID, a photographing date and time, and a gate ID are associated with each other. The user ID is a user ID whose face has been successfully authenticated during personal authentication. The photographing date and time may be the date and time when personal authentication was performed. The gate ID is information for identifying a gate through which the user U is scheduled to pass at the time of personal authentication.

[0056] Returning to FIG. 11, the explanation will be continued. Gate information 315 is information that associates gate ID 3151 with gate attribute information 3152. Gate attribute information 3152 is attribute information of the gate, and may include the facility type where the gate is installed and location information where the gate is installed. For example, if the gate functions as a ticket gate at a station, the facility type may be "railway" and the location information may be "Shinjuku Station." Also, if the gate functions as an entrance / exit gate for a theme park, the facility type may be "theme park" and the location information may be "main gate." Note that the location information may include location information indicated by longitude and latitude.

[0057] The appearance-related information 316 is information that associates an appearance group ID 3162 with an appearance ID 3161. The appearance group ID 3162 is made up of a plurality of appearance IDs 3161. Therefore, the appearance group ID 3162 is associated with a plurality of appearance IDs 3161.

[0058] The memory 320 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily storing information during operation of the control unit 340. The communication unit 330 is a communication interface with the network N.

[0059] The control unit 340 is a processor, that is, a control device, that controls each component of the information processing device 300. The control unit 340 loads the program 311 from the storage unit 310 into the memory 320 and executes the program 311. In this way, the control unit 340 realizes the functions of a registration unit 341, an acquisition unit 342, an authentication control unit 343, and a gate control unit 344.

[0060] The registration unit 341 is an example of the above-mentioned registration unit 11. When face information of the user U is registered by the face authentication device 100 at the time of service usage registration and the user ID is notified, the registration unit 341 registers the user ID as a user ID in the registration history information 313 in the storage unit 310. Furthermore, when the registration unit 341 acquires user attribute information from the user terminal 500, the registration unit 341 registers the acquired user attribute information in the storage unit 310 as user attribute information in the registration history information 313 in association with the user ID.

[0061] The registration unit 341 then sets an appearance group, i.e., multiple appearance IDs as options for the appearance information to be registered, for the registered user U. Setting an appearance group may mean selecting one appearance group from predetermined appearance groups. Setting an appearance group may also include setting multiple appearance IDs as options, configuring an appearance group to include these IDs, and issuing an appearance group ID. When the registration unit 341 configures a new appearance group, it registers an appearance group ID 3162 and an appearance ID 3161 as the appearance-related information 316. Then, the registration unit 341 outputs the appearance IDs included in the appearance group to the user terminal 500 via the network N. At this time, the registration unit 341 displays each appearance ID included in the appearance group for each user U on the display unit 540 of the user terminal 500 so that the user U can select it. When the registration unit 341 receives an appearance ID selected by the user U from the user terminal 500, it registers the set appearance group ID and the appearance ID selected by the user U as the registration history information 313.

[0062] Here, the registration unit 341 may set an appearance group for each user U based on the user attribute information registered in the registration history information 313. For example, if the planned stations include stations with a predetermined number of visitors or more, the registration unit 341 may set the appearance group to include an appearance ID indicating a complex appearance. A complex appearance may be a combination of multiple single actions, such as "touching ears, nose, and mouth in that order," or an outfit that combines multiple items, such as "wearing a yellow wristband with red clothing." Alternatively, in this case, the registration unit 341 may configure the appearance group so that the number of appearance IDs included as options is greater than that in a case where the planned stations do not include stations with a predetermined number of visitors or more. For example, the registration unit 341 may identify a gate that the user U plans to use based on the area of ​​residence. If the predicted number of users at the identified gate is greater than or equal to a predetermined number, the registration unit 341 may configure the appearance group to include an appearance ID indicating a complex appearance, as described above. Also, as described above, the registration unit 341 may configure the appearance group by increasing the number of appearance IDs included as options compared to a case where the predicted number of users is less than a predetermined number. Furthermore, if the user attribute information includes the purpose of use, the registration unit 341 may identify the gate that the user U plans to use based on the purpose of use and set an appearance group in the same manner as for the residential area described above. The gate that the user U plans to use is a gate that has gate attribute information that matches the purpose of use. As an example, if the purpose of use is "railway use," the registration unit 341 may configure an appearance group by increasing the number of appearance IDs included as options by a predetermined number compared to when "theme park use" is used. As another example, if the purpose of use is "event venue use," the registration unit 341 may predict the number of expected attendees based on the event type of the event venue and set an appearance group in accordance with the number of expected attendees in the same manner as described above. Note that if the planned date and time for the user U to use the event venue is registered as user attribute information, the appearance group may be set according to the planned date and time.As an example, if the scheduled date and time falls within a busy time period on a holiday, the registration unit 341 may configure an appearance group to include an appearance ID indicating a more complex appearance, or may increase the number of appearance IDs included as options, compared to other time periods or weekdays. For example, the registration unit 341 may configure and set an appearance group as described above and register an appearance ID in response to the registration of user attribute information at the time of event reservation. In this way, even at gates with a large number of users, erroneous authentication can be avoided by increasing the complexity of the appearance or the variety. Therefore, authentication accuracy can be improved.

[0063] Furthermore, when the similarity of the registration face information of each user U with another user U is equal to or greater than a predetermined threshold, the registration unit 341 may set the registration appearance ID of the user U to be different from the registration appearance ID of a similar user U. In this case, the registration unit 341 may set the appearance group of the user U to be different from that of the similar user U and have the user U select an appearance ID from the appearance group, thereby making the registration appearance ID different. In other words, the registration unit 341 sets the appearance IDs presented to the user U as options to be different from the appearance IDs presented to the similar user U and have the user U select an appearance ID from the set appearance IDs. The similarity may be calculated based on the facial feature information 112 stored in the face information DB 110 of the face authentication device 100. The calculation of the similarity is performed by the face authentication device 100, and the registration unit 341 may acquire user IDs whose similarity is equal to or greater than a predetermined threshold from the face authentication device 100 via the network N. In this way, if the facial structures are similar, multiple users U may be identified through facial recognition, but if the appearances to be authenticated are clearly different, it is possible to more clearly distinguish which of the identified users U is the person to be authenticated through appearance recognition. This makes it possible to avoid false authentication and improve authentication accuracy.

[0064] Then, when a predetermined update condition is met, the registration unit 341 updates the appearance information for registration, i.e., the appearance ID in the registration history information 313. Updating the appearance information for registration refers to the registration unit 341 changing the appearance group in the registration history information 313 for the user U who meets the predetermined update condition and updating the appearance ID for registration to an appearance ID selected from the changed appearance group. Here, changing the appearance group may include changing the appearance group in the registration history information 313 for the user U who meets the predetermined update condition to an appearance group including an appearance ID indicating a complex appearance. Furthermore, changing the appearance group may include changing the appearance group in the registration history information 313 for the user U who meets the predetermined update condition to an appearance group that includes more appearance IDs as options than the currently set appearance group. Note that updating the appearance information for registration may include outputting the appearance IDs included in the changed appearance group to the user terminal 500 and accepting the selection of the changed appearance ID from the user U. Updating the appearance information for registration may include adding a new record to the registration history information 313 and recording the changed appearance ID selected by the user U in the added record. Here, the predetermined update condition may be that the user U has passed through a gate. That is, the registration unit 341 updates the appearance ID in the registration history information 313 when the user U passes through a gate and enters or exits the gate. For example, if the type of appearance is a movement, the movement for authentication is changed each time the user enters or exits the gate. This increases the security level of personal authentication because even if the authentication movement performed when passing through the gate is discovered by others, the movement performed when passing through the next gate will be different from the movement performed when passing through the previous gate. Note that the registration appearance information may be updated not only every time the user passes through a gate, but also when the user passes through the gate a predetermined number of times using the same appearance information during authentication (e.g., performing the same movement during authentication).

[0065] The predetermined update condition may be a predetermined period of time having elapsed since the date and time of registration or the last update. That is, the registration unit 341 may update the appearance information for registration for each user U when a predetermined period of time has elapsed since the date and time of registration or the last update. The predetermined period may be, for example, one day, one week, or one month. The registration unit 341 may update the appearance information for registration for each user U at a predetermined time every day. Furthermore, when the registration unit 341 receives a request to update user attribute information from the user terminal 500, it may update the user attribute information in the registration history information 313 and update the appearance information for registration based on the updated user attribute information. For example, the registration unit 341 may update the appearance information for registration in response to a change in the type of gate that user U plans to use due to an update of the user attribute information. For example, a change in the gate type may occur when the purpose of use changes from a theme park to a train or when the boarding and disembarking station is different from usual. This improves the security level of personal authentication. Furthermore, for example, if an update of the user attribute information causes the predicted number of users of a gate that user U plans to use to change by more than a predetermined number, the registration unit 341 may update the appearance information for registration of that user U. As an example, a case in which the predicted number of users of a gate that user U plans to use changes by more than a predetermined number is when a large event is held near that gate, causing an increase in users compared to usual. This makes it possible to improve the security level of personal authentication while avoiding erroneous authentication.

[0066] The predetermined update condition may be that the failure rate of personal authentication is equal to or greater than a predetermined threshold. That is, the registration unit 341 may update the registration appearance information of the user U when the failure rate of the user U is equal to or greater than a predetermined threshold. The registration unit 341 may update the registration appearance information of the user U when the failure rate of face authentication is equal to or greater than a predetermined threshold, instead of the failure rate of personal authentication. In this case, face authentication may fail because the user U's face is similar to that of another user U. Therefore, the registration unit 341 may identify similar users whose registration face information is similar to that of the user U, and set (update) the registration appearance ID of the user U so that the registration appearance ID is different from that of the similar users. The identification of similar users is the same as described above. The setting (update) of the registration appearance ID is the same as described above, and includes changing the appearance group. This improves authentication accuracy.

[0067] Here, an example of a method for calculating the failure rate of user U will be described. First, the information processing device 300 identifies the user ID using a predetermined method in response to a failure of personal authentication. For example, if personal authentication fails, the user U presents an IC card, a barcode, or the like, and has the authentication terminal 400 read the user ID. The authentication terminal 400 transmits the acquired user ID and gate ID to the information processing device 300 via the network N. Next, the information processing device 300 records the fact that the personal authentication failed in the authentication history information 314, in association with the user ID. Note that the information processing device 300 may record in the authentication history information 314 a reason for the failure, indicating which authentication failed, based on the face authentication result and the appearance authentication result. Then, the information processing device 300 calculates the personal authentication failure rate and the failure rate for each authentication method for each user U using the authentication history information 314, and records the calculated failure rate in the storage unit 310 in association with the user ID.

[0068] The acquisition unit 342 is an example of the acquisition unit 12 described above. The acquisition unit 342 acquires a personal authentication request including captured image data from the authentication terminal 400 via the network N and the communication unit 330. The captured image data may be a still captured image or a video including multiple captured images (frames). In particular, when the appearance information for registration is motion information, the captured image data may be a video. The acquisition unit 342 supplies the acquired captured image data to the authentication control unit 343.

[0069] The authentication control unit 343 is an example of the authentication control unit 13 described above. The authentication control unit 343 controls face authentication for the facial region of the user U included in the captured image of the captured image data. That is, the authentication control unit 343 controls face authentication for the facial region of the user U included in the captured image of the captured image data captured at each gate. That is, the authentication control unit 343 causes the face authentication device 100 to perform face authentication for the captured image acquired from the authentication terminal 400. For example, the authentication control unit 343 transmits a face authentication request including the acquired captured image data, a gate ID, and the capture date and time to the face authentication device 100 via the network N. Note that, if the captured image data is a video, the authentication control unit 343 may include some captured images (frames) included in the captured image data in the face authentication request, or may include all frames in the face authentication request. At this time, facial feature information for authentication is generated for each of the multiple frames. Note that the authentication control unit 343 may detect the facial region of the user U from the captured image included in the captured image data and include an image of the facial region in the face authentication request. Furthermore, the authentication control unit 343 may extract facial feature information from the face region and include the facial feature information in the face authentication request. The authentication control unit 343 then receives the face authentication result from the face authentication device 100.

[0070] If facial authentication is successful, the authentication control unit 343 refers to the registration history information 313 and obtains the appearance ID recorded in the latest record of the user ID included in the facial authentication result. The authentication control unit 343 then controls appearance authentication for the detection area of ​​user U included in the captured image of the captured image data. For example, an appearance authentication request including the captured image data and the appearance ID is sent to the appearance authentication device 200 via the network N. Again, if the captured image data is a video, the authentication control unit 343 may include some or all of the frames included in the captured image data in the appearance authentication request. Furthermore, the authentication control unit 343 may include all of the frames included in the captured image data, excluding frames for which facial authentication failed, in the appearance authentication request. Note that if the appearance ID indicates a motion, particularly multiple motions, the authentication control unit 343 includes multiple frames in the appearance authentication request. At this time, appearance information for authentication is generated for each of the multiple frames. Furthermore, if the facial area to be facial authentication can be identified, the authentication control unit 343 may include information identifying the facial area in the appearance authentication request. In this case, the authentication control unit 343 may detect a detection area corresponding to the appearance ID of the user U from the captured image based on the face area, and include an image of the detection area in the appearance authentication request. Then, the authentication control unit 343 receives the appearance authentication result from the appearance authentication device 200.

[0071] The authentication control unit 343 performs personal authentication based on the results of face authentication and appearance authentication. For example, the authentication control unit 343 determines that personal authentication is successful only when both face authentication and appearance authentication are successful. The authentication control unit 343 determines that personal authentication is unsuccessful in cases other than those mentioned above. The authentication control unit 343 supplies the results of personal authentication to the gate control unit 344.

[0072] The gate control unit 344 is an example of the above-mentioned gate control unit 14. If the personal authentication result indicates success, the gate control unit 344 transmits a control signal to open the gate to the gate driving device 600 that controls the gate at the shooting location of the user U. On the other hand, if the personal authentication result indicates failure, the gate control unit 344 transmits a control signal to close the gate to the gate driving device that controls the gate at the shooting location of the user U. Note that the control signal transmitted from the gate control unit 344 to the gate driving device 600 may be a control signal to cause the gate driving device 600 to output information indicating whether personal authentication has been successful or unsuccessful, instead of a control signal to open or close the gate.

[0073] 14 is a flowchart showing the flow of registration processing according to embodiment 2. First, the registration unit 341 of the information processing device 300 receives a service registration request from the user terminal 500 via the network N (S401). At this time, if the registration unit 341 receives a captured image in addition to the registration request, it requests the face authentication device 100 via the network N to register the face information in the face information DB 110.

[0074] Next, the registration unit 341 acquires a user ID from the face authentication device 100 via the network N (S402). Then, the registration unit 341 acquires attribute information of the user U from the requesting user terminal 500 via the network N (S403). Then, the registration unit 341 registers the attribute information of the user U as user attribute information in the registration history information 313 in association with the user ID (S404). Next, the registration unit 341 sets an appearance group based on the user attribute information (S405). Then, the registration unit 341 references the appearance-related information 316 and acquires appearance IDs included in the appearance group. Next, the registration unit 341 transmits multiple appearance IDs included in the appearance group to the user terminal 500, and causes the user terminal 500 to display the appearance IDs in a selectable manner (S406). When the user U selects one appearance ID from among multiple appearance IDs displayed on the display unit 540 of the user terminal 500, the registration unit 341 acquires the selected appearance ID from the user terminal 500 (S407).Then, the registration unit 341 registers the registration date and time, the appearance group ID, and the selected appearance ID in association with the user ID in the registration history information 313 (S408).

[0075] FIG. 15 is a flowchart showing the flow of personal authentication processing according to the second embodiment. First, the authentication control unit 343 acquires captured image data from the authentication terminal 400 via the network N (S411). Next, the authentication control unit 343 transmits a face authentication request to the face authentication device 100 via the network N (S412). At this time, the authentication control unit 343 includes in the face authentication request at least one of the captured image data acquired in step S411, a portion of the captured image included in the captured image data, a facial area extracted from the captured image, and facial feature information extracted from the facial area. Then, the authentication control unit 343 receives a face authentication result from the face authentication device 100 via the network N (S413). If face authentication is successful, the face authentication result includes a message to that effect, a user ID, and information identifying the facial area. If face authentication is unsuccessful, the face authentication result also includes a message to that effect. Note that the face authentication result may include multiple user IDs. In such a case, the face authentication result includes information identifying the facial area corresponding to each user ID.

[0076] The authentication control unit 343 determines whether or not there is a user ID for which face authentication has been successful (S414). If there is no user ID for which face authentication has been successful (No in S414), the authentication control unit 343 determines that personal authentication has failed, and transmits a control signal to close the gate via the network N to the gate driving device 600 corresponding to the authentication terminal 400 that originated the image capture (S415). Then, the authentication control unit 343 ends the process.

[0077] On the other hand, if it is determined that there is a user ID for which face authentication has been successful (Yes in S414), the authentication control unit 343 identifies the user ID for which face authentication has been successful (S416). Specifically, the authentication control unit 343 extracts the user ID included in the face authentication result. In the following, if there are multiple user IDs included in the face authentication result, steps S417 to S419 are executed for each user ID.

[0078] In step S417, the authentication control unit 343 acquires the appearance ID included in the latest registration history information 313 for the extracted user ID. Then, in step S418, the authentication control unit 343 sends an appearance authentication request to the appearance authentication device 200 via the network N (S418). At this time, the authentication control unit 343 includes in the appearance authentication request at least one of the captured image data acquired in step S411, a portion of the captured image included in the captured image data, a detection area extracted from the captured image, or appearance feature information extracted from the detection area. Note that the authentication control unit 343 may also include information specifying a face area in the appearance authentication request. Then, the authentication control unit 343 receives an appearance authentication result from the appearance authentication device 200 via the network N (S419). The appearance authentication result includes information indicating whether appearance authentication was successful or unsuccessful. If appearance authentication is successful, the appearance authentication result may also include the user ID.

[0079] The authentication control unit 343 determines whether any user ID has been successfully authenticated (S420). If no user ID has been successfully authenticated (No in S420), the authentication control unit 343 determines that the personal authentication has failed, and transmits a control signal to close the gate via the network N to the gate driving device 600 corresponding to the authentication terminal 400 that captured the image (S415). The authentication control unit 343 then ends the process. On the other hand, if any user ID has been successfully authenticated (Yes in S420), the authentication control unit 343 determines that the personal authentication has been successful, and transmits a control signal to open the gate to the gate driving device 600 corresponding to the authentication terminal 400 that captured the image (S421). Then, in step S422, the authentication control unit 343 registers the user ID, the image capture date and time, the gate ID, the authentication result, and the reason for failure as authentication history information 314.

[0080] 16 is a flowchart showing the flow of the appearance information update process according to the second embodiment. When the predetermined update condition described above is satisfied (Yes in S431), the registration unit 341 executes the appearance information update process shown in steps S432 to S435. First, the registration unit 341 changes the appearance group for the user U whose appearance information is to be updated (S432). Then, the registration unit 341 refers to the appearance-related information 316 and acquires the appearance IDs included in the changed appearance group. Next, the registration unit 341 transmits the appearance IDs included in the appearance group to the user terminal 500 and causes the user terminal 500 to display the appearance IDs selectably (S433). When the user U selects one appearance ID from the appearance IDs displayed on the display unit 540 of the user terminal 500, the registration unit 341 acquires the selected appearance ID from the user terminal 500 (S434). Then, the registration unit 341 updates the appearance group ID and appearance ID in the registration history information 313 to the changed appearance group ID and the newly selected appearance ID (S435).

[0081] Next, an example of processing when the appearance type is an action will be described with reference to Figures 17 to 24. In Figures 17 to 24, the appearance group is called an action group, the appearance ID is called an action ID, and the appearance authentication is called an action authentication.

[0082] FIG. 17 is a sequence diagram illustrating the flow of the registration process according to the second embodiment. First, the user terminal 500 transmits a service registration request to the information processing device 300 (S500). The user terminal 500 also photographs the user U (S501) and transmits a face information registration request including the photographed image to the face authentication device 100 via the network N (S502). The face authentication device 100 then registers face information (facial feature information) of the user U based on the photographed image included in the received face information registration request (S503). The face authentication device 100 then notifies the information processing device 300 of the user ID via the network N (S504). The user terminal 500 also transmits user attribute information to the information processing device 300 via the network N (S505). The information processing device 300 registers the notified user ID and user attribute information in the registration history information 313 in association with each other (S506). The information processing device 300 then sets an operating group for the user U (S507). The information processing device 300 transmits the plurality of action IDs included in the action group to the user terminal 500 via the network N (S508). The user terminal 500 that has received the action IDs displays the action IDs (S509).

[0083] Here, Fig. 18 is a diagram showing an example of the display of the user terminal 500 according to the second embodiment. On the display unit 540 of the user terminal 500, actions 1 and 2 are displayed in a selectable manner along with a message saying "Please select today's action." Action 1 has an action ID of "raise right hand," and action 2 has an action ID of "raise left hand." In this figure, the user U has selected action 2.

[0084] 17, the explanation will be continued. The user terminal 500 accepts a selection operation of an action ID from the user U (S510), and transmits the selected action ID to the information processing device 300 via the network N (S511). Then, the information processing device 300 associates the user ID with the registration date and time, the action group, and the action ID, and registers them as registration history information 313 (S512).

[0085] 19 is a sequence diagram showing the flow of personal authentication processing according to the second embodiment. First, at gate A1, the authentication terminal 400-1 photographs a user U (S520) and transmits the photographed image data and the gate ID to the information processing device 300 via the network N (S521). The information processing device 300 transmits a face authentication request for the face region of user U in the received photographed image to the face authentication device 100 via the network N (S522). Then, the face authentication device 100 performs face authentication on the face region of user U in the photographed image included in the received face authentication request (S523). Here, it is assumed that there is a user ID whose face authentication was successful. The face authentication device 100 transmits a face authentication result including a notice that face authentication was successful and the user ID to the information processing device 300 via the network N (S524).

[0086] The information processing device 300, which has received the facial authentication result, acquires the action ID corresponding to the user ID included in the facial authentication result from the registration history information 313 (S525). Subsequently, the information processing device 300 transmits a action authentication request, including the action ID and the captured image data, to the appearance authentication device 200 via the network N (S526). The appearance authentication device 200 then performs action authentication on the detection area of ​​user U in the captured image data included in the received action authentication request (S527). Here, it is assumed that there is a user ID for which action authentication has been successful. The appearance authentication device 200 transmits the action authentication result, including a notification that the action authentication was successful, to the information processing device 300 via the network N (S528). Upon receiving the action authentication result, the information processing device 300 determines that personal authentication has been successful and transmits an open control signal to the gate driving device 600 that controls gate A1 (S529). This opens gate A1 (S530). Furthermore, the information processing device 300 registers the authentication history information 314 (S531). Then, the user U passes through (enters) gate A1 and moves to gate A2.

[0087] The information processing device 300 may determine that personal authentication has been successful only when both face authentication and motion authentication have been successful for the same captured image (frame). Specifically, when face feature information for authentication and motion feature information for authentication are generated based on the same frame included in the captured image data and both face authentication and motion authentication have been successful, the information processing device 300 determines that personal authentication has been successful.

[0088] FIG. 20 is a diagram illustrating an example of personal authentication processing according to the second embodiment. As shown in FIG. 20, the captured image data included in the face authentication request includes captured images 800-1 to 800-3 captured at different times. First, the face authentication device 100 detects the user U1 included in the captured images 800-1 to 800-3 and identifies the user ID of the user U1. Here, it is assumed that face authentication has been successful for the captured images 800-1 and 800-2. Therefore, the information processing device 300 transmits a motion authentication request to the appearance authentication device 200 for the captured images 800-1 and 800-2. This motion authentication request may include, for each of the captured images 800-1 and 800-2, the registration motion ID of the user U1 identified by face authentication and information identifying the facial region of the user U1. Then, the appearance authentication device 200 identifies a detection region near the facial region of the user U1 for each of the captured images 800-1 and 800-2 and determines whether the detection region corresponds to a predetermined motion ID. Here, it is assumed that the appearance authentication device 200 determines that only the captured image 800-2 corresponds to a predetermined action ID. In this case, the appearance authentication device 200 transmits an action authentication result indicating that the action authentication was successful to the information processing device 300. The information processing device 300, which has received the action authentication result, determines that personal authentication for user U1 has been successful. Note that in the captured image 800-2, the face area of ​​user U2 as well as that of user U1 is detected. Therefore, the appearance authentication device 200 also performs action authentication on user U2 for the captured image 800-2. However, the detection area of ​​user U2 in the captured image 800-2 cannot be identified, and the action authentication fails. Note that even if a detection area is identified for user U2 in the captured image 800-2 and the action corresponds to the action ID for registration of user U1, if the action does not correspond to the action ID for registration of user U2, the appearance authentication device 200 may determine that the action authentication has failed.

[0089] In this way, by determining that personal authentication is successful only when both face authentication and motion authentication are successful in the same frame, the security level of personal authentication can be improved and personal authentication can be performed appropriately. Furthermore, because the detection area for motion authentication is determined based on the face area where face authentication was successful, erroneous authentication is less likely to occur even when there are people around.

[0090] Alternatively, the information processing device 300 may determine that personal authentication has been successful even if the frame in which facial authentication has been successful and the frame in which motion authentication has been successful are different within the same captured image data. FIG. 21 is a diagram for explaining an example of personal authentication processing according to the second embodiment. As shown in FIG. 21, the captured image data included in the face authentication request includes captured images 810-1 to 810-3. Face authentication is successful for captured image 810-1, but fails for captured images 810-2 and 810-3. Here, the information processing device 300 calculates the movement distance from the shooting interval (frame rate) between captured image 810-1 and captured image 810-2, and determines that user U1 in captured image 810-1 is the same person as user U1 in captured image 810-2. Therefore, the information processing device 300 also sends a motion authentication request for captured image 800-2 to the appearance authentication device 200. This behavior authentication request may include the registration behavior ID of user U1 identified by face authentication of photographed image 810-1 and information identifying the facial area of ​​user U1 in photographed image 810-2. Appearance authentication device 200 performs behavior authentication on photographed image 800-2 and determines whether the detection area corresponds to the behavior ID included in the behavior authentication request. Here, it is assumed that appearance authentication device 200 determines that the detection area of ​​photographed image 800-2 corresponds to the behavior ID included in the behavior authentication request. In this case, appearance authentication device 200 transmits a behavior authentication result to information processing device 300 indicating that behavior authentication was successful. As a result, information processing device 300 determines that personal authentication was successful.

[0091] In this way, by determining whether personal authentication is successful or not based on a combination of face authentication results and motion authentication results for multiple frames in the same captured image data, personal authentication can be performed appropriately. For example, when performing personal authentication by walkthrough, even if part of the subject is not clearly captured in some frames, personal authentication can be performed appropriately. Furthermore, since a user U whose face has been successfully authenticated is tracked once and the detection area for motion authentication is determined based on the face area of ​​the user U determined to be the same person, erroneous authentication is unlikely to occur even when there are people around.

[0092] Furthermore, if the action ID indicates a combination of a plurality of actions (referred to as individual actions), the information processing device 300 may determine that personal authentication has been successful when the captured image data includes a frame in which face authentication and individual action authentication have been successful for each individual action. Fig. 22 is a diagram for explaining an example of personal authentication processing according to the second embodiment.

[0093] For example, the registration action ID of user U1 indicates the first individual action, then the second individual action, and then the third individual action. As shown in FIG. 22, the captured image data included in the face authentication request includes captured images 820-1 to 820-3. Captured image 820-1 is a frame (first frame) in which face authentication and authentication of the first individual action were successful. Captured image 820-2 is a frame (second frame) in which face authentication and authentication of the second individual action were successful. Captured image 820-3 is a frame (third frame) in which face authentication and authentication of the third individual action were successful. If the captured image data includes the first to third frames, the information processing device 300 may determine that personal authentication is successful. Note that if the order of the individual actions is fixed as in this example, the information processing device 300 may determine that personal authentication is successful if the capture order is the first frame → the second frame → the third frame.

[0094] 23 is a sequence diagram showing the flow of update processing according to the second embodiment. When a predetermined update condition is satisfied, the information processing device 300 changes the action group for the user U (S540). Then, the information processing device 300 transmits the action ID included in the action group after the change to the user terminal 500 via the network N (S541). The user terminal 500 that has received the action ID displays the action ID (S542).

[0095] 24 is a diagram showing an example of a display on the user terminal 500 according to the second embodiment. For example, as shown in this figure, a message saying "Please change the action" may be displayed on the display unit 540 of the user terminal 500 before the action ID is displayed. After that, a screen similar to that shown in FIG. 18 may be displayed on the display unit 540 of the user terminal 500.

[0096] Returning to Fig. 23, the explanation will be continued. The user terminal 500 accepts a selection operation of an action ID from the user U (S543), and transmits the selected action ID to the information processing device 300 via the network N (S544). Then, the information processing device 300 newly registers information associating the user ID with the update date and time, the action group, and the action ID as the registration history information 313, and updates the registration history information 313 (S545). Then, when the user U moves to gate A2, the same processes as those shown in steps S520 to S531 in Fig. 19 are performed.

[0097] As described above, according to the second embodiment, the information processing device 300 performs multi-factor authentication using image recognition, including face authentication and appearance authentication, and updates the registered appearance information used for appearance authentication when a predetermined update condition is met. Therefore, while improving authentication accuracy through multi-factor authentication, it is possible to increase the security level by reducing the risk of personal information, such as appearance information, being leaked and misused. This is particularly effective when performing personal authentication at station ticket gates or theme park entrance / exit gates where there are people nearby. Furthermore, by changing the update mode of the appearance information depending on the type of update condition, it is possible to perform appropriate personal authentication depending on the situation.

[0098] <Embodiment 3> Next, a third embodiment of the present disclosure will be described. In the second embodiment, the information processing device 300 performed personal authentication based on the face authentication result and one appearance authentication result. That is, the personal authentication was two-factor authentication. However, in the third embodiment, the information processing device 300 can perform personal authentication based on the face authentication result and multiple appearance authentication results. In the following, it is assumed that up to two types of appearance authentication can be performed in personal authentication, that is, up to three-factor authentication is possible. However, the number of types (number of elements) of appearance authentication is not limited to this. The type of appearance authentication (type of element) is the type of appearance information to be authenticated, and examples include movement and clothing.

[0099] FIG. 25 is a block diagram showing the configuration of an appearance authentication device 200a according to the third embodiment. The appearance authentication device 200a has basically the same configuration and functions as the appearance authentication device 200. However, the appearance authentication device 200a has a first appearance information DB 210-1 and a second appearance information DB 210-2 instead of the appearance information DB 210. The first appearance information DB 210-1 and the second appearance information DB 210-2 may have the same data structure as the appearance information DB 210, but the types of elements are different. For example, the first appearance information DB 210-1 stores motion information, and the second appearance information DB 210-2 stores clothing information. Specifically, the first appearance information DB 210-1 stores a first appearance ID 211-1 and corresponding first appearance feature information 212-1 as motion information. The second appearance information DB 210-2 also stores, as clothing information, a second appearance ID 211-2 and corresponding second appearance characteristic information 212-2.

[0100] Appearance authentication device 200a also includes detection unit 220a, feature point extraction unit 230a, registration unit 240a, and authentication unit 250a. Detection unit 220a, feature point extraction unit 230a, registration unit 240a, and authentication unit 250a basically function in the same way as detection unit 220, feature point extraction unit 230, registration unit 240, and authentication unit 250. However, registration unit 240a registers an appearance ID and appearance feature information in appearance information DB 210 corresponding to the element type. Furthermore, when appearance authentication device 200a receives an appearance authentication request including information indicating the element type from information processing device 300, detection unit 220a, feature point extraction unit 230a, and authentication unit 250a perform the various processes described above using appearance information DB 210 corresponding to the element type.

[0101] 26 is a flowchart showing the flow of appearance authentication processing according to the third embodiment. First, the detection unit 220a acquires an appearance authentication request from the information processing device 300 via the network N (S51a). At this time, the appearance authentication request includes captured image data and an appearance ID for registration for each element type of the user U identified by face authentication. Then, the appearance authentication device 200a repeats the processing of steps S52a to S57a, which correspond to steps S52 to S57 in FIG. 8, for each element type. Note that in step S56a, the authentication unit 250a may transmit to the information processing device 300 a notification that appearance authentication has been successful and the appearance ID in association with the user ID.

[0102] FIG. 27 is a diagram illustrating an example of the data structure of the registration history information 313 according to the third embodiment. The registration history information 313 in the storage unit 310 of the information processing device 300 according to the third embodiment can record appearance information for each element type. That is, the registration history information 313 has fields for an appearance group ID and an appearance ID set for each element type. The registration unit 341 of the information processing device 300 sets the number of element types, i.e., the number of elements and the element types, for each user U, and records the appearance group ID and appearance ID corresponding to the set element type in the registration history information 313. For example, for user U1, movement information (first appearance information) and clothing information (second appearance information) are registered as appearance information. Therefore, the authentication control unit 343 performs personal authentication of user U1 based on the second appearance authentication result in addition to the face authentication result and the first appearance authentication result. On the other hand, for user U2, only movement information (first appearance information) is registered as appearance information. Therefore, the authentication control unit 343 performs personal authentication of the user U2 based on the face authentication result and the first appearance authentication result.

[0103] Here, when a predetermined condition for adding the number of elements is satisfied, the registration unit 341 of the information processing device 300 according to the third embodiment adds a predetermined number of appearance information pieces of an element type different from the appearance information for registration already registered for that user U. For example, when the predetermined condition for adding the number of elements is satisfied for each user U, the registration unit 341 additionally registers, for that user U, second appearance information for registration of an element type different from the first appearance information for registration. The predetermined condition for adding the number of elements may be that the predicted number of users of the gate that the user U plans to use is equal to or greater than a predetermined number. Alternatively, the predetermined condition for adding the number of elements may be that the failure rate of personal authentication of the user U is equal to or greater than a predetermined threshold. When the predetermined condition for adding the number of elements is satisfied, the information processing device 300 may notify the user terminal 500 of the user U that additional appearance information will be registered.

[0104] If a predetermined element number reduction condition is satisfied, the registration unit 341 deletes a predetermined number of pieces of appearance information from the multiple pieces of appearance information for registration registered for the user U. For example, if the predetermined element number reduction condition is satisfied for each user U, the registration unit 341 deletes either the first appearance information or the second appearance information for registration registered for the user U. The predetermined element number reduction condition may be that the predicted number of users of the gate used by the user U is less than a predetermined number. Alternatively, the predetermined element number reduction condition may be that the failure rate of personal authentication for the user U is less than a predetermined threshold. If the predetermined element number reduction condition is satisfied for each user U, the registration unit 341 may delete all of the registration appearance information registered for the user U. In other words, when a user U who satisfies the predetermined element number reduction condition passes through a gate, appearance authentication may be omitted and only face authentication may be required. Therefore, if the predetermined element number reduction condition is satisfied, the information processing device 300 may notify the user terminal 500 of the user U that appearance authentication will be omitted.

[0105] 28 is a flowchart showing the flow of element number addition processing according to the third embodiment. When the above-described element number addition condition is satisfied (Yes in S601), the registration unit 341 executes the processing from step S602 onwards. In step S602, the registration unit 341 determines the number of elements to be added and the type of element to be added. Next, the registration unit 341 repeats the processing shown in steps S603 to S606 for each element type to be added. The processing shown in steps S603 to S606 is the same as the processing shown in steps S432 to S435 in FIG. 16, and therefore description thereof will be omitted.

[0106] 29 is a flowchart showing the flow of personal authentication processing according to embodiment 3. The steps shown in FIG. 29 include steps S417a to S420a instead of steps S417 to S420 in FIG.

[0107] In response to identifying a user ID whose face authentication has been successful (S416), the authentication control unit 343 refers to the latest registration history information 313 of the user U, and repeats the processing shown in steps S417a to S419a for each element type of the appearance ID for registration. Steps S417a to S419a correspond to steps S417 to S419. The authentication control unit 343 then determines whether there is a user ID whose appearance authentication has been successful corresponding to all element types for which the appearance ID for registration has been registered (S420a). If there is a user ID (Yes in S420a), the authentication control unit 343 proceeds to the processing in step S421; if not (No in S420a), the authentication control unit 343 proceeds to the processing in step S415. The processing thereafter is the same as in FIG. 15.

[0108] As described above, according to the third embodiment, when the conditions are met, the number of elements for appearance authentication is changed, so that it is possible to ensure appropriate authentication accuracy, authentication speed, and security level depending on the situation.

[0109] The present disclosure is not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit and scope of the present disclosure. For example, in the second and third embodiments, the registration unit 341 of the information processing device 300 sets an appearance group for the user U at the time of registration or update, and allows the user U to select appearance information for registration from among multiple pieces of appearance information. However, instead of this, the registration unit 341 may automatically set appearance information for the user U at the time of registration or update. In other words, the user U does not need to be able to select appearance information for registration. The information processing device 300 may then transmit the automatically set appearance information for registration to the user terminal 500 and display the appearance information for registration on the display unit 540 of the user terminal 500 along with a message saying "Please perform this action." Alternatively, the registration unit 341 may register or update the appearance information of the user U based on an image of the user U taken to register the appearance information. As an example, a camera may be installed at the user U's front door, and when the user U leaves the house, a photo of the user U's clothes may be taken and the captured image may be transmitted to the information processing device 300. The information processing device 300 estimates clothing information of the user U based on the captured image and registers this in the registration history information 313.

[0110] Furthermore, when the user U approaches an arbitrary gate, the information processing device 300 may notify the user terminal 500 of appearance information for registration. For example, the information processing device 300 may acquire location information of the user terminal 500 at predetermined intervals, and may determine that the user U has approached the arbitrary gate when the distance between the user terminal 500 and the arbitrary gate becomes less than a predetermined distance. The location information of the user terminal 500 is, for example, GPS information acquired from a GPS (Global Positioning System) receiver. Furthermore, when the distance between the user terminal 500 and the arbitrary gate becomes less than the predetermined distance, the user terminal 500 may request the information processing device 300 to update the appearance information for registration, and may acquire the updated appearance information for registration from the information processing device 300. Furthermore, when the user U approaches a gate that the user plans to use or a station or event venue where the gate that the user plans to use is installed, the information processing device 300 may notify the user terminal 500 of the appearance information for registration. For example, the information processing device 300 may determine that the user U is approaching the gate that the user U plans to use when the distance between the user terminal 500 and the gate, station, or event venue that the user U plans to use is less than a predetermined distance.

[0111] Alternatively, user U may send a registration or update request for appearance information to the information processing device 300 via the user terminal 500. In response to the appearance information registration or update request, the information processing device 300 registers or updates the appearance information related to the registration or update request as user U's appearance information to be registered. When registering or updating user U's appearance information to be registered in response to the appearance information registration or update request, if there is another user U with similar or identical appearance information to be registered, the information processing device 300 may update the appearance information to different appearance information. For example, when registering appearance information to be registered in response to user U's registration request, the information processing device 300 determines whether there is a user with similar appearance information to be registered. If there is a user with similar appearance information to be registered, or if there are a predetermined number or more similar users, the information processing device 300 displays appearance information different from the appearance information to be registered on the user terminal 500 along with a message indicating the reason, such as "There are multiple users with similar clothing (or behavior). Please change to this clothing (or behavior)." In this case, the information processing device 300 may automatically update the displayed appearance information as registration appearance information, or may update it in response to receiving a message of acceptance from the user terminal 500. The above-described updating of the registration appearance information is similar when there is a user U with similar registration facial information, or when there is a user U with similar registration facial information and another user U with similar or the same registration appearance information. For example, when the information processing device 300 registers registration appearance information in response to a registration request from user U, it determines whether there is a user with similar registration facial information and similar registration appearance information. Then, the information processing device 300 displays on the user terminal 500 appearance information different from the registration appearance information of the user with similar registration facial information, along with a message indicating the reason, such as "There is a user with a similar face and similar clothing (or behavior). Please change to this clothing (or behavior)." In this case, the information processing device 300 may also update the registration appearance information of user U to appearance information of an element type of registration appearance information different from that of a user with similar registration facial information and / or appearance information.For example, if there is another user whose clothing information is similar to that of user U, the information processing device 300 updates the appearance information for registration to movement information. At this time, the information processing device 300 displays the movement information on the user terminal 500 together with a message saying, "There is a user with a similar face who is using clothing information for authentication. Please change the element used for authentication from clothing to this movement."

[0112] The information processing device 300 may cooperate with a point system and store points associated with each user U. The points can be used at stores or public transportation facilities, and can be exchanged for merchandise or used as payment for purchases. The information processing device 300 may add a predetermined amount of points to the user U when the user U is successfully authenticated at the gate. The information processing device 300 may also add a predetermined amount of points to the user U when the user U has registered appearance information in advance (pre-registration) and the user U is successfully authenticated at the gate. Pre-registration may include a case where the user U has requested registration of appearance information for registration in advance and the registration appearance information has been registered, and a case where the information processing device 300 has requested registration in advance and the user U has provided the registration appearance information to the information processing device 300. The information processing device 300 may also add a predetermined amount of points to the user U when the user U has pre-registered and is successfully authenticated at the gate, compared to a case where the user U has not pre-registered. Providing such an incentive can encourage use of the service. Note that the incentive is not limited to points, and may be any other benefit.

[0113] Furthermore, in the above-described second and third embodiments, the face authentication device 100 and the appearance authentication devices 200, 200a are connected to the information processing device 300 via the network N. However, instead of this, some or all of the functions of the face authentication device 100 and the appearance authentication device 200 may be included in the information processing device 300. Therefore, at least one of the face information registration process, the face authentication process, and the appearance authentication process may be performed in the information processing device 300.

[0114] In the above-described first to third embodiments, the information processing device uses a facial authentication result as the first determination result. However, the information processing device may use other biometric authentication results instead of or in addition to the facial authentication result. In this case, the other biometric authentication is biometric authentication using biometric information obtainable by a camera, and may be, for example, iris authentication, palm authentication, fingerprint authentication, humanoid authentication, or skeletal authentication. It may also be, for example, biometric authentication using a QR code (registered trademark) printed on a card worn around the neck. In this case, the "facial authentication" in the above description may be replaced with "biometric authentication," "facial information" or "facial feature information" may be replaced with "biometric information" or "biometric feature information," "facial image" may be replaced with "biometric image," and "facial authentication device" may be replaced with "biometric authentication device."

[0115] Although the above-described embodiment has been described as a hardware configuration, the present disclosure is not limited to this. Any processing in the present disclosure can also be realized by causing a CPU to execute a computer program.

[0116] In the above example, the program can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, DVDs (Digital Versatile Discs), and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may also be supplied to a computer by various types of transitory computer-readable media. Examples of transitory computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transitory computer-readable media can be supplied to a computer via wired communication paths such as electrical wires and optical fibers, or via wireless communication paths.

[0117] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of a person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; An information processing device comprising: (Appendix 2) The first appearance information is movement information indicating a movement or clothing information indicating clothing. 10. The information processing device according to claim 1. (Appendix 3) the first appearance information is operation information, The captured image data includes a plurality of frames, The face information for authentication and the first appearance information for authentication are generated based on the same frame included in the captured image data. 3. The information processing device according to claim 1 or 2. (Appendix 4) The registration unit updates the first appearance information for registration associated with the face information for registration in response to a change in the type of gate that the person to be registered is scheduled to use. 4. The information processing device according to claim 1. (Appendix 5) The registration unit updates first appearance information for registration associated with face information for registration in response to the person to be registered passing through the gate. 5. The information processing device according to any one of claims 1 to 4. (Appendix 6) When the predicted number of users of a gate that the person to be registered plans to use changes by a predetermined number or more, the registration unit updates the first appearance information for registration associated with the face information for registration of the person to be registered. 6. An information processing device according to any one of appendices 1 to 5. (Appendix 7) The registration unit updates the first appearance information for registration associated with the face information for registration of a person to be registered whose failure rate in personal authentication is equal to or greater than a predetermined threshold. 7. An information processing device according to any one of claims 1 to 6. (Appendix 8) When a predetermined period of time has elapsed, the registration unit updates the first appearance information for registration associated with the face information for registration for each person to be registered. 8. An information processing device according to any one of appendices 1 to 7. (Appendix 9) The registration unit sets first appearance information for registration for each person to be registered based on attribute information of the person to be registered. 9. An information processing device according to any one of appendices 1 to 8. (Appendix 10) When the similarity of the facial information for registration of each person to be registered with another person to be registered is equal to or greater than a predetermined threshold, the registration unit sets the first appearance information for registration of the person to be registered to first appearance information different from the first appearance information for registration corresponding to the other person to be registered. 10. An information processing device according to any one of appendices 1 to 9. (Appendix 11) The registration unit A first appearance information group consisting of a plurality of first appearance information is set for each person to be registered; registering the first appearance information selected from the first appearance information group as first appearance information for registration in association with face information for registration; For the person to be registered who satisfies the predetermined condition, a first appearance information group consisting of a plurality of first appearance information is changed; The first appearance information for registration is updated to the first appearance information selected from the changed first appearance information group. 11. An information processing device according to claim 1. (Appendix 12) the registration unit changes the first appearance information group corresponding to the registration target person who satisfies the predetermined condition to a first appearance information group including a larger number of first appearance information than the set first appearance information group; The first appearance information for registration is updated to the first appearance information selected from the changed first appearance information group. 12. An information processing device according to any one of claims 1 to 11. (Appendix 13) The registration unit displays each of the first appearance information included in the first appearance information group on a terminal used by the person to be registered so that the person to be registered can select it. 13. The information processing device according to claim 11 or 12. (Appendix 14) When a predetermined condition is satisfied for each person to be registered, the registration unit additionally registers second appearance information for registration that is different from the first appearance information for registration in association with the face information for registration for the person to be registered; The authentication control unit performs personal authentication of the person to be authenticated based on the first determination result, the second determination result, and a third determination result indicating whether second appearance information for authentication generated based on the captured image data corresponds to the second appearance information for registration. 14. An information processing device according to any one of claims 1 to 13. (Appendix 15) When the predicted number of users of the gate that the person to be registered is scheduled to use is equal to or greater than a predetermined number, the registration unit additionally registers second appearance information for registration that is different from the first appearance information for registration, in association with the face information for registration for the person to be registered. 15. The information processing device according to claim 14. (Appendix 16) The registration unit additionally registers second appearance information for registration, which is different from the first appearance information for registration, in association with the face information for registration, for a person to be registered who has a failure rate of personal authentication equal to or higher than a predetermined threshold. 16. The information processing device according to claim 14 or 15. (Appendix 17) an authentication terminal that generates a photographed image of a person to be authenticated; an information processing device communicably connected to the authentication terminal; Equipped with The information processing device includes: a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of the person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; have Information processing system. (Appendix 18) the first appearance information is operation information, The captured image data includes a plurality of frames, The face information for authentication and the first appearance information for authentication are generated based on the same frame included in the captured image data. 18. The information processing system of claim 17. (Appendix 19) A step of registering first appearance information for registration in association with face information for registration for each person to be registered; acquiring photographed image data of a person to be authenticated; performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; allowing the person to pass through the gate if the personal authentication is successful; updating the first appearance information for registration associated with the face information for registration for each person to be registered when the person satisfies a predetermined condition; An information processing method comprising: (Appendix 20) On the computer, A process of registering first appearance information for registration in association with face information for registration for each person to be registered; A process of acquiring photographed image data of a person to be authenticated; a process of performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; If the personal authentication is successful, the process of allowing the person to pass through the gate is For each person to be registered, if a predetermined condition is satisfied, a process of updating the first appearance information for registration associated with the face information for registration; A program to execute. [Explanation of symbols]

[0118] 10. Information processing equipment 11 Registration Department 12 Acquisition Department 13 Authentication control section 14 Gate control section 100 Facial Recognition Device 110 Face Information DB 111 User ID 112 Facial Feature Information 120 Face detection unit 130 Feature point extraction unit 140 Registration Department 150 Authentication Department 200,200a Appearance authentication device 210 Appearance Information DB 210-1 First Appearance Information DB 210-2 Second Appearance Information DB 211 Appearance ID 211-1 First Appearance ID 211-2 Second Appearance ID 212 Appearance characteristics information 212-1 First Appearance Feature Information 212-2 Second Appearance Feature Information 220, 220a Detector 230,230a Feature point extraction unit 240,240a Registration Department 250,250a Authentication Section 300 Information processing device 310 Storage section 311 Program 313 Registration History Information 314 Authentication History Information 315 Gate Information 3151 Gate ID 3152 Gate attribute information 316 Appearance-related information 3161 Appearance ID 3162 Appearance Group ID 320 memory 330 Communications Department 340 Control Unit 341 Registration Department 342 Acquisition Department 343 Authentication control section 344 Gate control section 400 Authentication Terminal 410 Camera 420 Storage section 430 Communications Department 440 Display section 450 control section 451 Imaging control unit 453 Authentication control unit 454 Display control unit 500 user terminals 510 Camera 520 Storage section 530 Communications Department 540 Display section 550 control section 551 Shooting control unit 552 Registration Department 553 Acquisition Department 554 Display control unit 560 Input section 600 Gate Driver 800,810,820 Images 1000 Information Processing Systems N Network U User

Claims

1. a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of a person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; Equipped with When the predicted number of users of a gate that the person to be registered plans to use has changed by a predetermined number or more, the registration unit updates the first appearance information for registration associated with the face information for registration of the person to be registered. Information processing device.

2. the first appearance information is motion information, The captured image data includes a plurality of frames, The face information for authentication and the first appearance information for authentication are generated based on the same frame included in the captured image data. The information processing device according to claim 1 .

3. The registration unit updates the first appearance information for registration associated with the face information for registration in response to the person to be registered passing through the gate.

3. The information processing device according to claim 1.

4. The registration unit updates the first appearance information for registration associated with the face information for registration of a person to be registered whose failure rate in personal authentication is equal to or greater than a predetermined threshold. The information processing device according to claim 1 .

5. When the similarity of the facial information for registration of each person to be registered with another person to be registered is equal to or greater than a predetermined threshold, the registration unit sets the first appearance information for registration of the person to be registered to first appearance information different from the first appearance information for registration corresponding to the other person to be registered. The information processing device according to claim 1 .

6. When a predetermined condition is satisfied for each person to be registered, the registration unit additionally registers second appearance information for registration that is different from the first appearance information for registration in association with the face information for registration for the person to be registered; The authentication control unit performs personal authentication of the person to be authenticated based on the first determination result, the second determination result, and a third determination result indicating whether second appearance information for authentication generated based on the captured image data corresponds to the second appearance information for registration. The information processing device according to claim 1 .

7. an authentication terminal that generates a photographed image of a person to be authenticated; an information processing device communicably connected to the authentication terminal; Equipped with The information processing device includes: a registration unit that registers first appearance information for registration in association with face information for registration for each person to be registered, and updates the first appearance information for registration associated with the face information for registration when a predetermined condition is satisfied; an acquisition unit that acquires photographed image data of the person to be authenticated; an authentication control unit that performs personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; a gate control unit that allows the person to pass through the gate if the personal authentication is successful; have When the predicted number of users of a gate that the person to be registered plans to use has changed by a predetermined number or more, the registration unit updates the first appearance information for registration associated with the face information for registration of the person to be registered. Information processing system.

8. a step of registering first appearance information for registration in association with face information for registration for each person to be registered; acquiring photographed image data of a person to be authenticated; performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; allowing the person to pass through the gate if the personal authentication is successful; updating the first appearance information for registration associated with the face information for registration for each person to be registered when the person satisfies a predetermined condition; updating the first appearance information for registration associated with the face information for registration of the person to be registered when the predicted number of users of the gate that the person to be registered is scheduled to use has changed by a predetermined number or more; An information processing method comprising:

9. On the computer, A process of registering first appearance information for registration in association with face information for registration for each person to be registered; A process of acquiring photographed image data of a person to be authenticated; a process of performing personal authentication of the person to be authenticated based on a first determination result indicating whether or not face information for authentication generated based on the captured image data corresponds to the face information for registration, and a second determination result indicating whether or not first appearance information for authentication generated based on the captured image data corresponds to the first appearance information for registration; If the personal authentication is successful, the process of allowing the person to pass through the gate is For each person to be registered, if a predetermined condition is satisfied, a process of updating the first appearance information for registration associated with the face information for registration; When the predicted number of users of a gate that the person to be registered is scheduled to use has changed by a predetermined number or more, a process of updating the first appearance information for registration associated with the face information for registration of the person to be registered is performed. A program to execute.

Images