Information processing device, information processing system, information processing method, and recording medium

Abstract

This information processing device comprises: a biological information acquisition means for acquiring first biological information and second biological information about a subject; and a control means for controlling display so that a first spot pertaining to the first biological information about the subject is highlighted when first authentication for the subject using the first biological information fails, a second spot pertaining to the second biological information about the subject is highlighted when second authentication for the subject using the second biological information fails, and each of the first location and the second location is highlighted when the first authentication fails and the second authentication fails.

Description

Information processing device, information processing system, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing system, an information processing method, and a recording medium.

[0002] As a device for notifying a subject who has failed authentication, for example, a device has been proposed that displays information identifying the subject by surrounding the face of the subject who has failed authentication on a display unit (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2019-175122

[0004] An object of this disclosure is to provide an information processing device, an information processing system, an information processing method, and a recording medium that aim to improve the technology related to the above-mentioned prior art documents.

[0005] One aspect of an information processing device includes a biometric information acquisition means for acquiring first biometric information and second biometric information of a target, and a control means for controlling display so that if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

[0006] One aspect of the information processing system is an information processing system comprising a wearable device and an information processing device, which acquires first biometric information and second biometric information of a target, and controls the display so that if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

[0007] One aspect of the information processing method acquires first biometric information and second biometric information of a target, and controls the display so that if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and if the first authentication fails and the second authentication fails, each of the first location and the second location is highlighted.

[0008] One aspect of the recording medium has a computer program recorded thereon for causing a computer to execute an information processing method for acquiring first biometric information and second biometric information of a subject, and controlling display so that if a first authentication of the subject using the first biometric information fails, a first location related to the first biometric information of the subject is highlighted, if a second authentication of the subject using the second biometric information fails, a second location related to the second biometric information of the subject is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

[0009] 1 is a block diagram showing a configuration of an example of an information processing device according to an embodiment. FIG. 2 is a flowchart showing an example of an operation of the information processing device according to an embodiment. FIG. 3 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 4 is a flowchart showing an example of an operation of the information processing system according to an embodiment. FIG. 5 is a diagram showing an example of a displayed image. FIG. 6 is a diagram showing another example of a displayed image. FIG. 7 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 8 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 9 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 10 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 11 is a block diagram showing a configuration of an example of an information processing system according to an embodiment. FIG. 12 is a block diagram showing a configuration of an example of an information processing system according to an embodiment.

[0010] [1: First embodiment]

[0011] A first embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to Figures 1 and 2. Hereinafter, the first embodiment of the information processing device, the information processing system, the information processing method, and the recording medium will be described using an information processing device 10.

[0012] 1, the information processing device 10 includes a biometric information acquisition unit 11 and a display control unit 12. The operation of the information processing device 10 will be described with reference to the flowchart of FIG.

[0013] 2, the biometric information acquisition unit 11 acquires first biometric information and second biometric information of a target (step S11). The "target" is a subject of biometric authentication. Therefore, the "target" may also be referred to as a person to be authenticated.

[0014] Biometric information is information regarding the physical characteristics of a subject. Examples of biometric information include at least one of a facial image including the subject's face, a partial facial image including a part of the subject's face, an iris image including the subject's iris, a fingerprint pattern of the subject, a palmprint pattern of the subject, a vein pattern of the subject's fingers and palm, audio information including the subject's voice, and sound information including the subject's otoacoustic emissions. Examples of biometric authentication include at least one of face authentication, iris authentication, fingerprint authentication, palmprint authentication, vein authentication, voiceprint authentication, and earacoustic authentication. The first biometric information and the second biometric information may be biometric information used for different biometric authentication methods. For example, the first biometric information may be a facial image, and the second biometric information may be an iris image. The first biometric information and the second biometric information may be biometric information used for the same biometric authentication method. For example, the first biometric information may be an iris image including the iris of the subject's right eye, and the second biometric information may be an iris image including the iris of the subject's left eye.

[0015] When at least one of the first authentication of the target using the first biometric information and the second authentication of the target using the second biometric information fails, the display control unit 12 controls the display so that at least one of the first location related to the first biometric information of the target and the second location related to the second biometric information of the target is highlighted (step S12). Specifically, when the first authentication fails but the second authentication succeeds, the display control unit 12 controls the display so that the first location of the target is highlighted. When the first authentication succeeds but the second authentication fails, the display control unit 12 controls the display so that the second location of the target is highlighted. When the first authentication fails and the second authentication also fails, the display control unit 12 controls the display so that the first location and the second location of the target are highlighted.

[0016] Note that "if authentication fails" may mean that the matching score is smaller than a threshold in the case of 1:1 authentication, and may mean that the maximum matching score is smaller than a threshold in the case of 1:N authentication. Note that "if authentication is successful" may mean that the matching score is equal to or greater than a threshold in the case of 1:1 authentication, and may mean that the maximum matching score is equal to or greater than a threshold in the case of 1:N authentication. Note that the first authentication of the target using the first biometric information and the second authentication of the target using the second biometric information may be performed by the information processing device 10 or by a device different from the information processing device 10.

[0017] Note that "emphasizing at least one of the first and second locations of the target" may mean at least one of "making at least one of the first and second locations of the target more noticeable than other parts of the target" and "making at least one of the first and second locations of the target attract the observer's attention more than other parts of the target." A display mode that emphasizes at least one of the first and second locations of the target may include, for example, a display mode in which a frame surrounding the highlighted location is superimposed on the highlighted location. Note that the display controlled by the display control unit 12 may be, for example, a display related to AR (Augmented Reality) glasses or a display related to a personal computer display.

[0018] In this way, the information processing device 10 acquires the first biometric information and second biometric information of a subject, and controls the display so that if the first authentication of the subject using the first biometric information fails, a first location related to the subject's first biometric information is highlighted, if the second authentication of the subject using the second biometric information fails, a second location related to the subject's second biometric information is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are highlighted.

[0019] The information processing device 10 described above may be realized by a computer reading a computer program recorded on a recording medium. In this case, the recording medium may have recorded thereon a computer program for causing the computer to execute an information processing method that acquires first biometric information and second biometric information of a target, and controls display so that, if first authentication of the target using the first biometric information fails, a first location related to the target's first biometric information is highlighted, if second authentication of the target using the second biometric information fails, a second location related to the target's second biometric information is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are highlighted. [Technical Effect]

[0020] For example, biometric authentication is sometimes used for entrance / exit management. For example, security guards may be deployed at sites where biometric authentication is performed to prevent individuals from attempting to circumvent biometric authentication through fraudulent means. However, biometric authentication may fail even when the individual (i.e., a person whose biometric information is registered) attempts biometric authentication. Therefore, if the security guard must carefully observe the entire subject whose biometric authentication has failed, this may place an excessive burden on the security guard and increase the time required for entry / exit. In response to this, the information processing device 10 highlights at least one of the first and second locations of the subject when at least one of the first and second biometric authentications fails. For example, if at least one of the first and second authentications is iris authentication, iris authentication may fail if hair is covering part of the subject's eye. If iris authentication fails, the subject's eye may be highlighted. For example, if a security guard observes the highlighted subject's eye and finds that hair is covering part of the subject's eye, the security guard may determine that iris authentication failed due to hair. On the other hand, if a security guard observes the highlighted eyes of a target and at first glance does not find anything in the target's eyes that would impede iris authentication, the security guard may suspect impersonation. The information processing device 10 highlights the portion of the biometric information used in the failed biometric authentication, so the security guard only needs to carefully observe the highlighted portion. In other words, the information processing device 10 can reduce the burden on the security guard and prevent an increase in the time required for entry and exit. In this way, the information processing device 10 according to this disclosure can easily notify the user that biometric authentication has failed. [2: Second Embodiment]

[0021] A second embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to Figures 3 to 6. Hereinafter, the second embodiment of an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S as the information processing device. Note that, for the second embodiment, descriptions that overlap with the description of the first embodiment will be omitted as appropriate.

[0022] The information processing system S according to this embodiment supports access control of a target TP to an access-restricted area. The target TP is a person attempting to enter or exit the access-restricted area. The access-restricted area may be an area requiring relatively strict entry control. The access-restricted area may be an important facility. For example, in important facilities, given the possibility of false recognition by biometric authentication devices, access control is often performed by humans (e.g., security guards) in addition to identity verification by an authentication device. Examples of important facilities include at least one of government buildings, foreign embassies, defense-related facilities, dams, space-related facilities, and power plants. Power plants include nuclear power plants, thermal power plants, and hydroelectric power plants. The information processing system S supports access control by humans (e.g., security guards).

[0023] The security guard confirms that the target TP is a person permitted to enter and exit the restricted access area (referred to as an "authorized person"). The security guard may confirm that the target TP is an authorized person by comparing the information presented by the information processing system S with the target TP. The information presented by the information processing system S may be, for example, a facial image of the target TP. Alternatively, the security guard may confirm that the target TP is an authorized person by visually checking an identification card held by the target TP. The identification card may include a facial photograph of the target TP, and the security guard may confirm that the target TP is an authorized person by comparing the facial photograph of the target TP with the facial photograph on the identification card. Such visual confirmation by the security guard is referred to as a "visual permission determination."

[0024] Incidentally, a target TP who has been successfully authenticated in two or more biometric authentications is highly likely to be the actual person. Therefore, in this embodiment, the security guard is supported by being notified of the success or failure of the biometric authentication of the target TP. The security guard can reduce his or her workload by knowing the success or failure of the biometric authentication of the target TP using the information processing device 20.

[0025] For the convenience of the security guard, the information processing device 20 may be in a form that can be worn by the security guard. For example, the information processing device 20 may be a device equipped with a wearable device. The information processing device 20 may include a wearable device that the security guard wears on his head. The information processing device 20 may also include AR glasses that the security guard wears. In this embodiment, a case will be described in which the display of the AR glasses worn by the security guard is controlled.

[0026] The information processing system S is a system that performs multimodal biometric authentication, which combines first and second biometric authentication methods. The first and second authentication methods may be, for example, facial authentication and iris authentication. In this embodiment, a case will be described in which the first authentication method is facial authentication and the second authentication method is iris authentication. Other examples of the first and second authentication methods will be described in other embodiments. Note that multimodal biometric authentication may refer to biometric authentication performed by combining two or more biometric authentication methods. However, multimodal biometric authentication may also be a concept that includes performing biometric authentication of different parts of a subject (e.g., the right eye and the left eye) using a single biometric authentication method.

[0027] 3, the information processing system S includes an information processing device 20 and an authentication device 1000. The information processing device 20 and the authentication device 1000 are configured to be able to communicate with each other. Note that the information processing device 20 and the authentication device 1000 may communicate with each other wirelessly or via wires.

[0028] 3, the information processing device 20 includes a calculation device 21, a storage device 22, a communication device 23, an input device 24, and an output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26. The information processing device 20 may further include a camera CAM. The calculation device 21 may include, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a TPU (Tensor Processing Unit), and a quantum processor.

[0029] The storage device 22 may include, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and an optical disk array. In other words, the storage device 22 may include a non-transitory recording medium. The storage device 22 is capable of storing desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program.

[0030] The communication device 23 may be capable of communicating with a device (for example, the authentication device 1000) external to the information processing device 20. The communication device 23 may perform wired communication or wireless communication.

[0031] The input device 24 is a device capable of accepting information input to the information processing device 20 from outside. The input device 24 may include an operation device (e.g., a keyboard, a mouse, a touch panel, etc.) that can be operated by a user (e.g., a security guard) of the information processing device 20. The input device 24 may include a recording medium reading device that can read information recorded on a recording medium that is detachable from the information processing device 20, such as a USB (Universal Serial Bus) memory. Note that when information is input to the information processing device 20 via the communication device 23 (in other words, when the information processing device 20 acquires information via the communication device 23), the communication device 23 may function as an input device.

[0032] The output device 25 is a device capable of outputting information to the outside of the information processing device 20. The output device 25 may output visual information such as text or images, auditory information such as sound, or tactile information such as vibration, as the information. The output device 25 may include, for example, at least one of a display, a speaker, a printer, and a vibration motor. The output device 25 may be capable of outputting information to a recording medium detachable from the information processing device 20, such as a USB memory. Note that when the information processing device 20 outputs information via the communication device 23, the communication device 23 may function as the output device.

[0033] In this embodiment, the output device 25 includes a wearable device 251. In addition, in this embodiment, the output device 25 may include earphones 252. The arithmetic unit 21, the storage device 22, the communication device 23, and the input device 24 may be housed in a housing of the wearable device 251 serving as the output device 25. In other words, the information processing device 20 may constitute at least a part of the wearable device 251.

[0034] As described above, the wearable device 251 may be AR glasses (in other words, an optical see-through head-mounted display). However, the wearable device 251 may be an immersive head-mounted display or a transparent head-mounted display (for example, a video see-through head-mounted display). The wearable device 251 may also be referred to as a display device.

[0035] The camera CAM may be a camera capable of capturing light in the visible light wavelength range (a so-called visible light camera). In this case, the camera CAM may be used to capture a facial image of the target (i.e., the person to be authenticated who is the subject of biometric authentication). The camera CAM may be a camera capable of capturing light in the near-infrared wavelength range (a so-called near-infrared camera). In this case, the camera CAM may be used to capture an iris image of the target. Alternatively, the authentication device 1000 may be equipped with a visible light camera and a near-infrared camera. The camera CAM may be provided so as to capture an image from a position that overlooks the target, for example.

[0036] The arithmetic device 21 may have the image acquisition unit 211, the display control unit 212, and the matching score acquisition unit 214 as logically realized functional blocks or as physically realized processing circuits. At least one of the image acquisition unit 211, the display control unit 212, and the matching score acquisition unit 214 may be realized in a form in which a logical functional block and a physical processing circuit (i.e., hardware) are mixed. When at least one of the image acquisition unit 211, the display control unit 212, and the matching score acquisition unit 214 is a functional block, at least one of the image acquisition unit 211, the display control unit 212, and the matching score acquisition unit 214 may be realized by the arithmetic device 21 executing a predetermined computer program.

[0037] The arithmetic device 21 may acquire (in other words, read) the predetermined computer program from the storage device 22. The arithmetic device 21 may read the predetermined computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (not shown) included in the information processing device 20. The arithmetic device 21 may acquire (in other words, download or read) the predetermined computer program from a device (not shown) external to the information processing device 20 via the communication device 23. Note that the recording medium for recording the predetermined computer program executed by the arithmetic device 21 may be at least one of an optical disk, a magnetic medium, a magneto-optical disk, a semiconductor memory, and any other medium capable of storing a program.

[0038] The authentication device 1000 includes an authentication unit 10013 .

[0039] The image acquisition unit 211 acquires an image output from the camera CAM (a so-called visible light image) when the camera CAM is a visible light camera.The image acquisition unit 211 acquires an image output from the camera CAM (a so-called near-infrared image) when the camera CAM is a near-infrared camera.

[0040] The image acquisition unit 211 may acquire, as the first biometric information, a facial image including the target's face output from the camera CAM by the camera CAM capturing an image of the target's face. The image acquisition unit 211 may acquire, as the second biometric information, an eye image including the target's eyes output from the camera CAM by the camera CAM capturing an image of the target's eyes. The image acquisition unit 211 transmits the facial image as the first biometric information and the eye image as the second biometric information to the authentication device 1000 via the communication device 23. The "image acquisition unit 211" and the "display control unit 212" are components corresponding to the "biometric information acquisition unit 11" and the "display control unit 12" in the first embodiment described above, respectively.

[0041] The authentication unit 10013 performs multimodal biometric authentication of the target using the face image and eye image acquired by the image acquisition unit 211. Here, the authentication unit 10013 may compare the face image with a pre-registered face image to calculate a face matching score (referred to as a "face matching score") as the first matching score. Alternatively, the authentication unit 10013 may compare features extracted from the face image with pre-registered features to calculate the face matching score as the first matching score. The authentication unit 10013 may generate an iris image by extracting an area corresponding to the target's iris from the eye image. The authentication unit 10013 may compare the iris image with a pre-registered iris image to calculate a iris matching score (referred to as an "iris matching score") as the second matching score. Alternatively, the authentication unit 10013 may calculate the iris matching score as the second matching score by matching the feature amount extracted from the iris image with the feature amount registered in advance.

[0042] The matching score (e.g., face matching score and iris matching score) may mean a value indicating the degree of match between the target biometric information and pre-registered biometric information. The matching score may also mean a value indicating the degree of match between features extracted from the target biometric information and pre-registered features. The matching score may be expressed as a value ranging from 0 to 100, for example.

[0043] The authentication unit 10013 may generate an iris image of the left eye by extracting an area corresponding to the iris of the subject's left eye from the eye image. The authentication unit 10013 may generate an iris image of the right eye by extracting an area corresponding to the iris of the subject's right eye from the eye image. The authentication unit 10013 may calculate an iris matching score for the left eye by comparing the iris image of the left eye with a pre-registered iris image. Alternatively, the authentication unit 10013 may calculate an iris matching score for the left eye by comparing a feature extracted from the iris image of the left eye with a pre-registered feature. The authentication unit 10013 may calculate an iris matching score for the right eye by comparing an iris image of the right eye with a pre-registered iris image. Alternatively, the authentication unit 10013 may calculate an iris matching score for the right eye by comparing a feature extracted from the iris image of the right eye with a pre-registered feature. The authentication unit 10013 may calculate at least one of the iris matching score of the subject's left eye and the iris matching score of the subject's right eye. Hereinafter, in this embodiment, the distinction between the iris of the subject's left eye and the iris of the subject's right eye will be omitted.

[0044] In biometric authentication, if the matching score is equal to or greater than a predetermined threshold, it is determined that the subject (i.e., the person to be authenticated) is a person who has been registered in advance (i.e., a registered person). For example, in the case of multimodal biometric authentication in which multiple modals are used in combination, a matching score is calculated for each modal. In multimodal biometric authentication, if all of the multiple matching scores are equal to or greater than a predetermined threshold, it may be determined that the subject is a registered person.

[0045] The authentication unit 10013 may compare the face matching score with a first threshold value, which is an example of the predetermined threshold value. The authentication unit 10013 may compare the iris matching score with a second threshold value, which is another example of the predetermined threshold value. The authentication unit 10013 may determine that face authentication has been successful if the face matching score is equal to or greater than the first threshold value. The authentication unit 10013 may determine that iris authentication has been successful if the iris matching score is equal to or greater than the second threshold value.

[0046] The authentication device 1000 may transmit the face matching score and the iris matching score to the information processing device 20. The matching score acquisition unit 214 of the information processing device 20 may acquire the face matching score and the iris matching score from the authentication device 1000.

[0047] Note that instead of the face matching score and the iris matching score, the authentication device 1000 may transmit information indicating whether the face matching score is equal to or greater than a first threshold and information indicating whether the iris matching score is equal to or greater than a second threshold to the information processing device 20. In this case, the matching score acquisition unit 214 may acquire information indicating whether the face matching score is equal to or greater than the first threshold and information indicating whether the iris matching score is equal to or greater than the second threshold.

[0048] An information processing method executed by the information processing device 20 will be described with reference to Figures 4 and 5. Figure 4 is a flowchart showing an example of the flow of the information processing method executed by the information processing device 20. Figure 5 is a diagram showing an outline of the information processing method executed by the information processing device 20. Note that the information processing method shown in Figure 4(a) and the information processing method shown in Figure 4(b) may be performed in parallel. Alternatively, the information processing method shown in Figure 4(a) and the information processing method shown in Figure 4(b) may be performed in tandem.

[0049] As shown in FIG. 4A, the matching score acquisition unit 214 acquires, for example, the face matching score of the target TP1 (step S211). The display control unit 212 determines whether the face matching score is lower than a predetermined threshold (step S212). If it is determined that the face matching score is lower than the first threshold (step S212: Yes), the display control unit 212 controls the display on the wearable device 251 so that the face area of ​​the target TP1 is highlighted (step S213). Note that if it is determined in the processing of step S212 that the face matching score is equal to or higher than the first threshold (step S212: No), the operation shown in FIG. 4A may be terminated. FIG. 5A illustrates an example of the field of view of a security guard via the wearable device 251 when the face matching score is lower than the first threshold and the iris matching score is equal to or higher than the second threshold. As illustrated in FIG. 5A, the display on the wearable device 251 may be controlled so that the area HA including the face of the target TP1 is highlighted.

[0050] As shown in FIG. 4B, the matching score acquisition unit 214 acquires, for example, the iris matching score of the target TP1 (step S221). The display control unit 212 determines whether the iris matching score is lower than a second threshold (step S222). If it is determined that the iris matching score is lower than a predetermined threshold (step S222: Yes), the display control unit 212 controls the display so that the eye area of ​​the target TP1 is highlighted on the wearable device 251 (step S223). Note that if it is determined in the processing of step S222 that the iris matching score is equal to or greater than the second threshold (step S222: No), the operation shown in FIG. 4B may be terminated. FIG. 5B illustrates an example of the field of view of a security guard through the wearable device 251 when the iris matching score is lower than the second threshold and the face matching score is equal to or greater than the first threshold. As illustrated in FIG. 5(b), the display on the wearable device 251 may be controlled so that the area HB including the eyes of the subject TP1 is highlighted.

[0051] Furthermore, when the face matching score of the target TP1 is lower than the first threshold and the iris matching score of the target TP1 is lower than the second threshold, the display control unit 212 controls the display so that the face and eyes of the target TP1 are each highlighted on the wearable device 251. Figure 5(c) shows an example of the field of view of a security guard via the wearable device 251 when the face matching score of the target TP1 is lower than a predetermined threshold and the iris matching score of the target TP1 is lower than a predetermined threshold. As illustrated in Figure 5(c), the display on the wearable device 251 may be controlled so that an area HA including the face of the target TP1 is highlighted and an area HB including the eyes of the target TP1 is highlighted. Furthermore, if the face matching score of target TP2, a person different from target TP1, is lower than the first threshold and the iris matching score of target TP2 is equal to or higher than the second threshold, the display control unit 212 controls the display so that the area HC including the face of target TP2 on the wearable device 251 is highlighted, as illustrated in Figure 5 (c).

[0052] In other words, when the matching score is lower than a predetermined threshold, the display control unit 212 controls the display so that the area corresponding to the biometric information (e.g., at least one of a face image and an eye image) used to calculate the matching score in the wearable device 251 is highlighted.

[0053] For example, a security guard may determine that a person in which at least one of the areas including the face and the areas including the eyes is highlighted is a person who requires a visual permission determination. Furthermore, a security guard may determine that a person in which neither the area including the face nor the area including the eyes is highlighted is a person who may be allowed to enter or exit, without performing the visual permission determination. Even if biometric authentication is unsuccessful, if the security guard determines that the person may be allowed to enter or exit based on the visual permission determination, the security guard may allow the person to enter or exit. Furthermore, when a security guard finds a suspicious person, the security guard may check the route taken by the suspicious person in addition to the surveillance camera footage.

[0054] A display device may be provided near the entrance / exit gate of the entry / exit restricted area, allowing the target TP to visually check whether they have been permitted to enter or exit. The display device may display information indicating the target TP who has been permitted to enter or exit by the security guard. The information indicating the person who has been permitted to enter or exit by the security guard may be, for example, a facial image of the target TP. The display device may display at least one of items that are permitted to be brought into the entry / exit restricted area and items that are not permitted to be brought into the entry / exit restricted area. By presenting the items to the target TP using the display device, it is possible to alert the target TP. Furthermore, the wearable device 251 of the information processing device 20 worn by the security guard may also display at least one of items that are permitted to be brought into the entry / exit restricted area and items that are not permitted to be brought into the entry / exit restricted area. [Modification]

[0055] The display control unit 212 may control the output device 25 so that the image shown in Fig. 6 is displayed on the wearable device 251. The image shown in Fig. 6 includes a target TP, an analog meter 2511 indicating the face matching score of the target TP, and an analog meter 2512 indicating the iris matching score of the target TP. In the analog meter 2511, the index value of the needle indicates the face matching score. Similarly, in the analog meter 2512, the index value of the needle indicates the iris matching score. This configuration allows the security guard to intuitively determine whether or not authentication is possible.

[0056] The matching score may be displayed by an image imitating an LED (Light Emitting Diode) array meter, instead of the analog meters 2511 and 2512, for example.

[0057] 6, analog meters 2511 and 2512 are placed near the target TP. Note that the analog meters 2511 and 2512 do not have to be placed near the target TP. In other words, the analog meters 2511 and 2512 may be displayed in any position.

[0058] 6 displays an image of the target TP as seen from the front. However, the wearable device 251 may also display an image of the target TP as seen from the perspective of a security guard (i.e., a person wearing the wearable device 251). If the wearable device 251 is an optical see-through head-mounted display (i.e., AR glasses), only the analog meters 2511 and 2512 may be displayed on the wearable device 251.

[0059] The display control unit 212 is not limited to presenting the numerical value of the matching score, and may present, for example, at least one of characters, symbols, figures, and colors according to the matching score. Furthermore, the matching score may be presented as auditory information in addition to visual information. [Technical Effect]

[0060] The information processing system S according to this disclosure notifies the security guard of the results of biometric authentication in an easy-to-understand manner, thereby reducing the burden on the security guard in managing entry and exit. Furthermore, since the information processing system S and the security guard cooperate to manage entry, if the target is a registered user, the target can be permitted entry after identity verification by the security guard even if biometric authentication fails. [3: Third Embodiment]

[0061] A third embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 7 . Hereinafter, the third embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S3 as an information processing device. Note that, for the third embodiment, descriptions that overlap with the descriptions of the first and second embodiments will be omitted as appropriate. As shown in FIG. 7 , the information processing system S3 according to the third embodiment may include an information processing device 30 and an authentication device 1000.

[0062] As shown in FIG. 7 , the information processing device 30 includes a camera CAM3 in addition to a calculation device 21, a storage device 22, a communication device 23, an input device 24, and an output device 25. The "camera CAM3" may have the same functions as the "camera CAM" in the second embodiment described above. That is, the camera CAM3 may be a camera capable of capturing light in the visible wavelength range (a so-called visible light camera). In this case, the camera CAM3 may be used to capture a facial image of a target (i.e., an authenticated person who is the subject of biometric authentication). The camera CAM3 may be a camera capable of capturing light in the near-infrared wavelength range (a so-called near-infrared camera). In this case, the camera CAM3 may be used to capture an iris image of the target. Alternatively, the information processing device 30 may include a visible light camera and a near-infrared camera. Compared to the camera CAM, the camera CAM3 may be able to capture the target's face and iris from the front. For example, the camera CAM3 may be disposed in a wearable device 251 worn by a security guard. In contrast, the camera CAM in the second embodiment may be located in a position different from the wearable device 251. In this case, for example, a security guard may face the target directly so that the camera CAM3 captures the target's face and iris from the front. In other words, even if authentication using an image captured by the camera CAM fails, it can be said that authentication using an image captured by the camera CAM3 is often successful.

[0063] The arithmetic device 21 may have the image acquisition unit 311, the display control unit 312, the authentication unit 313, and the matching score acquisition unit 314 as logically realized functional blocks or as physically realized processing circuits. At least one of the image acquisition unit 311, the display control unit 312, the authentication unit 313, and the matching score acquisition unit 314 may be realized in a form in which a logical functional block and a physical processing circuit (i.e., hardware) are mixed. When at least one of the image acquisition unit 311, the display control unit 312, the authentication unit 313, and the matching score acquisition unit 314 is a functional block, at least one of the image acquisition unit 311, the display control unit 312, the authentication unit 313, and the matching score acquisition unit 314 may be realized by the arithmetic device 21 executing a predetermined computer program.

[0064] When the camera CAM3 is a visible light camera, the image acquisition unit 311 acquires an image (a so-called visible light image) output from the camera CAM3. When the camera CAM3 is a near-infrared camera, the image acquisition unit 311 acquires an image (a so-called near-infrared image) output from the camera CAM3. The image acquisition unit 311 may acquire, as the first biometric information, a facial image including the target's face output from the camera CAM3 by the camera CAM3 capturing an image of the target's face. The image acquisition unit 311 may acquire, as the second biometric information, an eye image including the target's eyes output from the camera CAM3 by the camera CAM3 capturing an image of the target's eyes. In this manner, the "image acquisition unit 311" may perform the same operation as the "image acquisition unit 211" in the second embodiment described above. In this embodiment, a case will be described in which the first authentication is facial authentication and the second authentication is iris authentication.

[0065] The authentication unit 313 can perform multimodal biometric authentication of the target using the face image and eye image acquired by the image acquisition unit 311. In other words, the authentication unit 313 can perform multimodal biometric authentication that combines face authentication and iris authentication as biometric authentication. In this way, the "authentication unit 313" may perform the same operation as the "authentication unit 10013."

[0066] The display control unit 312 may cause the face image captured by the camera CAM3 to be displayed on the wearable device 251. The display control unit 312 may superimpose a highlight on the face image captured by the camera CAM3. The security guard may make a visual inspection permission determination for the highlighted target TP.

[0067] Although the biometric authentication is performed in the information processing device 30, the biometric authentication may be performed in a device (e.g., authentication device 1000) different from the information processing device 30. The camera CAM3 of the information processing device 30 captures images all the time, and the captured images may be saved. [Modification]

[0068] In addition to the camera CAM3, the information processing device 30 may include another camera corresponding to the camera CAM in the second embodiment. The image acquisition unit 311 may acquire a facial image including the face of the target TP output from the other camera as the first biometric information. The image acquisition unit 311 may acquire an eye image including the eyes of the target TP output from the other camera as the second biometric information. The authentication unit 313 may perform facial authentication of the target TP using the facial image. The authentication unit 313 may perform iris authentication of the target TP using the eye image. For example, if at least one of the facial authentication and iris authentication of the target TP in the authentication unit 313 fails, the image acquisition unit 311 may further acquire at least one of the facial image and eye image of the target TP. In this case, the display control unit 312 controls the display of the wearable device 251 so that the target TP is highlighted, and notifies the security guard that the biometric authentication of the target TP has failed. The security guard may input an image capture command to the camera CAM3 arranged on the wearable device 251 worn by the security guard via the input device 24. The image acquisition unit 311 acquires an image of the target TP from the camera CAM3. That is, the image acquisition unit 311 may acquire an image of the target TP in front of the security guard. The authentication unit 313 may perform biometric authentication using the image of the target TP in front of the security guard.

[0069] For example, if face authentication of the target TP using the face image of the target TP output from the other camera fails, the image acquisition unit 311 may acquire a face image of the face of the target TP standing in front of the security guard, captured by the camera CAM3. In this case, the authentication unit 313 may perform face authentication using the face image acquired by the image acquisition unit 311.

[0070] Furthermore, if the iris authentication of the target TP using the eye image of the target TP output from the other camera fails, the image acquisition unit 311 may acquire an eye image of the target TP's eye in front of the security guard, captured by the camera CAM3. In this case, the authentication unit 313 may perform iris authentication using the eye image acquired by the image acquisition unit 311. [Technical effect]

[0071] The information processing device 30 according to this disclosure includes a camera CAM3 disposed on a wearable device 251 worn by a security guard, and can acquire an image from the same direction as the security guard's view and perform biometric authentication using the image. [4: Fourth Embodiment]

[0072] A fourth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 8 . Hereinafter, the fourth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S4 as the information processing device. Note that, for the fourth embodiment, descriptions that overlap with the descriptions of the first to third embodiments will be omitted as appropriate. As shown in FIG. 8 , the information processing system S4 according to the fourth embodiment may include an information processing device 40 and an authentication device 1000. The information processing system S4 according to the fourth embodiment differs from the first to third embodiments in the operation of the display control unit 412. In this embodiment, a case will be described in which the first authentication is face authentication and the second authentication is iris authentication.

[0073] The display control unit 412 may change the display mode of highlighting the face region as the first location in accordance with the face matching score as the first matching score. The display control unit 412 may also change the display mode of highlighting the eye region as the second location in accordance with the iris matching score as the second matching score.

[0074] The display control unit 412 may control the display using two types of thresholds. The third threshold is a threshold for determining whether authentication has succeeded or failed. The fourth threshold may be a threshold for determining whether authentication is successful and is greater than the third threshold. For example, when the matching score is a value between 0 and 100, the third threshold may be set to 70. The fourth threshold may be set to 90. The third and fourth thresholds may be set arbitrarily. Here, in biometric authentication, for example, a false acceptance rate (FAR) and a false recognition rate (FRR) are often taken into consideration to achieve both authentication accuracy and user convenience. For this reason, the third threshold (i.e., the threshold for determining whether authentication has succeeded or failed) may be set to a value that, for example, allows an unregistered person (i.e., a stranger) to be mistaken for a registered person, but rarely rejects a registered person. The larger the matching score (e.g., face matching score and iris matching score), the lower the false acceptance rate. When the false acceptance rate is small enough to be ignored, it can be said that the success of authentication is reliable. Therefore, the fourth threshold (i.e., the threshold for determining that the success of authentication is reliable) may be set, for example, as the lower limit of the range of matching scores where the false acceptance rate is small enough to be ignored. The third and fourth thresholds may be applied to both the face matching score and the iris matching score.

[0075] The display control unit 412 may control the output device 25 so that, for example, the image shown in Fig. 9(a) is displayed on the wearable device 251. The image shown in Fig. 9(a) includes a target TP, a frame HA relating to the face matching score of the target TP, and a frame HB relating to the iris matching score of the target TP. The display mode of the frame HA (e.g., at least one of the type, thickness, and color of the frame line, and at least one of the color and pattern of the area surrounded by the frame line) may change depending on the face matching score. The display mode of the frame HB may change depending on the iris matching score.

[0076] For example, the display manner of the frame HA may include a first display manner when the face matching score is in a first range (e.g., 0 to 70), a second display manner when the face matching score is in a second range (e.g., 71 to 90) different from the first range, and a third display manner when the face matching score is in a third range (e.g., 91 to 100) different from the first and second ranges. In other words, the face of the target TP may be surrounded by a frame even when the face authentication of the target TP is successful.

[0077] Furthermore, the display modes of the frame HB may include a first display mode when the iris matching score is in a first range (e.g., 0 to 70), a second display mode when the iris matching score is in a second range (e.g., 71 to 90) different from the first range, and a third display mode when the iris matching score is in a third range (e.g., 91 to 100) different from the first and second ranges. In other words, the eyes of the target TP may be surrounded by a frame even when the iris authentication of the target TP is successful.

[0078] The display control unit 412 changes the display mode of the highlighting of the face region depending on whether the face matching score is lower than a third threshold, whether the face matching score is higher than the third threshold and lower than a fourth threshold, or whether the face matching score is higher than the fourth threshold. The display control unit 412 may control the display so that the face of the target TP is highlighted in a first display mode when the face matching score is lower than the third threshold. The display control unit 412 may control the display so that the face of the target TP is highlighted in a second display mode when the face matching score is higher than the third threshold and lower than the fourth threshold. The display control unit 412 may control the display so that the face of the target TP is highlighted in a third display mode when the face matching score is higher than the fourth threshold. Alternatively, the display control unit 412 may control the display so that the face of the target TP is not highlighted when the face matching score is higher than the fourth threshold.

[0079] For example, the display control unit 412 may surround the face region of the target TP with a red frame when the face matching score is lower than the third threshold. For example, the display control unit 412 may surround the face region of the target TP with a yellow frame when the face matching score is higher than the third threshold and lower than the fourth threshold. For example, the display control unit 412 may surround the face region of the target TP with a blue frame when the face matching score is higher than the fourth threshold.

[0080] The display control unit 412 changes the display mode of the highlighting of the eye region depending on whether the iris matching score is lower than the third threshold, whether the iris matching score is higher than the third threshold and lower than the fourth threshold, or whether the iris matching score is higher than the fourth threshold. The display control unit 412 may control the display so that the eyes of the target TP are highlighted in a first display mode when the iris matching score is lower than the third threshold. The display control unit 412 may control the display so that the eyes of the target TP are highlighted in a second display mode when the iris matching score is higher than the third threshold and lower than the fourth threshold. The display control unit 412 may control the display so that the eyes of the target TP are highlighted in a third display mode when the iris matching score is higher than the fourth threshold. Alternatively, the display control unit 412 may control the display so that the eyes of the target TP are not highlighted when the iris matching score is higher than the fourth threshold.

[0081] For example, the display control unit 412 may surround the eye region of the target TP with a red frame when the iris matching score is lower than the third threshold. For example, the display control unit 412 may surround the eye region of the target TP with a yellow frame when the iris matching score is higher than the third threshold and lower than the fourth threshold. For example, the display control unit 412 may surround the eye region of the target TP with a blue frame when the iris matching score is higher than the fourth threshold.

[0082] 9(b) shows an example of the field of view of a security guard via the wearable device 251 when the face matching score of target TP1 is lower than the third threshold, the iris score of target TP1 is higher than the third threshold and lower than the fourth threshold, and the face matching score of target TP2 is higher than the fourth threshold. As illustrated in FIG. 9(b), the display on the wearable device 251 may be controlled so that a frame H1 (solid-line frame) of the first display mode is superimposed on the face region of target TP1, a frame H2 (dashed-line frame) of the second display mode is superimposed on the eye region of target TP1, and a frame H3 (dotted-line frame) of the third display mode is superimposed on the face region of target TP2.

[0083] For example, if the target TP's face matching score is higher than the fourth threshold and the target TP's iris matching score is higher than the fourth threshold, the display control unit 412 may surround the target TP's face region with a blue frame and the target TP's eye region with a blue frame. If the target TP's face matching score is lower than the third threshold and the target TP's iris matching score is higher than the fourth threshold, the display control unit 412 may surround the target TP's face region with a red frame and omit the blue frame for the target TP's eye region. Furthermore, if the target TP's face matching score is higher than the third threshold, the target TP's face matching score is lower than the fourth threshold, and the target TP's iris matching score is higher than the fourth threshold, the display control unit 412 may surround the target TP's face region with a yellow frame and omit the blue frame for the target TP's eye region. Furthermore, if the face matching score of the target TP is higher than the fourth threshold and the iris matching score of the target TP is lower than the third threshold, the display control unit 412 may surround the eye region of the target TP with a red frame and omit the blue frame of the face region of the target TP. Furthermore, if the face matching score of the target TP is higher than the fourth threshold and the iris matching score of the target TP is higher than the third threshold and the iris matching score of the target TP is lower than the fourth threshold, the display control unit 412 may surround the eye region of the target TP with a yellow frame and omit the blue frame of the face region of the target TP.

[0084] In this case, the security guard may determine that a person whose face area is surrounded by a blue frame is a person for whom the security guard's visual inspection permission decision can be omitted. In other words, the security guard may allow a person whose face area is surrounded by a blue frame to enter or exit without calling out to them. In this way, a person whose identity is confirmed through successful face authentication and iris authentication and who is allowed to enter or exit without being called out to by a security guard is referred to as a "person for whom verification can be omitted." Furthermore, the registration information may include information indicating treatment at the time of entry or exit. If a person for whom verification can be omitted is a person who should be given special treatment, the person may be given the special treatment.

[0085] A person for whom face authentication and iris authentication are successful but at least one of the face matching score and the iris matching score is lower than the fourth threshold is referred to as a "person to be identified." A person to be identified may be a person for whom a visual inspection permission determination is required by a security guard. In other words, a security guard may determine a person to be identified as a person for whom a visual inspection permission determination is required. A security guard may determine that a person who is not surrounded by a red frame but is surrounded by a yellow frame is a person to be identified.

[0086] A person for whom at least one of face authentication and iris authentication has failed is referred to as a "re-authentication person." A re-authentication person may be a person who needs to undergo biometric authentication again. The re-authentication may be authentication using an image captured by the camera CAM3 provided in the information processing device 40. The security guard may acquire biometric information from the area surrounded by a red frame.

[0087] Furthermore, the re-authentication person may be a person who requires a visual permission determination by a security guard. The visual permission determination for the re-authentication person may be stricter than the visual permission determination for the person to be confirmed. For example, the visual permission determination for the re-authentication person may be performed by visually matching the face with a registered face image presented by the information processing system S4. In contrast, the visual permission determination for the re-authentication person may be performed by visually matching the face with a registered face image presented by the information processing system S4, as well as by presenting an identification card and visually matching the face with a facial photograph included on the identification card. The security guard may perform biometric authentication again and then perform a visual permission determination for the re-authentication person.

[0088] Note that when the authentication score is equal to the third threshold, the highlighting may be performed in the first display mode. In this case, the display control unit 412 may perform the highlighting in the first display mode when the authentication score is equal to or less than the third threshold. Alternatively, when the authentication score is equal to the third threshold, the highlighting may be performed in the second display mode. In this case, the display control unit 412 may perform the highlighting in the second display mode when the authentication score is equal to or greater than the third threshold.

[0089] As described above, the display control unit 412 may notify which person the target TP is by color-coding the frame. Furthermore, the display control unit 412 may notify by displaying "OK" near a person for whom confirmation may be omitted, "?" near a person to be confirmed, or "NG" near a person to be re-authenticated.

[0090] The information processing device 40 may be equipped with earphones. In this case, the information processing device 40 may notify information by voice. If the security guard carries a device with a vibration function (such as a watch, smartphone, or walkie-talkie) instead of or in addition to the AR glasses, the display control unit 412 may control the vibration function to notify. In this case, the display control unit 412 may express the distinction between a person who can be omitted from confirmation, a person who needs to be confirmed, and a person who needs to be re-authenticated by changing the vibration cycle. Alternatively, the device that vibrates may be changed depending on whether the person is a person who can be omitted from confirmation, a person who needs to be confirmed, or a person who needs to be re-authenticated. [Technical Effect]

[0091] The information processing device 40 according to this disclosure displays, for example, an image shown in FIG. 9 on the wearable device 251, allowing the security guard to intuitively determine what action to take with respect to the target TP. [5: Fifth Embodiment]

[0092] A fifth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 10 . Hereinafter, the fifth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S5 as an information processing device. Note that, for the fifth embodiment, descriptions that overlap with the descriptions of the first to fourth embodiments will be omitted as appropriate. As shown in FIG. 10 , the information processing system S5 according to the fifth embodiment may include an information processing device 50 and an authentication device 1000.

[0093] In the second to fourth embodiments, the first location is the face and the second location is the iris, but the face and the iris are merely an example of a combination of the first location and the second location. The first location and the second location may be different parts of the subject.

[0094] For example, the biometric information related to the first location and the biometric information related to the second location may be the same type of biometric information. For example, the first location may be the iris of the right eye, and the second location may be the iris of the left eye. That is, both the first authentication and the second authentication may be iris authentication. In this case, the image acquisition unit 511 acquires a right-eye image including the subject's right eye and a left-eye image including the subject's left eye. Furthermore, the authentication unit 513 performs iris authentication using the right-eye image and iris authentication using the left-eye image.

[0095] For example, the first location may be a fingerprint of the right thumb and the second location may be a fingerprint of the right index finger. Alternatively, the first location may be a fingerprint of the right thumb and the second location may be a fingerprint of the left thumb, i.e., both the first authentication and the second authentication may be fingerprint authentication.

[0096] A face includes multiple characteristic features such as an outline, nose, eyes, ears, and mouth. Biometric information related to a face may include information acquired from the entire face, information acquired from the outline of the face, information acquired from the nose area, information acquired from the eye area, information acquired from the ear area, information acquired from the mouth area, etc. In other words, information acquired from each area of ​​the face may be treated as one piece of biometric information.

[0097] For example, the display control unit 512 may control the display so that the nose region of the target TP is highlighted in accordance with the result of comparing the matching score based on a comparison of feature amounts extracted from the nose region of the target TP with a threshold value. Also, the display control unit 512 may control the display so that the eye region of the target TP is highlighted in accordance with the result of comparing the matching score based on a comparison of feature amounts extracted from the eye region of the target TP with a threshold value. [Technical Effect]

[0098] The information processing system S5 according to this disclosure can employ an appropriate combination of the first biometric information and the second biometric information.

[0099] A sixth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 11 . Hereinafter, the sixth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S6 as the information processing device. Note that, for the sixth embodiment, descriptions that overlap with the descriptions of the first to fifth embodiments will be omitted as appropriate. As shown in FIG. 11 , the information processing system S6 according to the sixth embodiment may include an information processing device 60 and an authentication device 1000. In this embodiment, a case will be described in which the first authentication is face authentication and the second authentication is iris authentication.

[0100] As shown in FIG. 11 , the arithmetic unit 21 included in the information processing device 60 may include, as logically realized functional blocks or physically realized processing circuits, an image acquisition unit 311, a display control unit 312, an authentication unit 313, and a matching score acquisition unit 314, as well as a fusion score calculation unit 615. The fusion score calculation unit 615 calculates the fusion score based on the first matching score and the second matching score. For example, the fusion score calculation unit 615 may calculate the fusion score by assigning different weights to the first matching score and the second matching score (for example, by calculating a weighted average of the first matching score and the second matching score). In the sixth embodiment, the authentication unit 313 may perform biometric authentication of the target using the fusion score. The display control unit 612 may determine the highlighting mode depending on the fusion score.

[0101] For example, when the fusion score is higher than the third threshold and lower than the fourth threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in the second display mode. For example, the fusion score may be a weighted average of the first matching score and the second matching score. For example, the first matching score is lower than the fourth threshold. For example, the second matching score is higher than the fourth threshold. The weight of the first matching score is greater than the weight of the second matching score. In this case, the target part (corresponding to the first location) related to the biometric information used to calculate the first matching score may be more suitable for biometric authentication of the target than the target part (corresponding to the second location) related to the biometric information used to calculate the second matching score. In such a situation, when the fusion score is higher than the third threshold and lower than the fourth threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in the second display mode, regardless of the values ​​of the first matching score and the second matching score. The third threshold and the fourth threshold correspond to the third threshold and the fourth threshold in the fourth embodiment, respectively. Also, the second display mode of the highlighting corresponds to the second display mode of the highlighting in the fourth embodiment.

[0102] Furthermore, when the fusion score is lower than a third threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in a first display manner. For example, the fusion score may be a weighted average of the first matching score and the second matching score. For example, the first matching score is lower than the third threshold. For example, the second matching score is higher than the third threshold. The weight of the first matching score is greater than the weight of the second matching score. In such a situation, when the fusion score is lower than the third threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in a first display manner, regardless of the values ​​of the first matching score and the second matching score. Note that the first display manner of highlighting corresponds to the first display manner of highlighting in the fourth embodiment.

[0103] Furthermore, when the fusion score is higher than a fourth threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in a third display manner. For example, the fusion score may be a weighted average of the first matching score and the second matching score. For example, the first matching score is higher than the fourth threshold. For example, the second matching score is lower than the fourth threshold. The weight of the first matching score is greater than the weight of the second matching score. In such a situation, when the fusion score is higher than the fourth threshold, the display control unit 612 may control the display so that each of the first location and the second location is highlighted in a third display manner, regardless of the values ​​of the first matching score and the second matching score. Note that the third display manner of highlighting corresponds to the third display manner of highlighting in the fourth embodiment. [Technical Effects]

[0104] The information processing device 60 according to this disclosure can give priority to parts that are more suitable for authentication by weighting and fusing the scores. [7: Seventh Embodiment]

[0105] A seventh embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 12 . Hereinafter, the seventh embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S7 as the information processing device. Note that, for the seventh embodiment, descriptions that overlap with the descriptions of the first to sixth embodiments will be omitted as appropriate. As shown in FIG. 12 , the information processing system S7 according to the seventh embodiment may include an information processing device 70 and an authentication device 1000. The information processing system S7 according to the seventh embodiment differs from the first to sixth embodiments in the operation of the display control unit 712. The display control unit 712 changes the display mode of the highlighting of the first location in accordance with the first matching score. Furthermore, the display control unit 712 changes the display mode of the highlighting of the second location in accordance with the second matching score.

[0106] When there are multiple locations to be highlighted, the display control unit 712 changes the display mode of the highlighting for each of the multiple locations. When there are multiple locations to be highlighted, the display control unit 712 prioritizes each of the multiple locations. The display control unit 712 changes the display mode of the highlighting for each of the multiple locations to express the priority. The display control unit 712 outputs information that enables the security guard to understand that the higher the importance of an item, the higher the priority of the item to be handled.

[0107] The display control unit 712 may assign priorities to each of the multiple locations based on the matching scores used for authentication. The display control unit 712 controls the display so that the priorities are clear. For example, if the "re-authentication person" should be processed with priority over the "identification-required person," the display control unit 712 may make the highlighting in the first display mode more noticeable than the highlighting in the second display mode. Conversely, if the "identification-required person" should be processed with priority over the "re-authentication person," the display control unit 712 may make the highlighting in the second display mode more noticeable than the highlighting in the first display mode. For example, the display control unit 712 may darken the color of the location that should be processed with priority. Furthermore, the display control unit 712 may darken the border of the location that should be processed with priority.

[0108] The display control unit 712 may determine the order in which the processes should be prioritized, or may display a number near each of the multiple locations. Furthermore, the display control unit 712 may assign a level of importance to each of the multiple locations. [Technical effect]

[0109] If there is too much information displayed on the wearable device 251, it is difficult for the security guard to check the information. The information processing device 70 according to this disclosure can notify the security guard which target TP should be prioritized for verification. The information processing device 70 can support efficient identity verification even when there are many target TPs. [8: Eighth Embodiment]

[0110] An eighth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 13 . Hereinafter, the eighth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S8 as an information processing device. Note that, for the eighth embodiment, descriptions that overlap with the descriptions of the first to seventh embodiments will be omitted as appropriate. As shown in FIG. 13 , the information processing system S8 according to the eighth embodiment may include an information processing device 80 and an authentication device 1000.

[0111] 13 , the arithmetic device 21 included in the information processing device 80 may include, as logically realized functional blocks or as physically realized processing circuits, an image acquisition unit 311, a display control unit 812, an authentication unit 313, and a matching score acquisition unit 314, as well as a reception unit 816. The reception unit 816 receives information indicating that the highlighted portion has been checked from a user using the information processing device 80. The user using the information processing device 80 may be, for example, a security guard. The security guard may perform an input operation on the input device 24 to input information indicating that processing such as a visual inspection permission determination has been completed for the highlighted portion.

[0112] For example, the receiving unit 816 may receive input by detecting eye movement. The receiving unit 816 may receive, from the results of tracking the gaze of the security guard, that the highlighted portion has been checked. For example, if the results of tracking the gaze of the security guard indicate that the security guard has gazed at the highlighted portion for a predetermined period of time, the receiving unit 816 may receive that the highlighted portion has been checked.

[0113] The security guard may also input information by voice indicating that the highlighted portion has been confirmed. Furthermore, if the security guard visually determines whether entry or exit is permitted, the security guard may press a predetermined button to indicate that the notified information has been confirmed.

[0114] When the display control unit 812 receives information indicating that the highlighted portion has been checked, the display control unit 812 performs control to remove the highlighting of the highlighted portion. [Technical effect]

[0115] If there is too much information displayed on the wearable device 251, it is difficult for the security guard to check the information. The information processing device 80 according to this disclosure deletes information that has already been checked, making it easier for the security guard to check the information. [9: Ninth Embodiment]

[0116] A ninth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 14 . Hereinafter, the ninth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S9 as an information processing device. Note that, for the ninth embodiment, descriptions that overlap with the descriptions of the first to eighth embodiments will be omitted as appropriate. As shown in FIG. 14 , the information processing system S9 according to the ninth embodiment may include an information processing device 90 and an authentication device 1000. In this embodiment, a case will be described in which the first biometric information is a face image and the second biometric information is an eye image.

[0117] The display control unit 912 displays the registered face image of the target TP when face authentication of the target TP fails. The display control unit 912 may display the registered face image of the target TP when identification information of the target TP is received. In this case, the security guard may determine whether visual inspection is permitted by comparing the registered face image of the target TP with the target TP.

[0118] Furthermore, if iris authentication of the target TP fails, the display control unit 912 may display the registered eye image of the target TP. If the display control unit 912 receives the identification information of the target TP, the display control unit 912 may display the registered eye image of the target TP. In this case, the security guard may determine whether visual inspection is permitted by comparing the registered eye image of the target TP with the target TP.

[0119] If biometric authentication fails, there may be cases where the registration information has not been registered in the first place. For example, there may be cases where the target TP has received permission to enter through a prior application, but has forgotten to register their biometric information. The security guard may check whether the registration information corresponding to the target TP has been registered. If the registration information corresponding to the target TP has not been registered, the security guard may encourage the target TP to re-register their registration information.

[0120] Furthermore, if the quality of the registration information is poor, the matching score will be low and authentication may not be successful. The security guard may check whether registration information of appropriate quality has been registered. If the security guard determines that authentication has failed because the quality of the registered image is low, he or she may encourage the person to re-register the registration information. [Technical Effect]

[0121] Failure of biometric authentication is often caused by (1) the fact that registration information has not been registered in the first place, or (2) the fact that the difference between the currently acquired information and the registration information has become large due to aging or the like. The information processing device 90 according to this disclosure can identify the cause of the failure of biometric authentication and deal with the failure of biometric authentication. [10: Tenth Embodiment]

[0122] A tenth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to FIG. 15 . Hereinafter, the tenth embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system S10 as an information processing device. Note that, for the tenth embodiment, descriptions that overlap with the descriptions of the first to ninth embodiments will be omitted as appropriate. As shown in FIG. 15 , the information processing system S10 according to the tenth embodiment may include an information processing device 100 and an authentication device 1000.

[0123] As shown in FIG. 15 , the arithmetic device 21 included in the information processing device 100 may include, as a logically realized functional block or a physically realized processing circuit, an image acquisition unit 311, a display control unit 1012, an authentication unit 313, and a matching score acquisition unit 314, as well as a possession determination unit 1017. The possession determination unit 1017 detects possessions carried by a subject who has been permitted to enter or exit from the acquired image. The image may be an image captured by a camera CAM (see the second embodiment) or may be an image captured by a camera CAM3. Note that, for example, the camera CAM3 may be disposed in a wearable device 251 worn by a security guard. In contrast, the camera CAM in the second embodiment may be disposed in a position different from the wearable device 251.

[0124] After allowing the target TP to enter and exit, the security guard may check the TP's belongings. The display control unit 1012 may control the display so that at least one of the items permitted to be brought into the entry / exit restricted area and the items not permitted to be brought into the entry / exit restricted area is highlighted.

[0125] The possession determination unit 1017 determines whether the possession is permitted based on the image of the possession. If the possession is not permitted, the display control unit 1012 may control the display so as to highlight the possession.

[0126] The items that can be brought into the entry / exit restricted area differ depending on the attributes of the target TP identified by biometric authentication and the classification of the entry / exit restricted area. The display control unit 1012 displays information on the wearable device 251 regarding at least one of the items that the target TP is permitted to bring into the entry / exit restricted area and the items that the target TP is not permitted to bring into the entry / exit restricted area, based on information indicating the attributes of the target TP identified by biometric authentication and information indicating the classification of the entry / exit restricted area.

[0127] The display control unit 1012 may notify the security guard of the approval status of the application to take out an item. The display control unit 1012 may cause the wearable device 251 to display information indicating the approval status of the application to take out an item. The display control unit 1012 may cause the wearable device 251 to display information indicating the person whose application to take out an item has currently been approved. The display control unit 1012 may cause the wearable device 251 to display information indicating the approval status of the application to take out an item for a person who has been determined to be permitted to enter and exit. The display control unit 1012 may cause the wearable device 251 to display information indicating the status of the application to take out an item from a target TP for which entry and exit is permitted, such as whether an application for approval has not been made, approval has not been obtained, or approval has been obtained. The display control unit 1012 may output an alert when an application for approval has not been made or approval has not been obtained for the application to take out an item from a target TP for which entry and exit is permitted.

[0128] Furthermore, an item that is restricted in at least one of bringing into and taking out of the entry / exit restricted area, for example, an item that requires permission to take out, may be equipped with an item recognition mechanism such as an identification tag. An identification tag is a tag that allows identification information to be read without contact. When an item with an identification tag is detected among the belongings of a target TP who has been permitted to enter or exit, the display control unit 1012 may cause the wearable device 251 to display information indicating the item with the identification tag. The security guard may confirm whether the target TP is a person permitted to bring in or take out the relevant item into or out of the entry / exit restricted area. [Technical Effect]

[0129] The information processing device 100 according to this disclosure can assist security guards in efficiently checking items.

[0130] An eleventh embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described with reference to Fig. 16. Hereinafter, the eleventh embodiment relating to an information processing device, an information processing system, an information processing method, and a recording medium will be described using an information processing system 500. Note that, for the eleventh embodiment, descriptions that overlap with the descriptions of the first to tenth embodiments will be omitted as appropriate.

[0131] The information processing system 500 performs the above-described information processing method. As shown in FIG. 16 , the information processing system 500 includes a wearable device 5001 and an information processing device 5002. The information processing system 500 may perform the above-described information processing method by executing a computer program. For example, the operation of step S11 shown in FIG. 2 (acquiring first biometric information and second biometric information of the target) may be performed by the wearable device 5001, and the operation of step S12 shown in FIG. 2 (controlling the display so that at least one of the first authentication of the target using the first biometric information and the second authentication of the target using the second biometric information is highlighted when at least one of the first authentication of the target using the first biometric information and the second authentication of the target using the second biometric information fails) may be performed by the information processing device 5002. Alternatively, the operation of step S11 shown in FIG. 2 may be performed by the information processing device 5002, and the operation of step S12 shown in FIG. 2 may be performed by the wearable device 5001. 4 may be performed by the wearable device 5001, and the remaining steps may be performed by the information processing device 5002. [12: Supplementary Note] Some or all of the above embodiments may also be described as, but are not limited to, the following supplementary notes. [Supplementary Note 1] An information processing device comprising: a biometric information acquisition means for acquiring first biometric information and second biometric information of a target; and a control means for controlling display so that, when a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, when a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and when both the first authentication and the second authentication fail, the first location and the second location are highlighted.[Supplementary Note 2] The information processing device according to Supplementary Note 1, further comprising: a matching score acquisition means for acquiring a first matching score calculated in the first authentication of the target and a second matching score calculated in the second authentication of the target; wherein the control means controls the display so that the first location is highlighted when the first matching score is lower than a first predetermined value, the second location is highlighted when the second matching score is lower than a second predetermined value, and both the first location and the second location are highlighted when the first matching score is lower than the first predetermined value and the second matching score is lower than the second predetermined value. [Supplementary Note 3] The information processing device according to Supplementary Note 2, comprising a wearable device, wherein the control means controls a display so that the first location on the wearable device is highlighted when the first matching score is lower than a first predetermined value, controls a display so that the second location on the wearable device is highlighted when the second matching score is lower than a second predetermined value, and controls a display so that each of the first location and the second location on the wearable device is highlighted when the first matching score is lower than the first predetermined value and the second matching score is lower than a second predetermined value. [Supplementary Note 4] The information processing device according to Supplementary Note 2, wherein the control means changes a display mode of highlighting the first location in accordance with the first matching score, and changes a display mode of highlighting the second location in accordance with the second matching score. [Supplementary Note 5] The information processing device described in Supplementary Note 2, wherein the control means changes the display mode of the highlighting of the first location when the first matching score is lower than the first predetermined value, when the first matching score is higher than the first predetermined value and lower than a third predetermined value that is higher than the first predetermined value, and when the first matching score is higher than the third predetermined value; and changes the display mode of the highlighting of the second location when the second matching score is lower than the second predetermined value, when the second matching score is higher than the second predetermined value and lower than a fourth predetermined value that is higher than the second predetermined value, and when the second matching score is higher than the fourth predetermined value.[Supplementary Note 6] The information processing device of Supplementary Note 1, wherein the biometric information acquisition means acquires the first biometric information and the second biometric information from an imaging means that images at least one of the first location and the second location. [Supplementary Note 7] The information processing device of Supplementary Note 2, wherein the control means displays on a display device an image in which an indicated value of a needle of an analog meter indicates at least one of the first matching score and the second matching score. [Supplementary Note 8] The information processing device of Supplementary Note 2, further comprising: a calculation means for calculating a fusion score based on the first matching score and the second matching score, wherein the control means controls display so that each of the first location and the second location is highlighted when the fusion score is lower than a fifth predetermined value. [Supplementary Note 9] The information processing device of Supplementary Note 8, wherein the calculation means calculates the fusion score by assigning different weights to the first matching score and the second matching score. [Supplementary Note 10] The information processing device of claim 2, wherein the control means changes a display mode of highlighting each of the multiple locations when there are multiple locations to be highlighted. [Supplementary Note 11] The information processing device according to Supplementary Note 10, wherein the control means assigns a priority to each of the plurality of locations and changes a display mode of highlighting each of the plurality of locations so as to express the priority. [Supplementary Note 12] The information processing device according to Supplementary Note 1, wherein, when information indicating that the highlighted locations have been confirmed has been received from a person using the information processing device, the control means controls to remove the highlighting of the highlighted locations. [Supplementary Note 13] The information processing device according to Supplementary Note 1, wherein, when the first authentication of the target fails, the control means displays first registered information registered with the target, and when the second authentication of the target fails, the control means displays second registered information registered with the target. [Supplementary Note 14] The information processing device according to Supplementary Note 1, wherein, when identification information of the target has been received, the control means displays at least one of the first registered information registered with the target and the second registered information registered with the target.[Supplementary Note 15] The information processing device according to Supplementary Note 1, wherein the control means determines whether the possessions of the subject are permitted items based on a possession image obtained by capturing an image of the possessions held by the subject, and controls the display to highlight the possessions if the possessions are not permitted items. [Supplementary Note 16] The information processing device according to Supplementary Note 4, wherein the biometric information acquisition means acquires at least one of the first biometric information and the second biometric information when at least one of the first authentication of the subject and the second authentication of the subject fails. [Supplementary Note 17] The information processing device according to Supplementary Note 12, wherein the control means prioritizes each of the plurality of locations based on the first matching score or the second matching score. [Supplementary Note 18] The information processing device according to Supplementary Note 1, wherein the first location and the second location are different parts of the subject. [Supplementary Note 19] An information processing method comprising: acquiring first biometric information and second biometric information of a target; and controlling a display so that, if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted; if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted; and if both the first authentication and the second authentication fail, each of the first location and the second location is highlighted. [Supplementary Note 20] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method comprising: acquiring first biometric information and second biometric information of a target; and controlling a display so that, if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted; if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted; and if both the first authentication and the second authentication fail, each of the first location and the second location is highlighted.[Supplementary Note 21] An information processing system including a wearable device and an information processing device, acquiring first biometric information and second biometric information of a target, and controlling display such that, when a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, when a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and when both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

[0132] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 18, which are dependent on Supplementary Notes 1, and Supplementary Notes 19 to 21, may be dependent in the same manner as Supplementary Notes 2 to 18. Furthermore, not limited to Supplementary Notes 1, 19, 20, and 21, some or all of the configurations described as Supplements may be dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0133] This disclosure may be modified as appropriate within the scope that does not contradict the gist or idea of ​​the invention that can be read from the claims and the entire specification, and information processing devices, information processing systems, information processing methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure.

[0134] 10, 20, 30, 40, 50, 60, 70, 80, 90, 100, 5002 Information processing device 11 Biometric information acquisition unit 12, 212, 312, 412, 612, 712, 812, 912, 1012 Display control unit S, S3, S4, S5, S6, S7, S8, S9, S10, 500 Information processing system 1000 Authentication device 10011, 311, 511 Image acquisition unit 10013, 313, 513, 913 Authentication unit 214, 314 Matching score acquisition unit 251, 5001 Wearable device 615 Fusion score calculation unit 816 Reception unit 1017 Possession determination unit

Claims

1. An information processing device comprising: a biometric information acquisition means for acquiring first biometric information and second biometric information of a target; and a control means for controlling display so that, if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted; if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted; and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

2. An information processing device as described in claim 1, comprising a matching score acquisition means for acquiring a first matching score calculated in the first authentication of the target and a second matching score calculated in the second authentication of the target, wherein the control means controls the display so that the first location is highlighted when the first matching score is lower than a first predetermined value, the second location is highlighted when the second matching score is lower than a second predetermined value, and both the first location and the second location are highlighted when the first matching score is lower than the first predetermined value and the second matching score is lower than the second predetermined value.

3. An information processing device as described in claim 2, comprising a wearable device, wherein the control means controls the display so that the first location on the wearable device is highlighted when the first matching score is lower than a first predetermined value, controls the display so that the second location on the wearable device is highlighted when the second matching score is lower than a second predetermined value, and controls the display so that the first location and the second location on the wearable device are each highlighted when the first matching score is lower than the first predetermined value and the second matching score is lower than the second predetermined value.

4. The information processing device according to claim 2, wherein the control means changes the display mode of the highlighting of the first location according to the first matching score, and changes the display mode of the highlighting of the second location according to the second matching score.

5. The information processing device according to claim 2, wherein the control means changes the display mode of the highlighting of the first location when the first matching score is lower than the first predetermined value, when the first matching score is higher than the first predetermined value and lower than a third predetermined value that is higher than the first predetermined value, and when the first matching score is higher than the third predetermined value; and changes the display mode of the highlighting of the second location when the second matching score is lower than the second predetermined value, when the second matching score is higher than the second predetermined value and lower than a fourth predetermined value that is higher than the second predetermined value, and when the second matching score is higher than the fourth predetermined value.

6. The information processing device according to claim 1, wherein the biometric information acquisition means acquires at least one of the first biometric information and the second biometric information from an imaging means that images at least one of the first location and the second location.

7. The information processing device according to claim 2, wherein the control means displays an image on a display device in which the indicated value of the needle of an analog meter indicates at least one of the first matching score and the second matching score.

8. An information processing device as described in claim 2, further comprising a calculation means for calculating a fusion score based on the first matching score and the second matching score, and wherein the control means controls the display so that each of the first location and the second location is highlighted when the fusion score is lower than a fifth predetermined value.

9. The information processing device according to claim 8, wherein the calculation means calculates the fusion score by assigning different weights to the first matching score and the second matching score.

10. The information processing device according to claim 2, wherein, when there are multiple locations to be highlighted, the control means changes the highlighting display mode for each of the multiple locations.

11. The information processing device according to claim 10, wherein said control means assigns a priority to each of said plurality of locations and changes the display mode of the highlighting of each of said plurality of locations so as to express said priority.

12. The information processing device according to claim 1, wherein when information indicating that the highlighted portion has been confirmed has been received from a person using the information processing device, the control means controls the device to remove the highlighting of the highlighted portion.

13. The information processing device according to claim 1, wherein the control means displays the first registration information registered for the target when the first authentication of the target fails, and displays the second registration information registered for the target when the second authentication of the target fails.

14. The information processing device according to claim 1, wherein the control means, when receiving identification information of the target, displays at least one of the first registration information in which the target is registered and the second registration information in which the target is registered.

15. The information processing device according to claim 1, wherein the control means determines whether the possessions of the subject are permitted items based on an image of the possessions taken from the possessions of the subject, and controls the display to highlight the possessions if the possessions are not permitted items.

16. The information processing device according to claim 4, wherein the biometric information acquisition means acquires at least one of the first biometric information and the second biometric information when at least one of the first authentication of the subject and the second authentication of the subject fails.

17. The information processing device according to claim 12, wherein the control means assigns a priority to each of the plurality of locations based on the first matching score or the second matching score.

18. An information processing system comprising a wearable device and an information processing device, which acquires first biometric information and second biometric information of a target, and controls display so that if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

19. An information processing method comprising: acquiring first biometric information and second biometric information of a target; and controlling display so that, if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted; if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted; and if both the first authentication and the second authentication fail, the first location and the second location are each highlighted.

20. A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method, which comprises acquiring first biometric information and second biometric information of a target, and controlling display so that if a first authentication of the target using the first biometric information fails, a first location related to the first biometric information of the target is highlighted, if a second authentication of the target using the second biometric information fails, a second location related to the second biometric information of the target is highlighted, and if both the first authentication and the second authentication fail, the first location and the second location are highlighted.

Images